Vehicle control method and device, vehicle and storage medium

Abstract

The present application relates to a kind of vehicle control method, device, vehicle and storage medium, comprising, when vehicle is in parking state, obtain user driving habit information and the video frame sequence containing face, face object detection tracking and non-face object detection are carried out to video frame sequence, obtain the face feature information and trajectory distance information of face object, the non-face feature information of non-face object;In trajectory distance information is in the preset approach distance range, and there is at least one face feature information and target face feature information matching condition, based on face feature information, user driving habit information and non-face feature information corresponding control instruction is generated to realize the automatic opening control of car door and / or trunk.No need in the case where user carries vehicle digital key, and need user manual opening control operation to the car door, trunk of vehicle, the present application can realize the automatic opening control of vehicle, greatly improve the user driving experience.

Description

Technical Field

[0001] This invention relates to the field of automotive control technology, specifically to a vehicle control method, device, vehicle, and storage medium. Background Technology

[0002] In recent years, due to the increasing intelligence of equipment and the diversification of information, users' requirements for vehicles have gone beyond basic functions such as safety, speed, and energy efficiency. Vehicle appearance, comfort, personalization, and human-vehicle interaction are now attracting more and more attention from car owners. Among these, the vehicle welcome function, as a feature that can enhance the user experience, is becoming increasingly popular. However, existing vehicle welcome functions are relatively simple, requiring users to carry the vehicle's digital key and manually control the opening and closing of the doors and trunk, significantly reducing the user experience. Summary of the Invention

[0003] The purpose of this invention is to provide a vehicle control method, device, vehicle, and storage medium that allows users to automatically open and control the vehicle doors and / or trunk without a digital key or manual operation when the vehicle is parked, greatly improving the user experience.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] A vehicle control method, wherein the method is applied in a vehicle control center, the method comprising:

[0006] When the vehicle is parked, user driving habit information and video frame sequences containing faces are acquired; among them, user driving habit information is the result of user habit control operation obtained by counting the most control operations performed by the user on the vehicle within a preset statistical time period.

[0007] Face object detection and tracking and non-face object detection are performed on video frame sequences to obtain face feature information of face objects, trajectory distance information of the same face object, and non-face feature information of non-face objects; wherein, the trajectory distance information is a video frame subsequence containing the same detection box identifier;

[0008] It determines whether the trajectory distance information is within a preset approach distance range, and whether there is at least one facial feature information in the facial feature information that matches the target facial feature information of the pre-stored target facial object; wherein, the target facial object is a user with the authority to operate the vehicle;

[0009] If the trajectory distance information is determined to be within the preset approach distance range, and at least one facial feature information in the facial feature information matches the target facial feature information, the corresponding control command is generated based on the facial feature information, user driving habit information and non-facial feature information to automatically open the vehicle doors and / or trunk.

[0010] Furthermore, face object detection and tracking, as well as non-face object detection, are performed on the video frame sequence to obtain facial feature information of face objects, trajectory distance information of the same face object, and non-face feature information of non-face objects, including:

[0011] The first detection model, which is pre-trained, is used to detect faces in the video frame sequence to obtain the facial feature information of the faces.

[0012] Track each video frame containing a face object to obtain the trajectory distance information of the same face object;

[0013] The pre-trained second detection model is used to detect non-face objects in the video frame sequence, and the non-face feature information of the non-face objects is obtained.

[0014] Furthermore, based on facial feature information, user driving habit information, and non-facial feature information, corresponding control commands are generated to automatically open the vehicle doors and / or trunk, including:

[0015] Determine whether there is at least one facial feature in the facial feature information that matches the infant feature information and / or whether there is at least one non-facial feature in the non-facial feature information that matches the pet feature information;

[0016] If it is determined that at least one facial feature in the facial feature information matches the characteristics of an infant and / or at least one non-facial feature in the non-facial feature information matches the characteristics of a pet, a first control command is generated, and the front and rear doors of the vehicle are automatically opened based on the first control command.

[0017] If it is determined that there is no facial feature information that matches infant feature information and there is no non-facial feature information that matches pet feature information, count the number of objects corresponding to facial objects with different facial feature information in the video frame sequence.

[0018] Extracting the area information of the object from non-facial feature information;

[0019] The volume of the object is determined based on the object area information and the frame area information of the video frame.

[0020] Based on the number of objects, user driving habits, and cargo volume, corresponding control commands are generated to automatically open the vehicle doors and / or trunk.

[0021] Furthermore, based on the number of objects, user driving habits, and cargo volume, corresponding control commands are generated to automatically open the vehicle doors and / or trunk, including:

[0022] Determine if the number of objects is greater than the first preset number of objects;

[0023] If it is determined that the number of objects is greater than the first preset number of objects, the step of generating a first control command is executed, and the front and rear doors of the vehicle are automatically opened based on the first control command;

[0024] If the number of objects is determined to be equal to or less than the first preset number of objects, a second control command is generated, and the front door of the vehicle is automatically opened based on the second control command.

[0025] Determine whether the volume of the load is greater than the preset volume of the load;

[0026] If it is determined that the volume of the cargo is greater than the preset cargo volume, a third control command is generated, and the vehicle's trunk is automatically opened based on the third control command.

[0027] Obtain the video frame sequence of the trunk;

[0028] Determine whether the usable size of the rear box exceeds the preset usable size based on the video frame sequence of the box;

[0029] If it is determined that the size of the trunk exceeds the preset size, a fourth control command is generated, and the rear door of the vehicle is automatically opened based on the fourth control command. In addition, a first alarm prompt is generated and sent to the alarm device on the vehicle and / or the terminal of the target face object to provide an alarm prompt.

[0030] If it is determined that the volume of the cargo is equal to or less than the preset cargo volume, the user habit control operation results that match the number of objects are retrieved from the user's vehicle usage habit information; wherein, the user's vehicle usage habit information includes the correspondence between the user habit control operation results and the number of users.

[0031] Determine whether the user's habitual control operation result is the result of opening the trunk control operation or the result of opening the rear door control operation;

[0032] If it is determined that the user's habitual control operation result is the result of opening the trunk, the third control command is generated, and the trunk of the vehicle is automatically opened based on the third control command.

[0033] If it is determined that the user's habitual control operation result is the result of opening the rear door, a fourth control command is generated, and the rear door of the vehicle is automatically opened based on the fourth control command.

[0034] Furthermore, after generating the first control command and automatically opening the front and rear doors of the vehicle based on the first control command, the method further includes:

[0035] Determine whether the volume of the load is greater than the preset volume of the load;

[0036] If it is determined that the volume of the cargo is greater than the preset cargo volume, the third control command is generated, and the trunk of the vehicle is automatically opened based on the third control command.

[0037] Obtain the video frame sequence of the trunk;

[0038] Determine whether the usable size of the rear box exceeds the preset usable size based on the video frame sequence of the box;

[0039] If it is determined that the size of the trunk exceeds the preset size, the system generates a first alarm and sends the first alarm to the alarm device on the vehicle and / or the terminal of the target face object to provide an alarm notification.

[0040] Furthermore, after generating corresponding control commands based on facial feature information, user driving habit information, and non-facial feature information to automatically open the vehicle doors and / or trunk, the method further includes:

[0041] Obtain the in-vehicle video frame sequence of facial objects on the vehicle;

[0042] Determine whether all facial figures in the vehicle are wearing seat belts based on the in-vehicle video frame sequence;

[0043] Once it is determined that all passengers are wearing seatbelts, a first closing command is generated to automatically close the vehicle doors and / or trunk.

[0044] Furthermore, before generating corresponding control commands based on facial feature information, user driving habit information, and non-facial feature information to automatically open the vehicle doors and / or trunk, the method further includes:

[0045] Receive the distance of the obstacle from the vehicle;

[0046] Determine if the distance to the obstacle is less than the preset distance;

[0047] If the distance to an obstacle is determined to be less than a preset distance, a vehicle movement command is generated, and the vehicle is parked from its initial position to a safe position based on the vehicle movement command.

[0048] Furthermore, after generating corresponding control commands based on facial feature information, user driving habit information, and non-facial feature information to automatically open the vehicle doors and / or trunk, the method further includes:

[0049] Determine whether the trajectory distance information is outside the preset approach distance range and exceeds the maximum departure distance;

[0050] If it is determined that the trajectory distance information is not within the preset approach distance range but does not exceed the maximum departure distance, a second alarm prompt is generated according to the preset time interval, and the second alarm prompt is sent to the alarm device on the vehicle and / or the terminal of the target face object to provide an alarm prompt.

[0051] Calculate the total duration of the second alarm notification;

[0052] Determine whether the total duration of the prompts exceeds the preset total duration;

[0053] If the total duration exceeds the preset total duration, or if the trajectory distance information exceeds the maximum departure distance, a second closing command is generated to automatically close the vehicle's open doors and / or trunk, and to park the vehicle from a safe position back to the initial position.

[0054] Furthermore, the method also includes:

[0055] Count the number of instructions that generate the second close instruction;

[0056] When the number of commands reaches the preset number of commands, the maximum approach distance in the preset approach distance range is adjusted based on the lower limit distance of the vehicle.

[0057] Furthermore, the method also includes:

[0058] Receive temporary authorization instructions from the terminal of the target face object;

[0059] Temporary face images of temporary face objects are collected based on temporary authorization instructions;

[0060] Send a temporary facial image to the user terminal;

[0061] Receive temporary permissions for a temporary face object issued by the terminal; wherein, the temporary permissions include the right to control the vehicle doors and / or trunk, and the duration of the temporary permissions;

[0062] Temporary facial recognition objects can control the opening and closing of vehicle doors and / or trunk based on temporary permissions.

[0063] Furthermore, the method also includes:

[0064] Receive a temporary face image of the target face object sent by the terminal.

[0065] If temporary facial feature information corresponding to a temporary face image is identified from the facial feature information, the video frame containing the temporary face object is sent to the user terminal from the video frame sequence;

[0066] Receive temporary permissions for a temporary face object issued by the terminal; wherein, the temporary permissions include the right to control the vehicle doors and / or trunk, and the duration of the temporary permissions;

[0067] Temporary facial recognition objects can control the opening and closing of vehicle doors and / or trunk based on temporary permissions.

[0068] Furthermore, the method also includes:

[0069] Collect video frame sequences of operations performed by a temporary face object on the vehicle control operation;

[0070] When a temporary face object is identified as having completed its operation on the vehicle based on the video frame sequence, or when the temporary face object's operation on the vehicle's opening control reaches the duration of the temporary permission, the temporary permission is cleared and an operation completion notification is generated and sent to the user terminal.

[0071] Furthermore, the method also includes:

[0072] When a temporary face object is detected in the video frame sequence and its operation on the vehicle exceeds the user's authorized permissions, a third alarm is generated and sent to the vehicle's alarm device and / or a terminal for alarm notification.

[0073] A vehicle control device, wherein the device is used in a vehicle control center, the device comprising:

[0074] The acquisition module is used to acquire user driving habit information and video frame sequences containing faces when the vehicle is parked; wherein, the user driving habit information is the result of user habit control operation obtained by counting the most control operations performed by the user on the vehicle within a preset statistical time period.

[0075] The detection and tracking module is used to detect and track face objects and detect non-face objects in a video frame sequence, and obtain the face feature information of face objects, the trajectory distance information of the same face object, and the non-face feature information of non-face objects; wherein, the trajectory distance information is a video frame sub-sequence containing the same detection box identifier;

[0076] The judgment module is used to determine whether the trajectory distance information is within the preset approach distance range, and whether there is at least one facial feature information in the facial feature information that matches the target facial feature information of the pre-stored target facial object; wherein, the target facial object is a user with the authority to operate the vehicle;

[0077] The control module is activated to generate corresponding control commands based on facial feature information, user driving habit information, and non-facial feature information when it is determined that the trajectory distance information is within a preset approach distance range and at least one facial feature information in the facial feature information matches the target facial feature information, so as to automatically control the opening of the vehicle doors and / or trunk.

[0078] A vehicle, wherein the vehicle is equipped with a vehicle control center, wherein the vehicle control center is used to execute the above-described vehicle control method.

[0079] A storage medium storing one or more programs that can be executed by one or more processors to implement the vehicle control method described above.

[0080] The beneficial effects of this invention are:

[0081] This embodiment provides a vehicle control method, device, vehicle, and storage medium, including: when the vehicle is parked, acquiring user driving habit information and a video frame sequence containing faces; performing face object detection and tracking and non-face object detection on the video frame sequence to obtain facial feature information of the face object, trajectory distance information of the same face object, and non-face feature information of the non-face object; wherein, the trajectory distance information is a video frame sub-sequence containing the same detection box identifier; when it is determined that the trajectory distance information is within a preset approach distance range, and at least one face feature information in the face feature information matches the target face feature information, corresponding control commands are generated based on the face feature information, user driving habit information, and non-face feature information to automatically open the vehicle doors and / or trunk. This invention enables automatic opening and closing of vehicle doors and / or trunk while the vehicle is parked, using only facial and non-facial feature information obtained from video frame sequences, as well as user driving habit information. Compared to existing methods where users still need to manually open and close the vehicle doors and trunk even when carrying the vehicle's digital key, this invention provides convenient and flexible vehicle control, greatly improving the user experience. Attached Figure Description

[0082] Figure 1 A flowchart illustrating an embodiment of the vehicle control method provided by the present invention;

[0083] Figure 2 A schematic diagram of camera installation distribution provided by the present invention;

[0084] Figure 3 A flowchart illustrating an embodiment of another vehicle control method provided by the present invention;

[0085] Figure 4 A flowchart illustrating an embodiment of another vehicle control method provided by the present invention;

[0086] Figure 5 A flowchart illustrating an embodiment of another vehicle control method provided by the present invention;

[0087] Figure 6 A flowchart illustrating an embodiment of another vehicle control method provided by the present invention;

[0088] Figure 7 This is a block diagram illustrating an embodiment of a vehicle control device provided by the present invention.

[0089] Among them, 1-vehicle; 200-external camera; 201-trunk camera; 202-interior camera; 701-acquisition module; 702-detection and tracking module; 703-judgment module; 704-opening control module. Detailed Implementation

[0090] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0091] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0092] This embodiment proposes a vehicle control method, wherein the method is applied to a vehicle control center, which can be understood as a vehicle controller deployed on the vehicle. (See [link to relevant documentation]). Figure 1 This is a flowchart illustrating an embodiment of a vehicle control method provided in this example. Figure 1 As shown, the process may include the following steps:

[0093] Step 101: With the vehicle in a parked state, acquire user driving habit information and a video frame sequence containing faces;

[0094] Parking status refers to a vehicle state where the vehicle is not started, and only the cameras deployed on the vehicle and the vehicle control center are operating at low power.

[0095] The above video frame sequence consists of connected video frames captured by external cameras deployed on the vehicle. To capture a sequence of video frames including faces around the vehicle, multiple external cameras need to be deployed on the vehicle. For ease of understanding... Figure 2 A schematic diagram of camera installation distribution is shown, such as... Figure 2 As shown, a total of four external cameras 200 are installed on vehicle 1, respectively mounted on the front and rear windshields and on the left and right A-pillars. In practice, the installation positions and number of external cameras on the vehicle can be set according to actual needs and are not limited here.

[0096] User vehicle usage habit information is the result of user habit control operations obtained by statistically analyzing the most frequent control operations performed by users on the vehicle within a preset statistical time period. Typically, this user vehicle usage habit information is related to the number of users, meaning that the control operations performed on the vehicle differ with different numbers of users. Therefore, the obtained user habit control operation results will vary. Thus, the user vehicle usage habit information in this embodiment includes the correspondence between user habit control operation results and the number of users.

[0097] For example, if there are two users within a three-month period, and the most frequent control operation is opening the rear door, then the user's habitual control operation result is the rear door opening control operation result. If there is only one user, and the most frequent control operation is opening the trunk, then the user's habitual control operation result is the trunk opening control operation result. If there is only one user and no control operation is performed on the trunk or rear door, then the user's habitual control operation result is none.

[0098] Step 102: Perform face object detection and tracking and non-face object detection on the video frame sequence to obtain face feature information of face objects, trajectory distance information of the same face object, and non-face feature information of non-face objects; wherein, the trajectory distance information is a video frame sub-sequence containing the same detection box identifier;

[0099] Non-face objects refer to objects other than human faces in a video frame sequence, such as cargo (i.e., luggage), pets, houses, plants, roads, etc.

[0100] Facial feature information refers to the facial features that characterize the face of a human object. Facial object detection can identify the facial features corresponding to each human object from a video frame sequence. Non-facial feature information refers to the features that characterize non-human objects. For example, if a non-human object is a pet dog, then the non-facial feature information would be the area and height of its hair, the shape of its ears, etc.

[0101] The trajectory distance information is the walking trajectory of the face object. By tracking the face object, the walking trajectory of each face object included in the above video frame sequence can be obtained.

[0102] Step 103: Determine whether the trajectory distance information is within the preset approach distance range, and whether there is at least one facial feature information in the facial feature information that matches the target facial feature information of the pre-stored target facial object; wherein, the target facial object is a user with the authority to operate the vehicle;

[0103] The preset approach distance range is the range of distance for controlling the vehicle. For example, the preset approach distance range is [0 meters, 2 meters]. The specific preset approach distance range can be set according to actual needs and is not limited here.

[0104] In this embodiment, if at least one of the multiple trajectory distance information is within the preset approach distance range, and the trajectory distance information corresponds to at least one of the multiple trajectory distance information, then step 104 can be executed to automatically control the vehicle doors and / or trunk; if at least one of the multiple trajectory distance information is within the preset approach distance range but the face objects corresponding to at least one of the trajectory distance information are all users without vehicle operation permissions (i.e., the face feature information corresponding to the face object does not match the target face feature information of the pre-stored target face object), or if at least one face object is a user with vehicle operation permissions but the trajectory distance information corresponding to the face object is not within the preset approach distance range, then the vehicle doors and / or trunk cannot be automatically controlled.

[0105] Step 104: If it is determined that the trajectory distance information is within the preset approach distance range, and at least one facial feature information in the facial feature information matches the target facial feature information, the corresponding control command is generated based on the facial feature information, user driving habit information and non-facial feature information to automatically open the vehicle doors and / or trunk.

[0106] The vehicle control method provided in the above embodiments can achieve automatic vehicle control using only facial feature information, user driving habit information and non-facial feature information. It eliminates the need for users to manually control the vehicle doors and trunk when they carry the vehicle digital key. This control method is convenient and flexible, greatly improving the user's driving experience.

[0107] In this embodiment, step 102 can be implemented through steps A1 to A3:

[0108] Step A1: Use the pre-trained first detection model to detect face objects in the video frame sequence and obtain the face feature information of the face objects;

[0109] The first detection model mentioned above can be a face detection model obtained by training deep neural networks such as SSD (SingleShot MultiBox Detector), RetinaNet, or Faster RCNN using labeled face feature information training samples. The first detection model trained above can accurately detect the face objects contained in each video frame, as well as the face feature information of the face objects.

[0110] Step A2: Track each video frame including the face object to obtain the trajectory distance information of the same face object;

[0111] The aforementioned trajectory distance information includes both the walking trajectory of the face object and the straight-line distance between the face object and the vehicle at each point on the walking trajectory.

[0112] In this embodiment, IOU (Intersection over Union) matching tracking algorithm can be used to track each video frame to obtain the trajectory distance information of the same face object. The aforementioned IOU matching tracking algorithm is a standard for measuring the accuracy of detecting corresponding objects in a specific dataset. This standard is used to measure the correlation between the actual and the predicted values. The higher the correlation, the higher the standard value. In this embodiment, for simplicity, the aforementioned standard value can be implemented by a pre-trained first detection model. This model can calculate the overlap rate between object detection boxes containing face objects in video frames containing face objects, that is, the ratio of their intersection to their union. If the calculated overlap rate between object detection boxes in two adjacent video frames is higher than a preset overlap rate threshold or is the maximum overlap rate, then the face objects corresponding to the two object detection boxes are determined to be the same target object. The aforementioned IOU matching tracking algorithm can obtain the trajectory distance information of the same face object in the aforementioned video frame sequence.

[0113] In addition to using the aforementioned IOU matching tracking algorithm to track face objects, face objects can also be tracked by calculating the linear correlation between object detection boxes in two video frames based on the coordinate information of the object detection boxes including the face object using Mahalanobis distance or covariance distance. The higher the linear correlation, the more likely it is the same face object, thus obtaining the trajectory distance information of each face object.

[0114] Step A3: Detect non-face objects in the video frame sequence using a pre-trained second detection model to obtain non-face feature information of the non-face objects.

[0115] Similarly, the second detection model mentioned above can be a detection model obtained by training deep neural networks such as SSD (SingleShot MultiBox Detector), RetinaNet, or Faster R-CNN using labeled non-face feature information training samples. The trained second detection model can accurately detect non-face objects and non-face feature information of non-face objects contained in each video frame.

[0116] Steps A1-A2 and A3 can be executed in any order; that is, steps A1-A2 and A3 can be performed simultaneously, or step A3 can be executed before steps A1-A2, or steps A1-A2 can be executed before step A3.

[0117] In this embodiment, step 104 can be implemented through the following process:

[0118] Determine whether there is at least one facial feature in the facial feature information that matches the infant feature information and / or whether there is at least one non-facial feature in the non-facial feature information that matches the pet feature information;

[0119] If it is determined that at least one facial feature in the facial feature information matches the characteristics of an infant and / or at least one non-facial feature in the non-facial feature information matches the characteristics of a pet, a first control command is generated, and the front and rear doors of the vehicle are automatically opened based on the first control command.

[0120] In this embodiment, the situation where a person with vehicle operation authority is carrying an infant and / or pet is considered the first priority. Since infants and pets cannot sit in the front seat, if it is determined that at least one facial feature in the facial feature information matches the infant feature information and / or at least one non-facial feature information matches the pet feature information, a first control command needs to be generated to open the front and rear doors of the vehicle so that the infant and / or pet can sit in the rear seat of the vehicle.

[0121] Typically, when carrying infants and / or pets, the amount of cargo carried increases. In this embodiment, to facilitate automatic control of the trunk, it is necessary to determine whether the trunk will open automatically based on the cargo volume. The process for determining the cargo volume is as follows: extracting the cargo area information from non-facial feature information; determining the cargo volume based on the cargo area information and the frame area information of the video frame. That is, the ratio of the cargo area information to the frame area information determines the proportion of the cargo in the video frame. It is clear that the larger the proportion, the larger the cargo volume, and vice versa. Since the proportion is directly proportional to the cargo volume, the proportion can be regarded as the cargo volume.

[0122] In specific implementation, after generating the first control command and automatically opening the front and rear doors of the vehicle based on the first control command, the following steps are also required: determining whether the cargo volume is greater than the preset cargo volume; if the cargo volume is greater than the preset cargo volume, generating the third control command and automatically opening the vehicle's trunk based on the third control command; acquiring the trunk's video frame sequence; determining whether the trunk's usable size exceeds the preset usable size based on the trunk's video frame sequence; if the trunk's usable size exceeds the preset usable size, generating the first alarm prompt, and sending the first alarm prompt to the vehicle's alarm device and / or the target face object's terminal for alarm notification.

[0123] The alarm device can be a vehicle alarm, and the terminal can be a mobile phone, smartwatch, or other device currently being used by the person whose face is being viewed.

[0124] In practice, if the volume of the object being transported exceeds the preset volume, it indicates that the person carrying a lot of luggage needs to have the trunk opened to accommodate it. In this case, the following steps are taken: Figure 1 Two trunk cameras 201, installed in the trunk, capture video frame sequences of the trunk to further determine whether the trunk can accommodate all luggage. If not, an alarm is triggered to alert the person with the face. If the cargo volume is no larger than a preset cargo volume, it indicates that the person with the face is not carrying much luggage, and the trunk does not need to be opened automatically. The number and installation location of the trunk cameras, as well as the preset cargo volume and preset usage dimensions, can all be set according to actual needs and are not limited here.

[0125] If it is determined that there is no facial feature information that matches infant feature information and there is no non-facial feature information that matches pet feature information, count the number of objects corresponding to facial objects with different facial feature information in the video frame sequence.

[0126] The number of objects refers to the number of face objects whose trajectory distance information is within a preset proximity distance range, indicating that there are face objects of this number that need to take the vehicle.

[0127] Extracting the area information of the object from non-facial feature information;

[0128] The volume of the object is determined based on the object area information and the frame area information of the video frame.

[0129] The process for determining the volume of the aforementioned cargo can be found in the above description and will not be limited here.

[0130] Based on the number of objects, user driving habits, and cargo volume, corresponding control commands are generated to automatically open the vehicle doors and / or trunk.

[0131] Specifically, generating corresponding control commands based on the number of objects, user driving habits, and cargo volume to automatically open the vehicle doors and / or trunk can be achieved through the following process:

[0132] Determine if the number of objects is greater than the first preset number of objects;

[0133] For vehicles with only a driver's seat and a front passenger seat in the front row, the number of the first preset objects mentioned above is preferably 2.

[0134] If the number of objects is determined to be greater than the first preset number of objects, a first control command is generated, and the front and rear doors of the vehicle are automatically opened based on the first control command.

[0135] If the number of objects is determined to be greater than the first preset number of objects, that is, when the number of objects is greater than 2, it means that the driver's seat and the passenger seat of the vehicle cannot meet the current number of passenger face objects. Therefore, the step of automatically opening the front and rear doors of the vehicle is executed.

[0136] After generating the first control command and automatically opening the front and rear doors of the vehicle based on the first control command, it is also necessary to perform the control operation of determining whether the trunk needs to be opened based on the volume of the cargo. This process is the same as the process performed after opening the front and rear doors when carrying infants and / or pets, and will not be described in detail here.

[0137] If the number of objects is determined to be equal to or less than the first preset number of objects, a second control command is generated, and the front door of the vehicle is automatically opened based on the second control command.

[0138] In practice, if the number of objects is equal to or less than the first preset number of objects, i.e., the number of objects is 2 or 1, the vehicle will automatically open the front door.

[0139] Determine whether the volume of the load is greater than the preset volume of the load;

[0140] If it is determined that the volume of the cargo is greater than the preset cargo volume, a third control command is generated, and the vehicle's trunk is automatically opened based on the third control command.

[0141] If the cargo volume is larger than the preset cargo volume, it means that the face object is carrying a lot of luggage, and the vehicle needs to be automatically controlled to open the trunk so that the face object can put the luggage in the trunk.

[0142] If the volume of the cargo is determined to be equal to or less than the preset cargo volume, the video frame sequence of the trunk is obtained.

[0143] Determine whether the usable size of the rear box exceeds the preset usable size based on the video frame sequence of the box;

[0144] If it is determined that the size of the trunk exceeds the preset size, a fourth control command is generated, and the rear door of the vehicle is automatically opened based on the fourth control command. In addition, a first alarm prompt is generated and sent to the alarm device on the vehicle and / or the terminal of the target face object to provide an alarm prompt.

[0145] If the size of the trunk exceeds the preset size, it means that the trunk cannot hold all the luggage carried by the person with the face. An alarm can be triggered to notify the user. Since the person with the face is not carrying infants and / or pets, there is space in the back seat of the vehicle to place luggage. Therefore, in this embodiment, the rear door of the vehicle can be automatically opened so that the person with the face can place luggage in the back seat of the vehicle.

[0146] Search for user habit control operation results that match the number of objects based on user car usage habit information;

[0147] The user driving habit information includes the correspondence between user habit control operation results and the number of users. In this embodiment, since the user habit control operation results may be different for different object numbers, for example, when the number of objects is 2, the found user habit control operation result is the trunk opening control operation, and when the number of objects is 1, the found user habit control operation result is the rear door opening control operation result. Therefore, it is necessary to find the user habit control operation result that matches the current number of objects from the user driving habit information so as to determine whether to control the vehicle to automatically open the trunk or rear door through the user habit control operation result.

[0148] Determine whether the user's habitual control operation result is the result of opening the trunk control operation or the result of opening the rear door control operation;

[0149] If it is determined that the user's habitual control operation result is the result of opening the trunk, a third control command is generated, and the vehicle's trunk is automatically opened based on the third control command.

[0150] After automatically opening the vehicle's trunk, it is also necessary to determine whether to open the rear door based on the size of the trunk, which will not be elaborated here.

[0151] If it is determined that the user's habitual control operation result is the result of opening the rear door, a fourth control command is generated, and the rear door of the vehicle is automatically opened based on the fourth control command.

[0152] If the user's usual control operation results are not the result of opening the trunk or opening the rear door, then the automatic opening of the vehicle's trunk and rear door will not be controlled.

[0153] In practical applications, after automatically opening the vehicle doors and / or trunk, it is also possible to automatically close the opened doors and / or trunk. Specifically, this involves: acquiring a sequence of in-vehicle video frames of the passengers' faces; determining whether all passengers are wearing seat belts based on the in-vehicle video frame sequence; and generating a first closing command if it is determined that all passengers are wearing seat belts, so as to automatically close the opened doors and / or trunk.

[0154] In-vehicle video frame sequences can be obtained through, for example Figure 1 Data was collected by two in-vehicle cameras 202 installed inside the vehicle, such as... Figure 1 As shown, one in-vehicle camera is installed in the front cabin of the vehicle, and another in the rear cabin. The specific number of in-vehicle cameras can be installed according to actual needs and is not limited here.

[0155] Once the facial recognition of passengers is confirmed by the in-vehicle video frame sequence and the seatbelt is confirmed, it indicates that the passengers are ready to travel. At this point, the system can automatically control the opening of the car doors and / or the trunk to close, ensuring travel safety.

[0156] To facilitate understanding of the above description, Figure 3 A simplified flowchart of the vehicle control method is shown, such as... Figure 3 As shown, the process may include the following steps:

[0157] start;

[0158] Step 301: The video frame sequence identifies the facial object within a preset approach distance range from the vehicle;

[0159] In practice, the trajectory distance information of the face object is first determined by the video frame sequence. If the trajectory distance information is within the preset approach distance range, it means that the walking trajectory of the face object is approaching the vehicle and there may be a need for a vehicle.

[0160] Step 302: Identify whether the face object is a target face object with the authority to operate the vehicle based on the video frame sequence;

[0161] Step 302 determines whether the facial features of the face object match the facial features of the target face. If the facial features match the target face, the face object is the target face object with vehicle operation authority, and step 303 is executed. If the facial features do not match the target face, the face object is not the target face object with vehicle operation authority, and automatic vehicle control ends.

[0162] Step 303: Identify whether there are infants and / or pets based on the video frame sequence;

[0163] If an infant and / or pet is identified, proceed to step 304; if no infant and / or pet is identified, proceed to step 311.

[0164] Step 304: Automatically open the front and rear doors of the vehicle;

[0165] Step 305: Determine whether the volume of the load is greater than the preset volume of the load;

[0166] If the volume of the object is greater than the preset volume, proceed to step 306; if the volume of the object is equal to or less than the preset volume, proceed to step 309.

[0167] Step 306: Automatically open the vehicle's trunk;

[0168] Step 307: Determine whether the usable size of the rear box exceeds the preset usable size based on the box video frame sequence;

[0169] If it is determined that the usable size of the trunk exceeds the preset usable size, proceed to step 308; if it is determined that the usable size of the trunk does not exceed the preset usable size, proceed to step 309.

[0170] Step 308: Generate the first alarm prompt, indicating that the size of the face object is too large;

[0171] Step 309: Based on the in-vehicle video frame sequence, it is recognized that all the facial objects in the vehicle are wearing seat belts;

[0172] Step 310: Automatically close the open doors and / or trunk;

[0173] Automatic closing of open doors and / or the trunk is considered the end of automatic vehicle control.

[0174] Step 311: Determine if the number of objects is greater than 2;

[0175] If the number of objects is determined to be greater than 2, proceed to step 312; if the number of objects is determined to be equal to or less than 2, proceed to step 319.

[0176] Step 312: Automatically open the front and rear doors of the vehicle;

[0177] Step 313: Determine whether the volume of the load is greater than the preset volume of the load;

[0178] If the volume of the load is greater than the preset volume, proceed to step 314; if the volume of the load is equal to or less than the preset volume, proceed to step 317.

[0179] Step 314: Automatically open the vehicle's trunk;

[0180] Step 315: Determine whether the usable size of the rear box exceeds the preset usable size based on the box video frame sequence;

[0181] If it is determined that the usable size of the trunk exceeds the preset usable size, proceed to step 316; if it is determined that the usable size of the trunk does not exceed the preset usable size, proceed to step 317.

[0182] Step 316: Generate the first alarm prompt, indicating that the size of the face object is too large;

[0183] Step 317: Based on the in-vehicle video frame sequence, it is recognized that all the facial objects in the vehicle are wearing seat belts;

[0184] Step 318: Automatically close the open doors and / or trunk;

[0185] Automatic closing of open doors and / or the trunk is considered the end of automatic vehicle control.

[0186] Step 319: Automatically open the front doors of the vehicle;

[0187] Step 320: Determine if the number of objects is equal to 2;

[0188] If the number of objects is equal to 2, proceed to step 321; if the number of objects is not equal to 2 (i.e., 1), proceed to step 330.

[0189] Step 321: Determine whether the volume of the load is greater than the preset volume of the load;

[0190] If the volume of the object is greater than the preset volume, proceed to step 322; if the volume of the object is equal to or less than the preset volume, proceed to step 328.

[0191] Step 322: Automatically open the vehicle's trunk;

[0192] Step 323: Determine whether the usable size of the rear box exceeds the preset usable size based on the box video frame sequence;

[0193] If it is determined that the usable size of the trunk exceeds the preset usable size, proceed to step 324; if it is determined that the usable size of the trunk does not exceed the preset usable size, proceed to step 326.

[0194] Step 324: Generate the first alarm prompt, indicating that the size of the face object is too large;

[0195] Step 325: The rear door of the vehicle opens automatically;

[0196] Steps 319 and 320 are executed in no particular order.

[0197] Step 326: Based on the in-vehicle video frame sequence, it is recognized that all the facial objects in the vehicle are wearing seat belts;

[0198] Step 327: Automatically close the open doors and / or trunk;

[0199] Automatic closing of open doors and / or the trunk is considered the end of automatic vehicle control.

[0200] Step 328: Obtain the user habit control operation results corresponding to a number of objects of 2;

[0201] Step 329: Determine whether the user's habitual control operation result is the result of opening the trunk control operation or the result of opening the rear door control operation;

[0202] If the user's usual control operation result is the trunk opening control operation result, then proceed to step 322; if the user's usual control operation result is the rear door opening control operation result, then proceed to step 325; if the user's usual control operation result is neither the trunk opening control operation result nor the rear door opening control operation result, then proceed to step 326.

[0203] Step 330: Determine whether the volume of the load is greater than the preset volume of the load;

[0204] If the volume of the load is greater than the preset volume, proceed to step 331; if the volume of the load is equal to or less than the preset volume, proceed to step 337.

[0205] Step 331: Automatically open the vehicle's trunk;

[0206] Step 332: Determine whether the usable size of the rear box exceeds the preset usable size based on the box video frame sequence;

[0207] If it is determined that the usable size of the trunk exceeds the preset usable size, proceed to step 333; if it is determined that the usable size of the trunk does not exceed the preset usable size, proceed to step 335.

[0208] Step 333: Generate the first alarm prompt, indicating that the size of the face object is too large;

[0209] Step 334: Automatically open the rear door of the vehicle;

[0210] Steps 333 and 334 are executed in no particular order.

[0211] Step 335: Based on the in-vehicle video frame sequence, it is recognized that all the facial objects in the vehicle are wearing seat belts;

[0212] Step 336: Automatically close the open doors and / or trunk;

[0213] Automatic closing of open doors and / or the trunk is considered the end of automatic vehicle control.

[0214] Step 337: Obtain the user habit control operation result corresponding to an object quantity of 1;

[0215] Step 338: Determine whether the user's habitual control operation result is the result of opening the trunk control operation or the result of opening the rear door control operation;

[0216] If the user's usual control operation result is the trunk opening control operation result, then proceed to step 331; if the user's usual control operation result is the rear door opening control operation result, then proceed to step 334; if the user's usual control operation result is neither the trunk opening control operation result nor the rear door opening control operation result, then proceed to step 335.

[0217] Finish.

[0218] exist Figure 1 Based on this, see Figure 4 This is a flowchart illustrating another embodiment of the vehicle control method provided in this embodiment. Figure 4 As shown, the process may include the following steps:

[0219] Step 401: While the vehicle is parked, acquire user driving habit information and a video frame sequence containing a face.

[0220] Step 402: Perform face object detection and tracking and non-face object detection on the video frame sequence to obtain face feature information of face objects, trajectory distance information of the same face object, and non-face feature information of non-face objects; wherein, the trajectory distance information is a video frame sub-sequence containing the same detection box identifier;

[0221] Step 403: Determine whether the trajectory distance information is within the preset approach distance range, and whether there is at least one facial feature information in the facial feature information that matches the target facial feature information of the pre-stored target facial object;

[0222] Steps 401-403 above can be found in steps 101-103, and will not be repeated here.

[0223] Step 404: If it is determined that the trajectory distance information is within the preset approach distance range and that at least one facial feature information in the facial feature information matches the target facial feature information, the obstacle distance from the vehicle is received.

[0224] In the case where it is determined that the trajectory distance information is within the preset approach distance range and at least one facial feature information in the facial feature information matches the target facial feature information, it is also necessary to determine whether there are obstacles around the vehicle that affect the facial object from placing luggage and / or entering the vehicle. Therefore, step 404 needs to be executed to obtain the obstacle distance for subsequent distance determination.

[0225] Step 405: Determine whether the distance to the obstacle is less than the preset distance;

[0226] If the distance information of the obstacle is determined to be less than the preset distance, it means that the obstacle around the vehicle affects the use of the face object on the vehicle. Therefore, step 406 needs to be executed to automatically move the vehicle. If the distance information of the obstacle is determined to be greater than or equal to the preset distance, it means that the obstacle around the vehicle does not affect the use of the face object on the vehicle. Therefore, the vehicle does not need to be moved and step 407 is executed.

[0227] Step 406: Generate a vehicle relocation instruction, and park the vehicle from its initial position to a safe position based on the vehicle relocation instruction;

[0228] A safe location is one where there are no obstacles that would prevent a person from using the vehicle and where the safety of both the person and the vehicle is guaranteed.

[0229] Step 407: Generate corresponding control commands based on facial feature information, user driving habit information and non-facial feature information to automatically open the vehicle doors and / or trunk.

[0230] This step 407, which automatically controls the opening of the vehicle's doors and / or trunk, can be found in the descriptions of steps B1 to B6 above, and will not be detailed here. This step 407 enables automatic opening control of the vehicle's doors and / or trunk.

[0231] Step 408: Determine whether the trajectory distance information is not within the preset approach distance range and exceeds the maximum departure distance;

[0232] In this embodiment, the maximum departure distance is greater than the maximum value in the preset approach distance range. The specific maximum departure distance can be set according to actual needs, such as 3 meters, 4 meters or 5 meters, and is not limited here.

[0233] If the trajectory distance information is not within the preset approach distance range but does not exceed the maximum departure distance, proceed to step 409; if the trajectory distance information exceeds the maximum departure distance, it means that the trajectory distance information is definitely not within the preset approach distance range, then proceed to step 412.

[0234] Step 409: Generate a second alarm prompt according to a preset time interval, and send the second alarm prompt to the alarm device on the vehicle and / or the user terminal of the target face object to provide an alarm prompt;

[0235] Step 410: Calculate the total duration of the second alarm notification;

[0236] The total duration of this notification is the total duration from the first alarm notification to the most recent alarm notification, which is the total duration of multiple preset time intervals.

[0237] Step 411: Determine whether the total duration of the prompts exceeds the preset total duration;

[0238] If the total duration exceeds the preset total duration, proceed to step 412; if the total duration does not exceed the preset total duration, proceed to step 408.

[0239] Step 412: Generate a second closing command to automatically close the vehicle's open doors and / or trunk, and to park the vehicle from a safe position to the initial position.

[0240] Step 413: Count the number of times the second close instruction is generated;

[0241] Step 414: When the number of commands reaches the preset number of commands, adjust the maximum approach distance in the preset approach distance range based on the lower limit distance of leaving the vehicle.

[0242] The number of commands reaching the preset number indicates that the face object does not need to use the vehicle within the current preset approach distance range. To save energy for the vehicle control center to start the doors and / or trunk, the maximum approach distance within the preset approach distance range needs to be adjusted. For example, if the preset approach distance range is [0 meters, 2 meters), but the face object leaves the vehicle at 1.5 meters, the vehicle control center, after learning the behavior, will adjust the maximum approach distance within the preset approach distance range from 2 meters to 1.5 meters. The adjustment result can then be sent to the in-vehicle application to notify the target face object.

[0243] In addition to learning and adjusting the preset approach distance range through the vehicle control center, the target face object can also have its range adjusted through the in-vehicle application, and the adjusted preset approach distance range can be sent to the vehicle control center to update the current preset approach distance range.

[0244] Additionally, while the vehicle is parked, temporary authorization can be granted to allow temporary facial recognition data to use the vehicle. For specific implementation details, please refer to [link / reference needed]. Figure 5 This is a flowchart illustrating another embodiment of the vehicle control method provided in this embodiment. Figure 5 As shown, the process may include the following steps:

[0245] Step 501: Receive a temporary authorization instruction issued by the terminal using the target face object;

[0246] Since the target face object does not have a temporary face image of the temporary face object, a camera is needed to capture it so that the target face object can determine that the temporary face object can operate the vehicle.

[0247] Step 502: Acquire a temporary face image of the temporary face object based on the temporary authorization instruction;

[0248] Step 503: Send the temporary face image to the user terminal;

[0249] Step 504: Receive temporary permissions for the temporary face object issued by the terminal; wherein, the temporary permissions include the operation permissions to control the vehicle doors and / or trunk, and the duration of the temporary permissions;

[0250] If the target face object determines that the temporary face object can use the vehicle by using the temporary face image displayed on the terminal, the target face object needs to grant temporary permission to the vehicle control center through the vehicle application on the terminal to inform the vehicle control center that the temporary face object can use the vehicle and operate the vehicle. However, the temporary face object needs to operate within the duration of the temporary permission and cannot exceed the permission.

[0251] Step 505: The temporary face object performs opening control operations on the vehicle doors and / or trunk based on temporary permissions.

[0252] During the use of a vehicle by a temporary face object, a sequence of video frames of the temporary face object's control operations on the vehicle is collected; when the operation of the temporary face object on the vehicle is completed based on the sequence of video frames, or when the temporary face object's control operation on the vehicle reaches the duration of the temporary permission, the temporary permission is cleared and an operation completion notification is generated and sent to the user terminal.

[0253] If a temporary face is detected operating the vehicle beyond the authorized scope based on the video frame sequence, a third alarm is generated and sent to the vehicle's alarm device and / or a terminal for alarm notification.

[0254] When the target face object contains a temporary face image of a temporary face object, it can be used... Figure 6 The process enables temporary facial recognition objects to control vehicle operations, such as... Figure 6 As shown, the process may include the following steps:

[0255] Step 601: Receive the temporary face image of the target face object sent by the terminal using the target face object;

[0256] If the target face object can capture a temporary face object, or if a temporary face image of a temporary face object has been previously saved on the user terminal, the temporary face image will be sent to the vehicle control center.

[0257] Step 602: If temporary face feature information corresponding to a temporary face image is identified from the face feature information, the video frame containing the temporary face object is sent from the video frame sequence to the user terminal.

[0258] The video frame of the temporary face object is sent to the terminal for display, so that the target face object can reconfirm whether the temporary face object included in the video frame is the temporary face object on the temporary face image, thereby ensuring vehicle safety.

[0259] Step 603: Receive temporary permissions for the temporary face object issued by the terminal; wherein, the temporary permissions include the operation permissions to control the vehicle doors and / or trunk, and the duration of the temporary permissions;

[0260] Step 604: The temporary face object performs opening control operations on the vehicle doors and / or trunk based on temporary permissions.

[0261] Similarly, during the use of a vehicle by a temporary face object, a sequence of video frames of the temporary face object's control operations on the vehicle is collected; when the operation of the temporary face object on the vehicle is completed based on the sequence of video frames, or when the temporary face object's control operation on the vehicle reaches the duration of the temporary permission, the temporary permission is cleared and an operation completion notification is generated and sent to the user terminal.

[0262] If a temporary face is detected operating the vehicle beyond the authorized scope based on the video frame sequence, a third alarm is generated and sent to the vehicle's alarm device and / or a terminal for alarm notification.

[0263] For temporary users, this provides a safe and reliable way to facilitate them in opening the car door or trunk, which helps to meet the advantages of intelligent and safe vehicles.

[0264] See Figure 7 This is a block diagram of an embodiment of a vehicle control device provided in this embodiment, wherein the device is applied to a vehicle control center, such as... Figure 7 As shown, the device may include:

[0265] The acquisition module 701 is used to acquire user driving habit information and video frame sequence containing faces when the vehicle is in a parked state; wherein, the user driving habit information is the result of user habit control operation obtained by counting the most control operations performed by the user on the vehicle within a preset statistical time period.

[0266] The detection and tracking module 702 is used to perform face object detection and tracking and non-face object detection on the video frame sequence to obtain face feature information of face objects, trajectory distance information of the same face object, and non-face feature information of non-face objects; wherein, the trajectory distance information is a video frame sub-sequence containing the same detection box identifier;

[0267] The judgment module 703 is used to determine whether the trajectory distance information is within the preset approach distance range, and whether there is at least one facial feature information in the facial feature information that matches the target facial feature information of the pre-stored target facial object; wherein, the target facial object is a user with the authority to operate the vehicle;

[0268] The control module 704 is used to generate corresponding control commands based on facial feature information, user driving habit information and non-facial feature information when it is determined that the trajectory distance information is within a preset approach distance range and at least one facial feature information in the facial feature information matches the target facial feature information, so as to automatically control the opening of the vehicle doors and / or trunk.

[0269] This embodiment also provides a vehicle, wherein the vehicle is equipped with a vehicle control center, and the vehicle control center is used to execute the above-described vehicle control method.

[0270] This embodiment also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; it may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; and it may also include combinations of the above types of memory.

[0271] One or more programs in the storage medium can be executed by one or more processors to implement the above vehicle control method.

[0272] The processor is used to execute the vehicle control program stored in the memory to implement the steps of the vehicle control method.

[0273] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0274] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0275] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A vehicle control method, characterized in that, The method is applied to a vehicle control center, and the method includes: When the vehicle is parked, user driving habit information and a video frame sequence containing a face are acquired; wherein, the user driving habit information is the result of user habit control operation obtained by counting the most control operations performed by the user on the vehicle within a preset statistical time period. The video frame sequence is subjected to face object detection and tracking and non-face object detection to obtain the face feature information of the face object, the trajectory distance information of the same face object, and the non-face feature information of the non-face object; wherein, the trajectory distance information is a video frame sub-sequence containing the same detection box identifier; Determine whether the trajectory distance information is within a preset approach distance range, and whether at least one facial feature information in the facial feature information matches the target facial feature information of a pre-stored target facial object; wherein, the target facial object is a user with the authority to operate the vehicle; If it is determined that the trajectory distance information is within the preset approach distance range, and at least one facial feature information in the facial feature information matches the target facial feature information, a corresponding control command is generated based on the facial feature information, the user's driving habit information, and the non-facial feature information to automatically open the vehicle's doors and / or trunk. The step of generating corresponding control commands based on the facial feature information, the user's driving habit information, and the non-facial feature information to automatically open the vehicle doors and / or trunk includes: If it is determined that at least one facial feature in the facial feature information does not match the infant feature information and at least one non-facial feature in the non-facial feature information does not match the pet feature information, the number of objects corresponding to facial objects with different facial feature information in the video frame sequence is counted, and the carrying area information of the object is extracted from the non-facial feature information; the carrying volume is determined based on the carrying area information and the frame area information of the video frame. If it is determined that the number of objects is equal to or less than the first preset number of objects, a second control command is generated, and the front door of the vehicle is automatically opened based on the second control command. Determine whether the volume of the load is greater than the preset load volume; If it is determined that the volume of the cargo is greater than the preset cargo volume, a third control command is generated, and the trunk of the vehicle is automatically opened based on the third control command. Obtain the video frame sequence of the trunk; Based on the video frame sequence of the trunk, determine whether the usable size of the trunk exceeds the preset usable size; If it is determined that the usable size of the trunk exceeds the preset usable size, a fourth control command is generated, and the rear door of the vehicle is automatically opened based on the fourth control command. In addition, a first alarm prompt is generated and sent to the alarm device on the vehicle and / or the terminal of the target face object to provide an alarm prompt. If it is determined that the volume of the cargo is equal to or less than the preset cargo volume, the user habit control operation result that matches the number of objects is searched from the user vehicle usage habit information; wherein, the user vehicle usage habit information includes the correspondence between the user habit control operation result and the number of users.

2. The method according to claim 1, characterized in that, The step of performing face object detection and tracking and non-face object detection on the video frame sequence to obtain facial feature information of the face objects, trajectory distance information of the same face object, and non-face feature information of the non-face objects includes: The video frame sequence is subjected to face object detection using a pre-trained first detection model to obtain the face feature information of the face object; Track each video frame including the face object to obtain the trajectory distance information of the same face object; The video frame sequence is subjected to non-face object detection using a pre-trained second detection model to obtain the non-face feature information of the non-face objects.

3. The method according to claim 1, characterized in that, After generating a first control command and automatically opening the front and rear doors of the vehicle based on the first control command, the method further includes: Determine whether the volume of the load is greater than the preset load volume; If it is determined that the volume of the cargo is greater than the preset cargo volume, the step of generating a third control command and automatically opening the trunk of the vehicle based on the third control command is executed. Obtain the video frame sequence of the trunk; Based on the video frame sequence of the trunk, determine whether the usable size of the trunk exceeds the preset usable size; Upon determining that the usable size of the trunk exceeds the preset usable size, the system generates a first alarm notification and sends the first alarm notification to the alarm device on the vehicle and / or the terminal of the target face object to issue an alarm notification.

4. The method according to claim 1, characterized in that, After generating corresponding control commands based on the facial feature information, the user's driving habit information, and the non-facial feature information to automatically open the vehicle doors and / or trunk, the method further includes: Obtain the in-vehicle video frame sequence of the face object being transported; Based on the in-vehicle video frame sequence, it is determined whether all the facial figures riding in the vehicle are wearing seat belts; If it is determined that all the facial figures in the vehicle are wearing seat belts, a first closing command is generated to automatically close the open doors and / or trunk of the vehicle.

5. The method according to claim 4, characterized in that, Before generating corresponding control commands based on the facial feature information, the user's driving habit information, and the non-facial feature information to automatically open the vehicle doors and / or trunk, the method further includes: Receive the distance between the obstacle and the vehicle; Determine whether the distance to the obstacle is less than a preset distance; If the distance to the obstacle is determined to be less than the preset distance, a vehicle moving command is generated, and the vehicle is parked from the initial position to a safe position based on the vehicle moving command.

6. The method according to claim 5, characterized in that, After generating corresponding control commands based on the facial feature information, the user's driving habit information, and the non-facial feature information to automatically open the vehicle doors and / or trunk, the method further includes: Determine whether the trajectory distance information is not within the preset approach distance range and exceeds the maximum departure distance; If it is determined that the trajectory distance information is not within the preset approach distance range but does not exceed the maximum departure distance, a second alarm prompt is generated according to a preset time interval, and the second alarm prompt is sent to the alarm device on the vehicle and / or the terminal of the target face object to provide an alarm prompt; Calculate the total duration of the second alarm notification; Determine whether the total duration of the prompts exceeds the preset total duration; If the total duration of the prompt exceeds the preset total duration, or if the trajectory distance information exceeds the maximum value of the departure distance, a second closing command is generated to automatically close the open doors and / or trunk of the vehicle, and to park the vehicle from the safe position to the initial position.

7. The method according to claim 6, characterized in that, The method further includes: Count the number of times the second close instruction is generated; When the number of instructions reaches the preset number of instructions, the maximum approach distance in the preset approach distance range is adjusted based on the lower limit distance of the vehicle.

8. The method according to claim 1, characterized in that, The method further includes: Receive a temporary authorization instruction issued by the terminal using the target face object; Based on the temporary authorization instruction, a temporary face image of the temporary face object is acquired; The temporary facial image is sent to the user terminal; Receive temporary permissions for the temporary face object issued by the user terminal; wherein, the temporary permissions include the operation permissions to control the vehicle doors and / or trunk, and the duration of the temporary permissions; The temporary face object performs opening control operations on the vehicle doors and / or trunk based on the temporary permissions.

9. The method according to claim 1, characterized in that, The method further includes: Receive a temporary face image of the target face object sent by the terminal using the target face object; If temporary facial feature information corresponding to the temporary facial image is identified from the facial feature information, the video frame including the temporary facial object is sent from the video frame sequence to the user terminal; Receive temporary permissions for the temporary face object issued by the user terminal; wherein, the temporary permissions include the operation permissions to control the vehicle doors and / or trunk, and the duration of the temporary permissions; The temporary face object performs opening control operations on the vehicle doors and / or trunk based on the temporary permissions.

10. The method according to any one of claims 8 or 9, characterized in that, The method further includes: Collect a sequence of video frames showing the operation of the temporary face object in controlling the vehicle; Based on the operation video frame sequence, when the temporary face object is identified as having completed its operation on the vehicle, or when the temporary face object's operation on opening the vehicle reaches the duration of the temporary permission, the temporary permission is cleared and an operation completion notification is generated, and the operation completion notification is sent to the user terminal.

11. The method according to claim 10, characterized in that, The method further includes: When the operation of the temporary face object on the vehicle exceeds the operation authority based on the operation video frame sequence, a third alarm prompt is generated and sent to the alarm device on the vehicle and / or the user terminal to provide an alarm prompt.

12. A vehicle control device, characterized in that, The device is applied to a vehicle control center, and the device is applied to the method of claim 1, comprising: The acquisition module is used to acquire user driving habit information and a video frame sequence containing a face when the vehicle is parked; wherein, the user driving habit information is the result of user habit control operation obtained by counting the most control operations performed by the user on the vehicle within a preset statistical time period. The detection and tracking module is used to perform face object detection and tracking and non-face object detection on the video frame sequence, and obtain the face feature information of the face object, the trajectory distance information of the same face object, and the non-face feature information of the non-face object; wherein, the trajectory distance information is a video frame sub-sequence containing the same detection box identifier; The judgment module is used to determine whether the trajectory distance information is within a preset approach distance range, and whether there is at least one facial feature information in the facial feature information that matches the target facial feature information of a pre-stored target facial object; wherein, the target facial object is a user with the authority to operate the vehicle; The control module is activated to generate corresponding control commands based on the facial feature information, the user's driving habit information, and the non-facial feature information when it is determined that the trajectory distance information is within the preset approach distance range and at least one facial feature information in the facial feature information matches the target facial feature information, so as to automatically control the opening of the vehicle doors and / or trunk.

13. A vehicle, characterized in that, The vehicle is equipped with a vehicle control center, wherein the vehicle control center is used to execute the vehicle control method according to any one of claims 1 to 11.

14. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the vehicle control method according to any one of claims 1 to 11.

Images