Vehicle control method and apparatus, electronic device and storage medium

Abstract

Provided in the present disclosure are a vehicle control method and apparatus, an electronic device and a storage medium. The method comprises: analyzing vehicle video images of the inside of a target vehicle to determine a deflection magnitude of the head pose of a target person; according to the deflection magnitude and a preset first magnitude threshold, identifying a target head movement of the target person; and, on the basis of the target head movement of the target person, controlling the target vehicle. The method of the present disclosure achieves vehicle control by means of video identification, improving the identification accuracy, and simplifying an operation process.

Description

A method, apparatus, electronic device, and storage medium for vehicle control

[0001] Cross-references to related applications

[0002] This disclosure claims priority to Chinese Patent Application No. 202411036123.0, filed on July 31, 2024, entitled "A method, apparatus, electronic device and storage medium for vehicle control", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of vehicle control technology, and more specifically, to a method, apparatus, electronic device, and storage medium for vehicle control. Background Technology

[0004] With the development of automotive electronics technology, vehicles are becoming increasingly intelligent. Current technologies primarily utilize physical button control, touchscreen control, and voice control. Physical button and touchscreen controls require the driver to look at the control panel, which can interfere with normal driving and is difficult for other passengers to operate. Voice control is easily affected by noise during driving, leading to reduced recognition accuracy.

[0005] Summary of the Invention

[0006] In view of this, the purpose of this disclosure is to provide a method, apparatus, electronic device and storage medium for vehicle control to overcome the problems in the prior art.

[0007] In a first aspect, embodiments of this disclosure provide a method for vehicle control, the method comprising:

[0008] Analyze the video images inside the target vehicle to determine the degree of head deflection of the target person.

[0009] Based on the deflection amplitude and the preset first amplitude threshold, the target head movement of the target person is identified;

[0010] The target vehicle is controlled based on the target person's head movements.

[0011] In some technical solutions disclosed herein, the above-mentioned analysis of vehicle video images inside the target vehicle to determine the deflection range of the target person's head posture includes:

[0012] Analyze the vehicle video images inside the target vehicle to determine the target human face contained in each frame of the vehicle video images;

[0013] By comparing the target face positions contained in each group of images, the extreme values ​​of the target person's head posture in that group of images are determined; where each group of images is determined according to the acquisition order and a preset number;

[0014] Based on the extreme values ​​of the target person's head posture in each group of images, the deflection range of the target person's head posture is determined.

[0015] In some technical solutions disclosed herein, the above-mentioned analysis of vehicle video images inside the target vehicle to determine the deflection range of the target person's head posture includes:

[0016] Analyze the video images inside the target vehicle to determine the pitch angle, yaw angle, and roll angle changes of the target person's head posture;

[0017] The step of identifying the target head movement of the target person based on the deflection amplitude and a preset first amplitude threshold includes:

[0018] Based on the pitch angle change range, yaw angle change range, roll angle change range, and the corresponding first amplitude threshold, multiple target head movements of the target personnel are identified.

[0019] In some of the technical solutions disclosed herein, the aforementioned target head movements include nodding, horizontal head shaking, and other head shaking movements;

[0020] The step of identifying multiple target head movements of the target personnel based on the pitch angle change amplitude, yaw angle change amplitude, roll angle change amplitude, and corresponding first amplitude threshold includes:

[0021] Based on the pitch angle change range and the first pitch angle threshold, the nodding action of the target person is identified;

[0022] Based on the yaw angle change range and the first yaw range threshold, the horizontal head-shaking motion of the target person is identified;

[0023] Based on the roll angle change range and the first roll range threshold, other head-shaking movements of the target person are identified.

[0024] In some of the technical solutions disclosed herein, the above methods also include:

[0025] Analyze the video images inside the target vehicle to determine the head deflection time of the target person;

[0026] The step of identifying the target head movement of the target person based on the deflection amplitude and a preset first amplitude threshold includes:

[0027] Based on the deflection amplitude and a preset first amplitude threshold, the deflection time and a preset time threshold, the target head movement of the target person is identified.

[0028] In some technical solutions disclosed herein, the above-mentioned analysis of vehicle video images inside the target vehicle to determine the deflection range of the target person's head posture includes:

[0029] The video images inside the target vehicle were analyzed to identify the head postures of multiple candidates.

[0030] The target person is determined from the candidates based on their head posture.

[0031] The deflection range of the target person's head posture is determined based on the extreme position of the target person's head posture.

[0032] In some technical solutions disclosed herein, the target person is determined from the candidates based on their head posture:

[0033] Based on the head posture of the candidates, the first candidate who makes the start control action is selected as the target person.

[0034] In some technical solutions disclosed herein, if there are multiple candidates for the first position, the method further includes:

[0035] The target person is determined from the first candidates based on the area proportion of each candidate in the image and the shooting angle.

[0036] In some of the technical solutions disclosed herein, the above methods also include:

[0037] The target person is detected as the second candidate person who initiates the control action after the control action is terminated.

[0038] The second candidate will be designated as the new target candidate.

[0039] The control of the target vehicle based on the target person's head movements includes:

[0040] The target vehicle is controlled based on the new target person's new head movements.

[0041] In some technical solutions disclosed herein, the above-mentioned identification of the target person's head movement based on the deflection amplitude and a preset first amplitude threshold includes:

[0042] Based on the deflection amplitude and a preset first amplitude threshold, the initial head movement of the target person is identified;

[0043] By performing motion filtering on the initial head movements of the target person, the filtered head movements of the target person are obtained.

[0044] In some technical solutions disclosed herein, the above-mentioned method of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0045] By analyzing the monitoring video images outside the target vehicle, the driving scenario of the target vehicle is determined;

[0046] Based on the driving scenario of the target vehicle, the initial head motion is filtered to obtain the filtered target head motion.

[0047] In some technical solutions disclosed herein, the above-mentioned method of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0048] Based on a preset second amplitude threshold, the initial head motion is filtered to obtain the filtered target head motion.

[0049] In some technical solutions disclosed herein, the above-mentioned method of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0050] Based on a preset second amplitude threshold and a second time threshold, the initial head motion is filtered to obtain the filtered target head motion.

[0051] In some technical solutions disclosed herein, the above-mentioned method of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0052] By analyzing the historical control data of the target vehicle, the inferred head movement was determined;

[0053] Based on the inferred head movements, the initial head movements are filtered to obtain the filtered target head movements.

[0054] In some technical solutions disclosed herein, the above-mentioned method of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0055] Based on the control instructions of other control methods, the initial head motion is filtered to obtain the filtered target head motion.

[0056] In some technical solutions disclosed herein, the above-mentioned method of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0057] Based on the set driving preferences, the initial head movements of the target person are filtered to obtain the filtered head movements of the target person.

[0058] In some technical solutions disclosed herein, the above-mentioned method of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0059] The initial head movements were filtered using multiple filtering methods to obtain various filtering results.

[0060] The target head motion is obtained based on the various filtering results.

[0061] Secondly, embodiments of this disclosure provide a vehicle control device, the device comprising:

[0062] The determination module is used to analyze the video images of the target vehicle to determine the deflection range of the target person's head posture.

[0063] The recognition module is used to identify the target head movement of the target person based on the deflection amplitude and a preset first amplitude threshold.

[0064] The control module is used to control the target vehicle based on the target person's head movements.

[0065] Thirdly, embodiments of this disclosure provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the vehicle control method described above.

[0066] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the vehicle control method described above.

[0067] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0068] The method disclosed herein includes analyzing a video image of a vehicle inside a target vehicle to determine the deflection amplitude of the head posture of a target person; identifying the target head movement of the target person based on the deflection amplitude and a preset first amplitude threshold; and controlling the target vehicle based on the target head movement of the target person.

[0069] The method disclosed herein controls vehicles through video recognition, which improves recognition accuracy and simplifies the operation process.

[0070] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0071] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this disclosure and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0072] Figure 1 shows a schematic flowchart of a vehicle control method provided in an embodiment of this disclosure;

[0073] Figure 2 shows a schematic diagram of head movement recognition provided by an embodiment of the present disclosure;

[0074] Figure 3 shows a schematic diagram of a vehicle control device provided in an embodiment of this disclosure;

[0075] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this disclosure are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this disclosure. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this disclosure illustrate operations implemented according to some embodiments of this disclosure. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this disclosure, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0077] Furthermore, the described embodiments are merely some, not all, of the embodiments of this disclosure. The components of the embodiments of this disclosure typically described and illustrated in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the drawings is not intended to limit the scope of the claimed disclosure, but merely to illustrate selected embodiments of the disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0078] It should be noted that the term "comprising" will be used in the embodiments of this disclosure to indicate the presence of the features subsequently declared, but does not exclude the addition of other features.

[0079] Existing in-vehicle human-machine interaction methods can be mainly divided into the following categories:

[0080] Physical buttons: Physical buttons are a traditional method of human-machine interaction in vehicles. Users can perform corresponding operations by pressing buttons, such as starting the vehicle and adjusting the air conditioning. Physical buttons are characterized by high reliability and ease of operation, but they also have limitations such as limited functionality and difficulty in expansion.

[0081] Touchscreen interaction: Touchscreen interaction is one of the most common human-computer interaction methods, primarily relying on the in-vehicle touchscreen. Users can perform corresponding operations by touching buttons or icons on the screen. Touchscreen interaction is intuitive and easy to use, but it also has certain limitations; for example, using the touchscreen while driving may affect driving safety.

[0082] Voice interaction: Voice interaction is a safer and more convenient way for humans to interact with machines. Users can control the vehicle using voice commands, such as adjusting volume, navigation, and playing music. The advantage of voice interaction is that it frees up the driver's hands and reduces the driver's workload. However, voice interaction also has some problems, such as poor recognition rate and susceptibility to environmental noise.

[0083] The vehicle control method based on head movement recognition provided in this disclosure can significantly improve a series of shortcomings of the prior art, including but not limited to the following aspects:

[0084] A natural and intuitive interactive experience: Head movements are a natural and intuitive part of everyday human communication. Compared to touchscreens or voice interaction, recognizing head movements through cameras can more closely resemble people's natural interaction methods. Drivers and passengers can express their intentions with simple nods or shakes of the head, without the need for specialized touch operations or voice commands, making the interaction more natural and intuitive.

[0085] Reducing driver distraction and fatigue: Compared to touchscreens, head movement recognition can reduce driver distraction. Touchscreens require drivers to take their eyes off the road, while head movement recognition can be done with minimal impact on the driver's focus on the road ahead, thus reducing traffic risks associated with distraction and alleviating driver fatigue during long drives.

[0086] Suitable for noisy environments: Voice interaction may be interfered with in noisy in-car environments, leading to misrecognition or difficulty in executing commands. However, by recognizing head movements through a camera, a more reliable interaction method can be provided in noisy environments, unaffected by background noise, thereby improving the stability and reliability of the interaction.

[0087] Convenience of multitasking: Head movement recognition allows drivers to perform multitasking more easily, such as adjusting music, navigation, or in-vehicle settings, without stopping the vehicle or focusing on the touchscreen, improving the ease of interaction.

[0088] The following description is based on embodiments. Figure 1 shows a flowchart of a vehicle control method provided by an embodiment of this disclosure, wherein the method includes steps S101-S103; specifically:

[0089] S101. Analyze the video images of the target vehicle to determine the deflection range of the target person's head posture.

[0090] S102. Based on the deflection amplitude and the preset first amplitude threshold, identify the target head movement of the target person;

[0091] S103. Control the target vehicle based on the target person's head movement.

[0092] The following describes some embodiments of this disclosure in detail. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0093] In the first embodiment, this disclosure identifies the head movements of target occupants by acquiring vehicle video images, and then controls the target vehicle based on these head movements. The vehicle video images are captured from inside the target vehicle, primarily focusing on the heads of occupants. Specifically, a video image acquisition device, such as a dashcam or camera, is installed inside the vehicle. The image acquisition device captures images in real time, facing the seats inside the vehicle. After acquiring the current frame, face recognition is immediately performed on that frame to determine if it contains a face. If the frame does not contain a face, it is not used for vehicle control. If the frame contains a face, face detection continues to determine if the face has performed a preset target head movement.

[0094] The face recognition process here uses a pre-defined deep learning model. The current frame image is input into the deep learning model, and the output is whether the frame image contains a face. If a face is found, the position of the detection box containing the face is also output. The position of the detection box is specifically identified using the coordinates, width, and height of a point. In other words, after recognizing a face, the position and size of the face are also output.

[0095] Since this embodiment uses an image acquisition device, the captured image is a two-dimensional image. To detect whether there is target head movement in the frame, the two-dimensional image needs to be converted into a head pose in three-dimensional space. Head pose refers to the direction of a person's head relative to the camera. This embodiment uses a deep learning method, taking a two-dimensional image containing a face and the position and size of the face as input to a deep learning model, and outputting a head pose. After identifying the target person's head pose, the deflection amplitude of the target person's head pose is determined based on the head pose in each frame of the vehicle video.

[0096] After determining the deflection amplitude of the target person's head posture, it is also necessary to identify whether the target person has performed a target head movement. This embodiment sets a first amplitude threshold. By comparing the deflection amplitude with this first amplitude threshold, it is determined whether the target person has performed a target head movement. This first amplitude threshold can be set by the user or obtained through analysis of acquired historical video images. Specifically, the comparison process is as follows: when the deflection amplitude is greater than or equal to the first amplitude threshold, the target person is considered to have performed a target head movement; when the deflection amplitude is less than the first amplitude threshold, the target person is considered not to have performed a target head movement. For example, by analyzing historical video images, the maximum amplitude of the target person's driving movement is a1. Here, the first amplitude threshold can be set to a2 (a2 is greater than or equal to a1). By setting a first amplitude threshold greater than or equal to the amplitude of normal driving movements, control actions and driving actions can be distinguished.

[0097] After identifying the target person's head movements, the target vehicle is controlled based on those head movements. Specifically, control relationships can be pre-set, specifying which head movement corresponds to which control action. For example, the first head movement corresponds to turning on the radio, and the second head movement corresponds to turning off the radio. When the first head movement is detected, the target vehicle is controlled to turn on the radio; when the second head movement is detected, the target vehicle is controlled to turn off the radio.

[0098] This invention discloses real-time and accurate target personnel behavior recognition: it captures the head movements of target personnel in real time through an in-vehicle camera and uses the proposed algorithm to accurately recognize the movements, thereby achieving an instant response to the head movements of target personnel.

[0099] Low false recognition rate: By analyzing head movement deflection and combining it with a filtering strategy based on the first amplitude threshold, the probability of normal driving actions being misidentified as target head movements can be effectively reduced, thereby improving the accuracy and reliability of the system.

[0100] Personalized settings and optimizations: The system can be personalized and optimized according to the habits and preferences of different target users, improving the system's applicability and user experience, and enhancing the target users' acceptance of the system and their comfort in using it.

[0101] In the second embodiment, when analyzing the vehicle video images inside the target vehicle to determine the deflection range of the target person's head posture, it is necessary to first analyze the vehicle video images to determine whether each frame contains the target face. Here, the face recognition process uses a pre-set deep learning model. Each frame is input into the deep learning model, and the model outputs whether the frame contains a face. If a face is present, the position of the detection box containing the face is also output. The position of the detection box is specifically identified using the coordinates, width, and height of a point. That is, after recognizing a face, the position and size of the face are also output.

[0102] After obtaining the position of the target face in each frame of the image, this embodiment of the disclosure needs to determine the extreme values ​​of the target face position in order to determine the deflection range of the target person's head posture. When determining the extreme values ​​of the target face position, it is necessary to compare the target face position in each frame of the image. Considering that this disclosure pertains to the shooting process, this embodiment of the disclosure adopts a sliding window method for comparison. Here, the window is a group, and each group of images is determined according to the acquisition order and a preset number. The extreme values ​​of the position within that window are determined according to a preset window size. When a new image is acquired, the window is slid to determine the extreme values ​​of the position in the next window. The extreme values ​​of the position in these two windows are compared to obtain the extreme values ​​of the position in those two windows. This process is repeated to obtain the extreme values ​​of the position in the vehicle video image.

[0103] After obtaining the extreme position values ​​in the vehicle video image, the deflection amplitude of the target person's head posture can be obtained by converting the two-dimensional image into a head pose in three-dimensional space. Based on the deflection amplitude and a preset first amplitude threshold, the target person's target head movement is identified; and the target vehicle is controlled based on the target person's target head movement.

[0104] In the third embodiment, when analyzing the vehicle video image inside the target vehicle to determine the deflection amplitude of the target person's head posture, it is necessary to convert the two-dimensional image captured by the image acquisition device into a three-dimensional head posture. This yields a three-dimensional posture deflection angle parameter. The deflection angle parameter is an Euler rotation angle including pitch, yaw, and roll angles. Based on the position extrema in the two-dimensional image, the pitch angle change amplitude, yaw angle change amplitude, and roll angle change amplitude of the head posture can be obtained. When determining the target person's target action, the pitch angle change amplitude, yaw angle change amplitude, and roll angle change amplitude are compared with the corresponding first amplitude threshold to identify multiple target head actions of the target person.

[0105] The specific identification process is shown in Figure 2. S201: Identify the nodding action of the target person based on the pitch angle change range and the first pitch range threshold.

[0106] S202. Identify the horizontal head-shaking motion of the target person based on the yaw angle change amplitude and the first yaw amplitude threshold.

[0107] S203. Based on the roll angle change range and the first roll range threshold, identify other head-shaking movements of the target person.

[0108] In practical implementation, the head posture of the target person can be processed within a preset coordinate system. In this coordinate system, the horizontal axis is the X-axis, the vertical axis is the Y-axis, and the longitudinal axis is the Z-axis. When the face position is in the positive direction of the axis (including the X, Y, and Z axes), it is represented by a positive number; when it is in the negative direction, it is represented by a negative number, and it is directional. For example, if analyzing the video image of the target vehicle, and obtaining extreme values ​​of the face position on the X-axis as -3 (from negative to positive), +5 (from negative to positive), and -3 (from positive to negative), then the obtained head posture of the target face is from left to right and back to left. Since the position threshold in this disclosure is directional, the obtained head posture is also directional. Therefore, when setting the amplitude threshold, amplitude thresholds for different directions need to be set. Alternatively, the amplitude threshold in this disclosure is just a data point. When making a comparison, "+" and "-" (positive and negative identifiers) should be added before the data point respectively.

[0109] After identifying the target person's head movement, the target vehicle is controlled based on the target person's head movement.

[0110] In the fourth embodiment, to improve the accuracy of determining the target person's head movement when analyzing the vehicle video image, this embodiment also determines the deflection time of the target person's head posture. The specific deflection time can be determined based on the acquisition time of the vehicle video image. The process of determining the target head movement is based on the deflection amplitude and a preset first amplitude threshold, the deflection time, and a preset time threshold.

[0111] In other words, when determining the target head movement in this embodiment, each head posture must not only perform an action with a preset amplitude, but the duration of that action must also meet a preset duration (determined based on a preset time threshold). If the amplitude of an action exceeds the first amplitude threshold, but the duration does not meet the preset duration, this embodiment does not consider it a target head movement, i.e., it is not considered a control action. If the amplitude of an action does not exceed the first amplitude threshold, but the duration meets the preset duration, this embodiment does not consider it a target head movement, i.e., it is not considered a control action. If the amplitude of an action exceeds the first amplitude threshold, but the duration meets the preset duration, this embodiment considers it a target head movement, i.e., it is considered a control action.

[0112] After identifying the target person's head movement, the target vehicle is controlled based on the target person's head movement.

[0113] In practical implementation, head nodding recognition involves the driver's head moving up and down around the horizontal axis (X-axis), with the pitch angle changing within a range greater than the head nodding angle threshold, and the duration falling within the head nodding time threshold range. Horizontal head shaking recognition involves the driver's head swaying left and right around the vertical axis (Y-axis), with the yaw angle changing within a range greater than the head shaking angle threshold, and the duration falling within the head shaking time threshold range. Other (Indian-style) head shaking recognition involves the driver's head rotating left and right around the longitudinal axis (Z-axis), with the roll angle changing within a range greater than the Indian-style head shaking threshold, and the duration falling within the Indian-style head shaking time threshold range.

[0114] In the fifth embodiment, the target vehicle includes one or more passengers in addition to the driver. Control of the target vehicle includes driving control and other controls. Driving control refers to controls that affect driving, such as vehicle light control. Other controls are controls other than driving control, such as radio control. The personnel capable of performing these different types of controls vary. For driving control, only the driver can perform it. For other controls, passengers can also perform them. For different types of controls, this embodiment provides multiple shooting angles. The specific shooting angle can be determined by the number of people inside the vehicle or set manually.

[0115] In this embodiment of the disclosure, when analyzing vehicle video images inside a target vehicle, there may be situations where multiple candidate faces are identified. In such cases, it is necessary to determine the target person who can actually control the vehicle. That is, this embodiment of the disclosure needs to determine the target person from among multiple candidate individuals.

[0116] The process of identifying the target person can involve recognizing the head posture of each candidate, and designating the first candidate to initiate the control action as the target person. For example, the initiation control action here could be a head-shaking motion of a preset amplitude. If one of the five people in the vehicle initiates the action, that person is designated as the target person, and the vehicle is controlled accordingly. If two or more people initiate the action simultaneously, designating all of them as the target person could lead to control confusion. Therefore, this embodiment requires further screening.

[0117] The selection criteria are the area proportion and shooting angle of each of the first candidate personnel in the image. That is, when multiple candidate personnel simultaneously initiate control actions, this embodiment of the disclosure will select the candidate with the largest area proportion and the best shooting angle in each frame as the target control personnel. Because the candidate with the largest area proportion and the best shooting angle is closer to the image acquisition device, this embodiment of the disclosure considers this candidate as the target personnel controlling the target vehicle.

[0118] In the sixth embodiment, for the situation in the fifth embodiment where there are multiple people in the vehicle, to improve the smoothness of vehicle control, there can be multiple target personnel for other controls; that is, multiple personnel in the vehicle can take turns being controlled by other controls. The specific control flow is as follows: the current target personnel ends their current control, and then a new target personnel takes over control. The current target personnel ends control by performing an end-control action. After detecting that the current target personnel has performed an end-control action, other controls are in an unattended state. At this time, the vehicle personnel are detected again. If a new target personnel performs a start-control action, the new target personnel will take over control of the other controls. Here, the new target personnel can be the previous target personnel or other personnel in the vehicle.

[0119] In the seventh embodiment, in order to ensure the accuracy of vehicle control and eliminate interference from non-control actions, after identifying the initial head movement of the target person based on the deflection amplitude and the preset first amplitude threshold, this embodiment needs to filter the initial head movement of the target person to obtain the filtered target head movement.

[0120] When filtering initial head movements, this embodiment of the disclosure sets up a variety of filtering methods, which can be used individually or in combination. For example, the filtering methods include filtering based on driving scenario, filtering based on a preset second amplitude threshold, filtering based on a preset second amplitude threshold and second time threshold, filtering based on inferred head movements, filtering based on control commands from other control methods, and filtering based on set driving preferences, etc. The combined usage methods include: combining the driving scenario filtering method with the preset second amplitude threshold filtering method; combining the driving scenario filtering method, the preset second amplitude threshold and second time threshold filtering method, and the inferred head movement filtering method; combining the preset second amplitude threshold and second time threshold filtering method, the inferred head movement filtering method, the control command filtering method of other control methods, and the set driving preference filtering method; combining the driving scenario filtering method, the preset second amplitude threshold filtering method, the inferred head movement filtering method, the control command filtering method of other control methods, and the set driving preference filtering method; combining the driving scenario filtering method, the preset second amplitude threshold and second time threshold filtering method, the inferred head movement filtering method, the control command filtering method of other control methods, and the set driving preference filtering method.

[0121] For the driving scenario-based filtering method: This embodiment of the disclosure acquires monitoring video images outside the target vehicle, then analyzes the monitoring video images to determine the current driving scenario of the target vehicle. For example, driving scenario, parking scenario, etc. Each driving scenario has corresponding scenario actions and action amplitude thresholds. The initial head movements are filtered based on the scenario actions and action amplitude thresholds in each scenario, filtering out non-control actions to obtain the filtered target head movements.

[0122] For the filtering method based on a preset second amplitude threshold and a second time threshold, the embodiments of this disclosure can manually determine the second amplitude threshold and the second time threshold according to factors such as personal habits. Non-controlling actions located within the second amplitude threshold and the second time threshold are filtered out to obtain the filtered target head action.

[0123] Regarding the filtering method based on the predicted head movements, this embodiment of the disclosure determines the predicted head movements by analyzing the historical control data of the target vehicle. This historical control data can be obtained using the method of this disclosure or through other control methods in the prior art. By analyzing this historical control data, the control method for the target person within a given time period can be determined. Based on the correlation between the preset control method and head posture, the predicted head movements of the target person can be determined. If the initial head movements include the predicted head movements, then these movements are used as control actions to control the target vehicle.

[0124] For control command filtering based on other control methods, after receiving the initial head movement, if the user knows it was caused by a mis-touch, they can input control commands from other control methods to filter out the initial head movement. These other control methods can be input through the control method described in this disclosure, or through other methods. For example, the user can filter out the initial head movement by performing the initial head movement again in the opposite direction, indicating a filtering control command.

[0125] For the set driving preference filtering method, users can use pre-set driving preferences (which include multiple head movements or different head movements in different scenarios) to filter the initial head movements.

[0126] Figure 3 shows a schematic diagram of a vehicle control device provided in an embodiment of this disclosure, the device comprising:

[0127] The determination module is used to analyze the video images of the target vehicle to determine the deflection range of the target person's head posture.

[0128] The recognition module is used to identify the target head movement of the target person based on the deflection amplitude and a preset first amplitude threshold.

[0129] The control module is used to control the target vehicle based on the target person's head movements.

[0130] The analysis of the vehicle video images inside the target vehicle to determine the deflection range of the target person's head posture includes:

[0131] Analyze the vehicle video images inside the target vehicle to determine the target human face contained in each frame of the vehicle video images;

[0132] By comparing the target face positions contained in each group of images, the extreme values ​​of the target person's head posture in that group of images are determined; where each group of images is determined according to the acquisition order and a preset number;

[0133] Based on the extreme values ​​of the target person's head posture in each group of images, the deflection range of the target person's head posture is determined.

[0134] The analysis of the vehicle video images inside the target vehicle to determine the deflection range of the target person's head posture includes:

[0135] Analyze the video images inside the target vehicle to determine the pitch angle, yaw angle, and roll angle changes of the target person's head posture;

[0136] The step of identifying the target head movement of the target person based on the deflection amplitude and a preset first amplitude threshold includes:

[0137] Based on the pitch angle change range, yaw angle change range, roll angle change range, and the corresponding first amplitude threshold, multiple target head movements of the target personnel are identified.

[0138] The target head movements include nodding, horizontal head shaking, and other head shaking movements;

[0139] The step of identifying multiple target head movements of the target personnel based on the pitch angle change amplitude, yaw angle change amplitude, roll angle change amplitude, and corresponding first amplitude threshold includes:

[0140] Based on the pitch angle change range and the first pitch angle threshold, the nodding action of the target person is identified;

[0141] Based on the yaw angle change range and the first yaw range threshold, the horizontal head-shaking motion of the target person is identified;

[0142] Based on the roll angle change range and the first roll range threshold, other head-shaking movements of the target person are identified.

[0143] The device is also used to: analyze the vehicle video images inside the target vehicle to determine the deflection time of the target person's head posture;

[0144] The step of identifying the target head movement of the target person based on the deflection amplitude and a preset first amplitude threshold includes:

[0145] Based on the deflection amplitude and a preset first amplitude threshold, the deflection time and a preset time threshold, the target head movement of the target person is identified.

[0146] The analysis of the vehicle video images inside the target vehicle to determine the deflection range of the target person's head posture includes:

[0147] The video images inside the target vehicle were analyzed to identify the head postures of multiple candidates.

[0148] The target person is determined from the candidates based on their head posture.

[0149] The deflection range of the target person's head posture is determined based on the extreme position of the target person's head posture.

[0150] The target person is determined from the candidates based on their head posture.

[0151] Based on the head posture of the candidates, the first candidate who makes the start control action is selected as the target person.

[0152] If there are multiple candidates, the target person is determined from the candidates based on the area ratio of each candidate in the image and the shooting angle.

[0153] The target person is detected as the second candidate person who initiates the control action after the control action is terminated.

[0154] The second candidate will be designated as the new target candidate.

[0155] The control of the target vehicle based on the target person's head movements includes:

[0156] The target vehicle is controlled based on the new target person's new head movements.

[0157] The step of identifying the target head movement of the target person based on the deflection amplitude and a preset first amplitude threshold includes:

[0158] Based on the deflection amplitude and a preset first amplitude threshold, the initial head movement of the target person is identified;

[0159] By performing motion filtering on the initial head movements of the target person, the filtered head movements of the target person are obtained.

[0160] The step of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0161] By analyzing the monitoring video images outside the target vehicle, the driving scenario of the target vehicle is determined;

[0162] Based on the driving scenario of the target vehicle, the initial head motion is filtered to obtain the filtered target head motion.

[0163] The step of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0164] Based on a preset second amplitude threshold, the initial head motion is filtered to obtain the filtered target head motion.

[0165] The step of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0166] Based on a preset second amplitude threshold and a second time threshold, the initial head motion is filtered to obtain the filtered target head motion.

[0167] The step of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0168] By analyzing the historical control data of the target vehicle, the inferred head movement was determined;

[0169] Based on the inferred head movements, the initial head movements are filtered to obtain the filtered target head movements.

[0170] The step of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0171] Based on the control instructions of other control methods, the initial head motion is filtered to obtain the filtered target head motion.

[0172] The step of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0173] Based on the set driving preferences, the initial head movements of the target person are filtered to obtain the filtered head movements of the target person.

[0174] The step of filtering the initial head movements of the target person to obtain the filtered target head movements includes:

[0175] The initial head movements were filtered using multiple filtering methods to obtain various filtering results.

[0176] The target head motion is obtained based on the various filtering results.

[0177] As shown in FIG4, an embodiment of the present disclosure provides an electronic device for executing the vehicle control method of the present disclosure. The device includes a memory, a processor, a bus, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the steps of the vehicle control method described above.

[0178] Specifically, the aforementioned memory and processor can be general-purpose memory and processor, without any specific limitations. When the processor runs the computer program stored in the memory, it can execute the aforementioned vehicle control method.

[0179] Corresponding to the vehicle control method in this disclosure, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the vehicle control method described above.

[0180] Specifically, the storage medium can be a general-purpose storage medium, such as a removable disk or hard disk, and when the computer program on the storage medium is run, it can execute the vehicle control method described above.

[0181] In the embodiments provided in this disclosure, it should be understood that the disclosed systems and methods can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings or direct couplings or communication connections may be through some communication interfaces; indirect couplings or communication connections between systems or units may be electrical, mechanical, or other forms.

[0182] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0183] In addition, the functional units in the embodiments provided in this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0184] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0185] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0186] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure. All should be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims. Industrial applicability

[0187] The method disclosed herein includes analyzing a video image of a vehicle inside a target vehicle to determine the deflection amplitude of the head posture of a target person; identifying the target head movement of the target person based on the deflection amplitude and a preset first amplitude threshold; and controlling the target vehicle based on the target head movement of the target person.

[0188] The method disclosed herein controls vehicles through video recognition, which improves recognition accuracy and simplifies the operation process.

[0189] Furthermore, it is understood that the vehicle control method, apparatus, electronic device, and storage medium provided in this disclosure are reproducible and can be used in various industrial applications. For example, the vehicle control method, apparatus, electronic device, and storage medium provided in this disclosure can be used in the field of vehicle control technology.

Claims

1. A method of vehicle control, characterized by, The method comprises: analyzing a vehicle video image in a target vehicle to determine a deflection amplitude of a head posture of a target person; identifying a target head action of the target person according to the deflection amplitude and a preset first amplitude threshold value; controlling the target vehicle based on the target head action of the target person.

2. The method of claim 1, wherein, The analysis of the vehicle video image in the target vehicle to determine the deflection amplitude of the head posture of the target person comprises: analyzing the vehicle video image in the target vehicle to determine a target face contained in each frame of the vehicle video image; determining a position extreme value of the head posture of the target person in each group of images by comparing target face positions contained in each group of images, wherein each group of images is determined according to a preset number and a collection sequence; determining the deflection amplitude of the head posture of the target person according to the position extreme value of the head posture of the target person in each group of images.

3. The method of claim 1, wherein, The analysis of the vehicle video image in the target vehicle to determine the deflection amplitude of the head posture of the target person comprises: analyzing the vehicle video image in the target vehicle to determine a pitch angle variation amplitude, a yaw angle variation amplitude and a roll angle variation amplitude of the head posture of the target person. The identification of the target head action of the target person according to the deflection amplitude and the preset first amplitude threshold value comprises: identifying a plurality of target head actions of the target person according to the pitch angle variation amplitude, the yaw angle variation amplitude, the roll angle variation amplitude and corresponding first amplitude threshold values.

4. The method of claim 3, wherein, The target head action comprises a nodding action, a horizontal head shaking action and other head shaking actions. The identification of the plurality of target head actions of the target person according to the pitch angle variation amplitude, the yaw angle variation amplitude, the roll angle variation amplitude and the corresponding first amplitude threshold values comprises: identifying the nodding action of the target person according to the pitch angle variation amplitude and a first pitch amplitude threshold value; identifying the horizontal head shaking action of the target person according to the yaw angle variation amplitude and a first yaw amplitude threshold value; identifying the other head shaking action of the target person according to the roll angle variation amplitude and a first roll amplitude threshold value.

5. The method of claim 1, wherein, The method further comprises: analyzing the vehicle video image in the target vehicle to determine a deflection time of the head posture of the target person; The identification of the target head action of the target person according to the deflection amplitude and the preset first amplitude threshold value comprises: identifying the target head action of the target person according to the deflection amplitude and the preset first amplitude threshold value, the deflection time and a preset time threshold value.

6. The method of claim 1, wherein, The analysis of the vehicle video image in the target vehicle to determine the deflection amplitude of the head posture of the target person comprises: analyzing the vehicle video image in the target vehicle to identify head postures of a plurality of candidate persons; determining the target person from the candidate persons according to the head postures of the candidate persons; determining the deflection amplitude of the head posture of the target person according to the position extreme value of the head posture of the target person.

7. The method of claim 6, wherein, The determination of the target person from the candidate persons according to the head postures of the candidate persons comprises: According to the head posture of the candidate personnel, a first candidate personnel who makes a start control action is taken as the target personnel.

8. The method of claim 7, wherein, If the first candidate personnel is multiple, the method further comprises: According to the area proportion and the shooting angle of each first candidate personnel in the image, the target personnel is determined from the first candidate personnel.

9. The method of claim 7, wherein, The method further comprises: After the target personnel makes an end control action, a second candidate personnel who makes a start control action is detected; The second candidate personnel is taken as a new target personnel; The target vehicle is controlled based on the target head action of the target personnel, comprising: The target vehicle is controlled based on the new target head action of the new target personnel.

10. The method of claim 1, wherein, The target head action of the target personnel is identified according to the deflection amplitude and a preset first amplitude threshold, comprising: The initial head action of the target personnel is identified according to the deflection amplitude and a preset first amplitude threshold; The target head action is obtained by filtering the initial head action of the target personnel.

11. The method of claim 10, wherein, The target head action is obtained by filtering the initial head action of the target personnel, comprising: The driving scene of the target vehicle is determined by analyzing the monitoring video image outside the target vehicle; The initial head action is filtered according to the driving scene of the target vehicle to obtain the filtered target head action.

12. The method of claim 10, wherein, The target head action is obtained by filtering the initial head action of the target personnel, comprising: The initial head action is filtered based on a preset second amplitude threshold to obtain the filtered target head action.

13. The method of claim 10, wherein, The target head action is obtained by filtering the initial head action of the target personnel, comprising: The initial head action is filtered based on a preset second amplitude threshold and a second time threshold to obtain the filtered target head action.

14. The method of claim 10, wherein, The target head action is obtained by filtering the initial head action of the target personnel, comprising: The initial head action is filtered according to the speculative head action to obtain the filtered target head action. The target head action is obtained by filtering the initial head action of the target personnel, comprising:

15. The method of claim 10, wherein, The initial head action is filtered according to the control instruction of other control modes to obtain the filtered target head action. The target head action is obtained by filtering the initial head action of the target personnel, comprising:

16. The method of claim 10, wherein, The initial head action of the target personnel is filtered according to the set driving preference to obtain the filtered target head action. The target head action is obtained by filtering the initial head action of the target personnel, comprising:

17. The method of claim 10, wherein, ​ The initial head action is filtered through multiple filtering manners to obtain multiple filtering results; According to the multiple filtering results, the target head action is obtained.

18. An apparatus for vehicle control, characterized by The device comprises: A determination module is configured to analyze a vehicle video image in a target vehicle to determine a deflection amplitude of a head posture of a target person; An identification module is configured to identify a target head action of the target person according to the deflection amplitude and a preset first amplitude threshold; A control module is configured to control the target vehicle based on the target head action of the target person.

19. An electronic device, comprising: Comprise: A processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to execute the steps of the vehicle control method in any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to execute the steps of the vehicle control method in any one of claims 1 to 17.

Images