Vehicle control method, device, equipment and medium based on dynamic gestures

Abstract

The present application provides a vehicle control method, device, equipment and medium based on dynamic gestures, wherein a plurality of consecutive vehicle cabin images are obtained; hand recognition and key point positioning are performed on each cabin image frame to obtain the coordinates of each hand key point in each cabin image frame; whether a target static gesture appears in each cabin image frame is determined based on the coordinates of each hand key point in each cabin image frame; if the target static gesture appears in each cabin image frame, the direction and angle of the target dynamic gesture in a plurality of consecutive cabin images are determined based on the target key point coordinates in each cabin image frame; and the vehicle is controlled based on the direction and angle of the target dynamic gesture. The above method is used to achieve precise control of the vehicle and improve the user's vehicle experience.

Description

Technical Field

[0001] The present invention relates to the technical field of smart cockpits, and in particular to a vehicle control method, device, equipment and medium based on dynamic gestures. Background Art

[0002] With the diversified development of in-vehicle entertainment systems, there are more and more scenarios for users to interact with vehicles, such as playing music, navigating to destinations, setting air conditioning temperature, turning on air purifiers, etc. In the prior art, users usually speak control commands through voice interaction, and the system recognizes what the user says and controls the in-vehicle equipment to execute corresponding commands based on the control commands. However, the study found that for some people who cannot speak, it is impossible to control the vehicle through voice interaction, or when the noise in the car is large, it is difficult for the vehicle to accurately recognize the sound, so it is impossible to accurately control the vehicle, and at the same time reduces the user experience of using the vehicle. Summary of the invention

[0003] In view of this, an object of the present invention is to provide a vehicle control method, device, equipment and medium based on dynamic gestures to achieve precise control of the vehicle and improve the user's vehicle experience.

[0004] In a first aspect, an embodiment of the present application provides a vehicle control method based on dynamic gestures, the method comprising:

[0005] Acquire a plurality of consecutive frames of vehicle cabin images;

[0006] Perform hand recognition and key point positioning on each frame of the cabin image to obtain the coordinates of each hand key point in each frame of the cabin image;

[0007] Judging whether a target static gesture appears in each frame of the cabin image according to the coordinates of each key point of the hand in each frame of the cabin image;

[0008] If the target static gesture appears in each frame of the cabin image, then determining the gesture direction and gesture angle of the target dynamic gesture in a plurality of consecutive frames of the cabin image based on the target key point coordinates in each frame of the cabin image;

[0009] The vehicle is controlled based on the gesture direction and the gesture angle.

[0010] Optionally, the target static gesture is a finger-up gesture, the target key point is the hand key point coordinate indicating the fingertip of the target finger among the hand key point coordinates, and the target dynamic gesture is a finger circle gesture.

[0011] Optionally, judging whether a target static gesture appears in each frame of the cabin image according to the coordinates of each hand key point in each frame of the cabin image includes:

[0012] For each frame of the cabin image, judging whether the target finger is in an upward straightened state according to the coordinates of each target finger key point indicating the target finger in each hand key point coordinate of the cabin image, and judging whether the adjacent finger is in a curled state according to the coordinates of each adjacent finger key point of the adjacent finger of the cabin image;

[0013] If the target finger is in an upwardly stretched state, and the adjacent fingers are in a curled state, it is determined that the target static gesture appears in the frame of the cabin image.

[0014] Optionally, the determining the gesture direction and gesture angle of the target dynamic gesture in a plurality of consecutive frames of cabin images based on the coordinates of the target key points in each frame of cabin image includes:

[0015] For each frame of the cabin image, calculating the average coordinate value of the target key point coordinates of each historical frame of the cabin image before the frame of the cabin image, and using the average coordinate value to update the rotation axis of the frame of the cabin image;

[0016] Determine a single-frame rotation angle of the target fingertip relative to the rotation axis of the frame of the cabin image according to the target key point coordinates of the frame of the cabin image;

[0017] Determine the total rotation angle of the target fingertip in a plurality of consecutive frames of the vehicle cabin image according to the single-frame rotation angle of the target fingertip relative to the rotation axis of each frame of the vehicle cabin image;

[0018] The total rotation angle of the target fingertip is determined as the gesture angle, and the gesture direction is determined according to the total rotation angle.

[0019] Optionally, controlling the vehicle based on the gesture direction and the gesture angle includes:

[0020] Determining a volume control direction for the audio playback volume according to the gesture direction;

[0021] Determining a volume control amount for the audio playback volume according to the gesture angle;

[0022] The audio playback device of the vehicle is controlled according to the volume control direction and the volume control amount.

[0023] Optionally, the controlling the vehicle based on the gesture direction and the gesture angle includes:

[0024] determining a temperature control direction for the vehicle cabin temperature according to the direction of the gesture;

[0025] determining a temperature control amount for the vehicle cabin temperature according to the gesture angle;

[0026] The temperature control device of the vehicle is controlled with the temperature control direction and the temperature control amount.

[0027] Optionally, the target finger is an index finger.

[0028] In a second aspect, an embodiment of the present application provides a vehicle control device based on dynamic gestures, the device comprising:

[0029] A vehicle cabin image acquisition module, used to acquire a plurality of consecutive frames of vehicle cabin images;

[0030] A hand key point positioning module is used to perform hand recognition and key point positioning on each frame of the cabin image to obtain the coordinates of each hand key point in each frame of the cabin image;

[0031] A static gesture recognition module is used to determine whether a target static gesture appears in each frame of the cabin image according to the coordinates of each hand key point in each frame of the cabin image;

[0032] A dynamic gesture recognition module, for determining the gesture direction and gesture angle of the target dynamic gesture in a plurality of consecutive cabin image frames based on the target key point coordinates in each cabin image frame if the target static gesture appears in each cabin image frame;

[0033] A vehicle control module is used to control the vehicle based on the gesture direction and the gesture angle.

[0034] Optionally, the target static gesture is a finger-up gesture, the target key point is the hand key point coordinate indicating the fingertip of the target finger among the hand key point coordinates, and the target dynamic gesture is a finger circle gesture.

[0035] Optionally, judging whether a target static gesture appears in each frame of the cabin image according to the coordinates of each hand key point in each frame of the cabin image includes:

[0036] For each frame of the cabin image, judging whether the target finger is in an upward straightened state according to the coordinates of each target finger key point indicating the target finger in each hand key point coordinate of the cabin image, and judging whether the adjacent finger is in a curled state according to the coordinates of each adjacent finger key point of the adjacent finger of the cabin image;

[0037] If the target finger is in an upwardly stretched state, and the adjacent fingers are in a curled state, it is determined that the target static gesture appears in the frame of the cabin image.

[0038] Optionally, the determining the gesture direction and gesture angle of the target dynamic gesture in a plurality of consecutive frames of cabin images based on the coordinates of the target key points in each frame of cabin image includes:

[0039] For each frame of the cabin image, calculating the average coordinate value of the target key point coordinates of each historical frame of the cabin image before the frame of the cabin image, and using the average coordinate value to update the rotation axis of the frame of the cabin image;

[0040] Determine a single-frame rotation angle of the target fingertip relative to the rotation axis of the frame of the cabin image according to the target key point coordinates of the frame of the cabin image;

[0041] Determine the total rotation angle of the target fingertip in a plurality of consecutive frames of the vehicle cabin image according to the single-frame rotation angle of the target fingertip relative to the rotation axis of each frame of the vehicle cabin image;

[0042] The total rotation angle of the target fingertip is determined as the gesture angle, and the gesture direction is determined according to the total rotation angle.

[0043] Optionally, the controlling the vehicle based on the gesture direction and the gesture angle includes:

[0044] Determining a volume control direction for the audio playback volume according to the gesture direction;

[0045] Determining a volume control amount for the audio playback volume according to the gesture angle;

[0046] The audio playback device of the vehicle is controlled according to the volume control direction and the volume control amount.

[0047] Optionally, the controlling the vehicle based on the gesture direction and the gesture angle includes:

[0048] determining a temperature control direction for the vehicle cabin temperature according to the direction of the gesture;

[0049] determining a temperature control amount for the vehicle cabin temperature according to the gesture angle;

[0050] The temperature control device of the vehicle is controlled with the temperature control direction and the temperature control amount.

[0051] Optionally, the target finger is an index finger.

[0052] In a third aspect, an embodiment of the present application provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the dynamic gesture-based vehicle control method described in any optional implementation manner of the first aspect are performed.

[0053] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, executes the steps of the dynamic gesture-based vehicle control method described in any optional implementation manner in the first aspect above.

[0054] The technical solution provided by this application includes but is not limited to the following beneficial effects:

[0055] The present application can obtain images that can carry user control information without the user speaking by acquiring several consecutive frames of cabin images of the vehicle. Then, hand recognition and key point positioning are performed on each frame of cabin image to obtain the coordinates of each hand key point in each frame of cabin image. According to the coordinates of each hand key point in each frame of cabin image, it is judged whether the target static gesture appears in each frame of cabin image, and it can be determined whether the user has made a static gesture to control the vehicle through the user's hand information in each frame of cabin image. Then, if the target static gesture appears in each frame of cabin image, the direction and angle of the target dynamic gesture in several consecutive frames of cabin image are determined based on the target key point coordinates in each frame of cabin image, and after the user makes a static gesture to control the vehicle, the relevant information of the dynamic gesture presented by the user can be determined according to the change of the static gesture. Finally, the vehicle is controlled based on the direction and angle of the target dynamic gesture, and the vehicle can be controlled based on the gesture by collecting the cabin image, identifying the user's hand information in the image, and determining its gesture information without voice interaction, thereby realizing precise control of the vehicle and improving the user's vehicle use experience.

[0056] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 A flow chart of a vehicle control method based on dynamic gestures provided in the first embodiment of the present invention is shown;

[0059] Figure 2 A flow chart of a static gesture detection method provided by the first embodiment of the present invention is shown;

[0060] Figure 3A flow chart of a dynamic gesture recognition method provided by Embodiment 1 of the present invention is shown;

[0061] Figure 4 A flow chart of a vehicle volume control method provided by Embodiment 1 of the present invention is shown;

[0062] Figure 5 A flow chart of a vehicle temperature control method provided by Embodiment 1 of the present invention is shown;

[0063] Figure 6 A schematic diagram showing the sequence numbers of key points of a hand 21 provided in the first embodiment of the present invention is shown;

[0064] Figure 7 A vehicle control device based on dynamic gestures provided by the second embodiment of the present invention is shown;

[0065] Figure 8 A schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention is shown. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0067] Embodiment 1

[0068] To facilitate understanding of this application, Figure 1 The flowchart of a vehicle control method based on dynamic gestures provided in the first embodiment of the present invention is described in detail to explain the first embodiment of the present application.

[0069] See also Figure 1 As shown, Figure 1 A flow chart of a vehicle control method based on dynamic gestures provided in the first embodiment of the present invention is shown, and the method comprises steps S101 to S105:

[0070] S101: Acquire a plurality of consecutive frames of vehicle cabin images.

[0071] Specifically, a cabin video stream in the cabin is collected by a vehicle-mounted camera in the cabin, and each cabin image frame in the cabin video stream is captured to obtain a plurality of cabin image frames, each cabin image containing at least one hand.

[0072] S102: Perform hand recognition and key point positioning on each frame of the cabin image to obtain the coordinates of each hand key point in each frame of the cabin image.

[0073] Specifically, a classic deep learning method is used to perform hand recognition on each frame of the cabin image to detect the hands in the focus area of ​​the picture; then for the detected hands, the hand skeleton key point model is used to locate the key points to obtain the coordinates of each hand key point. This application uses the 21 standard hand key points as the detection result.

[0074] S103: judging whether a target static gesture appears in each frame of the cabin image according to the coordinates of each hand key point in each frame of the cabin image.

[0075] Specifically, since each frame of the image is a static image, only whether the hand in the image presents a static gesture can be detected in the static image. For each frame of the cabin image, the positional relationship between the coordinates of each key point of each hand in the cabin image frame related to the target static gesture is first determined, and then it is determined whether the positional relationship is the same as the positional relationship between the coordinates of each target key point under the target static gesture; if they are the same, it means that the target static gesture appears in the cabin image frame, and if they are not the same, it means that the target static gesture does not appear in the cabin image frame.

[0076] S104: If the target static gesture appears in each frame of the cabin image, determine the gesture direction and gesture angle of the target dynamic gesture in a plurality of consecutive frames of the cabin image based on the target key point coordinates in each frame of the cabin image.

[0077] Specifically, after combining multiple frames of static images, the dynamic gestures appearing in the video stream can be determined based on the gesture changes of the target static gesture in each frame. The target key point coordinates are the key point coordinates related to the target static gesture, which are pre-configured for the target static gesture. If the target static gesture appears in each frame of the cabin image, it means that the target static gesture is continuous and is a component of the same dynamic gesture. According to the changes in the target key point coordinates in each frame of the cabin image, the direction, angle and number of times the target dynamic gesture is executed in several consecutive frames of the cabin image can be determined according to the preset judgment rules.

[0078] S105: Control the vehicle based on the gesture direction and the gesture angle.

[0079] Specifically, the direction of the target dynamic gesture is used as the control direction, and the angle of the target dynamic gesture is used as the control amount to control the device or function to be controlled in the vehicle. In actual applications, the driver can first use voice (such as "adjust the volume", "adjust the temperature") to enter a specific mode to enable the state of real-time gesture adjustment, and then adjust the corresponding signal of the vehicle computer in real time through dynamic gestures.

[0080] In an optional implementation, the target static gesture is a finger-up gesture, the target key point is the hand key point coordinates indicating the fingertips of the target fingers among the hand key point coordinates, and the target dynamic gesture is a finger circle gesture.

[0081] Specifically, the finger-up gesture is a state where the target finger is pointing upward, and the adjacent fingers of the target finger are in a curled state. When the finger-up gesture is kept moving or rotating, a dynamic gesture of a finger circle gesture is generated, and the gesture direction of the dynamic gesture is that the finger circles counterclockwise or clockwise, and the angle is the rotation angle of the circle, one circle is 360 degrees, and the number is the number of circles drawn by the finger.

[0082] In an alternative embodiment, see Figure 2 As shown, Figure 2 A flowchart of a static gesture detection method provided by the first embodiment of the present invention is shown, wherein the method of judging whether a target static gesture appears in each frame of the cabin image according to the coordinates of each hand key point in each frame of the cabin image comprises steps S201 to S202:

[0083] S201: For each frame of the cabin image, determine whether the target finger is in an upwardly stretched state according to the coordinates of each target finger key point indicating the target finger in the hand key point coordinates of the frame of the cabin image, and determine whether the adjacent fingers are in a curled state according to the coordinates of each adjacent finger key point of the adjacent fingers of the frame of the cabin image.

[0084] Specifically, in the frame of the cabin image, whether the target finger is in an upward straightened state is determined according to the distance between the key point at the fingertip and the key point at the base of the target finger and the relative deflection between the key point at the fingertip and the key point at the base of the finger. In the frame of the cabin image, whether the adjacent finger is in a curled state is determined according to the relative distance between the key point at the fingertip and the key point at the base of the adjacent finger.

[0085] S202: If the target finger is in an upwardly stretched state and the adjacent fingers are in a curled state, it is determined that the target static gesture appears in the frame of the cabin image.

[0086] Specifically, if the target finger is in an upwardly stretched state and the adjacent fingers are in a curled state, it means that the gesture appearing in the cabin image of the frame is the target static gesture.

[0087] Alternatively, the target static gesture can be classified and determined based on the CNN model. It is necessary to collect enough image samples of the target static gesture and non-target static gesture in advance and train a classification model. The classification model can be used to determine whether a hand image contains the target static gesture.

[0088] In an alternative embodiment, see Figure 3 As shown, Figure 3 A flow chart of a dynamic gesture recognition method provided by the first embodiment of the present invention is shown, wherein the direction and number of target dynamic gestures in a plurality of consecutive frames of cabin images are determined based on the coordinates of the target key points in each frame of cabin image, including steps S301 to S304:

[0089] S301: For each cabin image frame, calculate the average coordinate value of the target key point coordinates of each historical cabin image frame before the cabin image frame, and use the average coordinate value to update the rotation axis of the cabin image frame.

[0090] Specifically, since the rotation axis may change when the target finger rotates, the rotation axis needs to be updated at regular intervals when determining the number of dynamic gestures. The specific updating method is to calculate the coordinate average of the target key point coordinates of each historical frame cabin image before the frame cabin image, and update the rotation axis of the frame cabin image with the coordinate average.

[0091] S302: Determine a single-frame rotation angle of the target fingertip relative to the rotation axis of the frame of the vehicle cabin image according to the target key point coordinates of the frame of the vehicle cabin image.

[0092] Specifically, according to the coordinates of the target key points of the frame cabin image, the atan2 function is used to calculate the single-frame rotation angle theta of the target fingertip relative to the rotation axis of the frame cabin image.

[0093] S303: Determine a total rotation angle of the target fingertip in a plurality of consecutive frames of the vehicle cabin image according to a single-frame rotation angle of the target fingertip relative to a rotation axis of each frame of the vehicle cabin image.

[0094] Specifically, first, for each frame of the cabin image, the rotation angle Δθ of the single-frame rotation angle θ of the cabin image frame relative to the single-frame rotation angle θ' of the cabin image frame before the cabin image frame is calculated. The calculation method of Δθ is: assuming θ, θ'∈[-π, π), record d = θ-θ', if d>π then Δθ=d-2π; otherwise if d<-π then Δθ=d+2π; otherwise Δθ=d. Then the total rotation angle ang is accumulated and calculated as the sum of Δθ of each frame of the cabin image.

[0095] S304: Determine the total rotation angle of the target fingertip as the gesture angle, and determine the gesture direction according to the total rotation angle.

[0096] Specifically, the total rotation angle ang obtained by cumulative calculation is determined as the gesture angle, and the number of target dynamic gestures is recorded as circles, circles=ang / 2π. When the calculated circles is a positive number, the gesture direction of the target dynamic gesture is determined to be counterclockwise, and when the calculated circles is a negative number, the gesture direction of the target dynamic gesture is determined to be clockwise.

[0097] In an alternative embodiment, see Figure 4 As shown, Figure 4 A flow chart of a vehicle volume control method provided by the first embodiment of the present invention is shown, wherein the vehicle is controlled based on the gesture direction and the gesture angle, including steps S401 to S403:

[0098] S401: Determine a volume control direction for the audio playback volume according to the gesture direction.

[0099] Specifically, when the gesture direction of the target dynamic gesture is clockwise, the volume control direction of the audio playback volume is determined to be positive control, that is, the volume is increased; when the gesture direction of the target dynamic gesture is counterclockwise, the volume control direction of the audio playback volume is determined to be negative control, that is, the volume is reduced.

[0100] S402: Determine a volume control amount for the audio playback volume according to the gesture angle.

[0101] S403: Controlling the audio playback device of the vehicle with the volume control direction and the volume control amount.

[0102] Specifically, when the accumulated value of the gesture angle ang reaches the preset angle value k_ang, the volume is directly adjusted to k_ang / k_vol (where k_vol is a preset constant). If k_ang / k_vol is lower than the lower volume limit, it is adjusted to the lower volume limit; if k_ang / k_vol is higher than the upper volume limit, it is adjusted to the upper volume limit.

[0103] In an alternative embodiment, see Figure 5 As shown, Figure 5 A flow chart of a vehicle temperature control method provided by the first embodiment of the present invention is shown, wherein the vehicle is controlled based on the gesture direction and the gesture angle, including steps S501 to S503:

[0104] S501: Control the vehicle based on the gesture direction and the gesture angle.

[0105] Specifically, when the gesture direction of the target dynamic gesture is clockwise, the temperature control direction of the cabin temperature is determined to be positive control, that is, to increase the temperature; when the gesture direction of the target dynamic gesture is counterclockwise, the temperature control direction of the cabin temperature is determined to be negative control, that is, to lower the temperature.

[0106] S502: Determine a temperature control amount for the vehicle cabin temperature according to the gesture angle.

[0107] S503: Controlling the temperature control device of the vehicle with the temperature control direction and the temperature control amount.

[0108] Specifically, when the accumulated value of the gesture angle ang reaches the preset angle value k_ang, the temperature is directly adjusted to k_ang / k_vol (where k_vol is a preset constant). If k_ang / k_vol is lower than the lower temperature limit, it is adjusted to the lower temperature limit; if k_ang / k_vol is higher than the upper temperature limit, it is adjusted to the upper temperature limit.

[0109] In an alternative embodiment, the target finger is the index finger.

[0110] Specifically, when the target finger is the index finger, its adjacent finger is the middle finger. The target static gesture detection method with the index finger as the target finger is to determine whether there is at least one frame of the cabin image within one second in which the index finger is in an upward straight state and the middle finger is in a curled state, and if so, it is determined that the target static gesture appears in the cabin image within one second.

[0111] The actual judgment method is as follows: 1. Define the rotation angle (a, b, c): the arc angle of "vector ab" turning to "vector bc", counterclockwise is positive, clockwise is negative. 2. Define the rotation angle (a, b, c, d): the arc angle of "vector ab" turning to "vector cd", counterclockwise is positive, clockwise is negative. 3. Define the index finger straightening: distance (5, 8) / palm circumference>0.1&&absolute value (rotation angle (5, 6, 7) + angle (6, 7, 8))<π * 0.1; 4. Define the middle finger curling: absolute value (rotation angle (0, 9, 10,11))>π * 0.5; 5. Gesture appearance judgment method: at least one frame in one second meets the definition of index finger straightening && definition of middle finger curling.

[0112] See also Figure 6 As shown, Figure 6A schematic diagram of the serial numbers of key points of a hand 21 provided in the first embodiment of the present invention is shown, wherein the serial numbers in the figure are the serial numbers of the key points, 1 represents key point 1, 2 represents key point 2, and so on, 20 represents key point 20. Figure 6 The definition of the index finger straightening and the middle finger curling are explained as follows: the rotation angle (5, 6, 7) is the vector between key points 5 and 6, recorded as the vector (5, 6) turning to the vector between key points 6 and 7, recorded as the radian of vector (6, 7), counterclockwise is positive, and clockwise is negative. The rotation angle (6, 7, 8) is the vector between key points 6 and 7, recorded as the vector (6, 7) turning to the vector between key points 7 and 8, recorded as the radian of vector (7, 8), counterclockwise is positive, and clockwise is negative. The rotation angle (0, 9, 10, 11) is the vector between key points 0 and 9, recorded as the vector (0,9) turning to the vector between key points 10 and 11, recorded as the radian of vector (10, 11), counterclockwise is positive, and clockwise is negative.

[0113] When the index finger is in an upward straight position, the distance and position relationship between the coordinates of each key point must satisfy the following requirements: the distance between the key point at the tip of the index finger (corresponding to key point 8 in the standard key points of hand 21) and the key point at the base of the index finger (corresponding to key point 5 in the standard key points of hand 21), denoted as distance (5, 8), divided by the circumference of the palm is greater than 0.1, and the absolute value of the rotation angle of key points 5, 6, and 7 in the standard key points of hand 21, denoted as angle (5, 6, 7), and the absolute value of the rotation angle of key points 6, 7, and 8 in the standard key points of hand 21, denoted as angle (6, 7, 8), is less than π * 0.1.

[0114] When the middle finger is in a curled state, the distance and position relationship between the coordinates of each key point needs to satisfy: the rotation angle of key point 0, key point 9, key point 10, and key point 11 in the 21 standard key points of the hand, recorded as rotation angle (0, 9, 10, 11), has an absolute value >π * 0.5.

[0115] Embodiment 2

[0116] See also Figure 7 As shown, Figure 7 A vehicle control device based on dynamic gestures provided by a second embodiment of the present invention is shown, and the device includes:

[0117] The vehicle cabin image acquisition module 701 is used to acquire a plurality of consecutive frames of vehicle cabin images;

[0118] The hand key point positioning module 702 is used to perform hand recognition and key point positioning on each frame of the cabin image to obtain the coordinates of each hand key point in each frame of the cabin image;

[0119] The static gesture recognition module 703 is used to determine whether a target static gesture appears in each frame of the cabin image according to the coordinates of each hand key point in each frame of the cabin image;

[0120] The dynamic gesture recognition module 704 is used to determine the gesture direction and gesture angle of the target dynamic gesture in a plurality of consecutive cabin image frames based on the target key point coordinates in each cabin image frame if the target static gesture appears in each cabin image frame;

[0121] The vehicle control module 705 is used to control the vehicle based on the gesture direction and the gesture angle.

[0122] In an optional implementation, the target static gesture is a finger-up gesture, the target key point is the hand key point coordinates indicating the fingertips of the target fingers among the hand key point coordinates, and the target dynamic gesture is a finger circle gesture.

[0123] In an optional implementation, judging whether a target static gesture appears in each frame of the cabin image according to the coordinates of each hand key point in each frame of the cabin image includes:

[0124] For each frame of the cabin image, judging whether the target finger is in an upward straightened state according to the coordinates of each target finger key point indicating the target finger in each hand key point coordinate of the cabin image, and judging whether the adjacent finger is in a curled state according to the coordinates of each adjacent finger key point of the adjacent finger of the cabin image;

[0125] If the target finger is in an upwardly stretched state, and the adjacent fingers are in a curled state, it is determined that the target static gesture appears in the frame of the cabin image.

[0126] In an optional implementation, the step of determining the gesture direction and gesture angle of the target dynamic gesture in a plurality of consecutive frames of cabin images based on the coordinates of the target key points in each frame of cabin image includes:

[0127] For each frame of the cabin image, calculating the average coordinate value of the target key point coordinates of each historical frame of the cabin image before the frame of the cabin image, and using the average coordinate value to update the rotation axis of the frame of the cabin image;

[0128] Determine a single-frame rotation angle of the target fingertip relative to the rotation axis of the frame of the cabin image according to the target key point coordinates of the frame of the cabin image;

[0129] Determine the total rotation angle of the target fingertip in a plurality of consecutive frames of the vehicle cabin image according to the single-frame rotation angle of the target fingertip relative to the rotation axis of each frame of the vehicle cabin image;

[0130] The total rotation angle of the target fingertip is determined as the gesture angle, and the gesture direction is determined according to the total rotation angle.

[0131] In an optional implementation, the controlling the vehicle based on the gesture direction and the gesture angle includes:

[0132] Determining a volume control direction for the audio playback volume according to the gesture direction;

[0133] Determining a volume control amount for the audio playback volume according to the gesture angle;

[0134] The audio playback device of the vehicle is controlled according to the volume control direction and the volume control amount.

[0135] In an optional implementation, the controlling the vehicle based on the gesture direction and the gesture angle includes:

[0136] determining a temperature control direction for the vehicle cabin temperature according to the direction of the gesture;

[0137] determining a temperature control amount for the vehicle cabin temperature according to the gesture angle;

[0138] The temperature control device of the vehicle is controlled with the temperature control direction and the temperature control amount.

[0139] In an alternative embodiment, the target finger is the index finger.

[0140] Embodiment 3

[0141] Based on the same application concept, see Figure 8 As shown, Figure 8 FIG. 4 shows a schematic diagram of the structure of a computer device provided by Embodiment 3 of the present invention, wherein: Figure 8 As shown, a computer device 800 provided in Embodiment 3 of the present application includes:

[0142] A processor 801, a memory 802 and a bus 803, wherein the memory 802 stores machine-readable instructions executable by the processor 801. When the computer device 800 is running, the processor 801 communicates with the memory 802 via the bus 803. When the processor 801 is running, the machine-readable instructions execute the steps of the dynamic gesture-based vehicle control method shown in the above-mentioned embodiment 1.

[0143] Embodiment 4

[0144] Based on the same application concept, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the dynamic gesture-based vehicle control method described in any one of the above embodiments are executed.

[0145] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0146] The computer program product for dynamic gesture-based vehicle control provided in an embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0147] The dynamic gesture-based vehicle control device provided in the embodiment of the present invention can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present invention has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference can be made to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0148] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0149] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0150] In addition, each functional unit in the embodiment provided by the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0151] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0152] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0153] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or perform equivalent replacements on some of the technical features thereof; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A vehicle control method based on dynamic gestures, characterized in that: The method comprises: Acquire a plurality of consecutive frames of vehicle cabin images; Perform hand recognition and key point positioning on each frame of the cabin image to obtain the coordinates of each hand key point in each frame of the cabin image; Determine whether a target static gesture appears in each frame of the cabin image according to the coordinates of each key point of the hand in each frame of the cabin image; If the target static gesture appears in each frame of the cabin image, then determining the gesture direction and gesture angle of the target dynamic gesture in a plurality of consecutive frames of the cabin image based on the target key point coordinates in each frame of the cabin image; controlling the vehicle based on the gesture direction and the gesture angle; The target static gesture is a finger-up gesture, the target key point is the hand key point coordinate indicating the fingertip of the target finger among the hand key point coordinates, and the target dynamic gesture is a finger circle gesture; The method of determining the gesture direction and gesture angle of the target dynamic gesture in a plurality of consecutive frames of cabin images based on the coordinates of the target key points in each frame of cabin image comprises: For each frame of the cabin image, calculating the average coordinate value of the target key point coordinates of each historical frame of the cabin image before the frame of the cabin image, and using the average coordinate value to update the rotation axis of the frame of the cabin image; Determine a single-frame rotation angle of the target fingertip relative to the rotation axis of the frame of the cabin image according to the target key point coordinates of the frame of the cabin image; Determine the total rotation angle of the target fingertip in a plurality of consecutive frames of the vehicle cabin image according to the single-frame rotation angle of the target fingertip relative to the rotation axis of each frame of the vehicle cabin image; The total rotation angle of the target fingertip is determined as the gesture angle, and the gesture direction is determined according to the total rotation angle.

2. The method according to claim 1, characterized in that The step of judging whether a target static gesture appears in each frame of the cabin image according to the coordinates of each hand key point in each frame of the cabin image includes: For each frame of the cabin image, judging whether the target finger is in an upward straightened state according to the coordinates of each target finger key point indicating the target finger in each hand key point coordinate of the cabin image, and judging whether the adjacent finger is in a curled state according to the coordinates of each adjacent finger key point of the adjacent finger of the cabin image; If the target finger is in an upwardly stretched state, and the adjacent fingers are in a curled state, it is determined that the target static gesture appears in the frame of the cabin image.

3. The method according to claim 1, characterized in that The controlling the vehicle based on the gesture direction and the gesture angle includes: Determining a volume control direction for the audio playback volume according to the gesture direction; Determining a volume control amount for the audio playback volume according to the gesture angle; The audio playback device of the vehicle is controlled according to the volume control direction and the volume control amount.

4. The method according to claim 1, characterized in that: The controlling the vehicle based on the gesture direction and the gesture angle includes: determining a temperature control direction for the vehicle cabin temperature according to the direction of the gesture; determining a temperature control amount for the vehicle cabin temperature according to the gesture angle; The temperature control device of the vehicle is controlled with the temperature control direction and the temperature control amount.

5. The method according to claim 1, characterized in that The target finger is the index finger.

6. A vehicle control device based on dynamic gestures, characterized in that: The device comprises: A vehicle cabin image acquisition module, used to acquire a plurality of consecutive frames of vehicle cabin images; A hand key point positioning module is used to perform hand recognition and key point positioning on each frame of the cabin image to obtain the coordinates of each hand key point in each frame of the cabin image; A static gesture recognition module is used to determine whether a target static gesture appears in each frame of the cabin image according to the coordinates of each hand key point in each frame of the cabin image; A dynamic gesture recognition module, for determining the gesture direction and gesture angle of the target dynamic gesture in a plurality of consecutive cabin image frames based on the target key point coordinates in each cabin image frame if the target static gesture appears in each cabin image frame; a vehicle control module, configured to control the vehicle based on the gesture direction and the gesture angle; The target static gesture is a finger-up gesture, the target key point is the hand key point coordinate indicating the fingertip of the target finger among the hand key point coordinates, and the target dynamic gesture is a finger circle gesture; The method of determining the gesture direction and gesture angle of the target dynamic gesture in a plurality of consecutive frames of cabin images based on the coordinates of the target key points in each frame of cabin image comprises: For each frame of the cabin image, calculating the average coordinate value of the target key point coordinates of each historical frame of the cabin image before the frame of the cabin image, and using the average coordinate value to update the rotation axis of the frame of the cabin image; Determine a single-frame rotation angle of the target fingertip relative to the rotation axis of the frame of the cabin image according to the target key point coordinates of the frame of the cabin image; Determine the total rotation angle of the target fingertip in a plurality of consecutive frames of the vehicle cabin image according to the single-frame rotation angle of the target fingertip relative to the rotation axis of each frame of the vehicle cabin image; The total rotation angle of the target fingertip is determined as the gesture angle, and the gesture direction is determined according to the total rotation angle.

7. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the dynamic gesture-based vehicle control method as described in any one of claims 1 to 5 are performed.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the dynamic gesture-based vehicle control method as described in any one of claims 1 to 5.

Images