Vehicle machine control method and device, vehicle and storage medium

Abstract

The embodiment of the invention discloses a vehicle machine control method and device, a vehicle and a storage medium, and relates to the technical field of vehicle control. The method comprises the following steps: acquiring an in-vehicle image acquired by a camera of a current vehicle, and identifying original hand images of at least two users from the in-vehicle image; segmenting various types of finger images from the original hand image, and combining the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image; performing gesture recognition on the combined hand image to obtain a control gesture corresponding to the combined hand image; and controlling the vehicle machine of the current vehicle to execute a vehicle machine control instruction corresponding to the control gesture. According to the technical scheme provided by the embodiment of the invention, a vehicle-mounted terminal control function is realized, and the accuracy of gesture recognition in a multi-user scene is improved, so that the accuracy and reliability of vehicle-mounted terminal linkage are improved, and smoother and more natural interaction experience is provided for users.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of vehicle control technology, and in particular, to a vehicle control method, device, vehicle and storage medium. Background Art

[0002] With the rapid development of the smart cockpit ecosystem, gesture recognition, as one of the important functions of the interactive experience for drivers and passengers in the cabin, is increasingly attracting the attention of consumers.

[0003] In the prior art, a deep learning model is used to perform gesture recognition on the hand images of the driver and passenger to obtain the corresponding predicted gestures, which can then be used to control the vehicle's computer to execute the computer control instructions corresponding to the predicted gestures, such as switching to the next song or increasing the volume.

[0004] However, current gesture recognition methods mainly focus on single static gesture recognition or simple dynamic gesture recognition. However, in multi-user environments, especially in-vehicle environments, there is a problem of low recognition accuracy when multiple gestures are performed simultaneously. Summary of the invention

[0005] The embodiments of the present application provide a vehicle control method, device, vehicle and storage medium, which realize the vehicle control function and improve the accuracy of gesture recognition to solve the problem of low recognition accuracy when multiple gestures are performed simultaneously in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a vehicle control method, the method comprising:

[0007] Obtaining an in-vehicle image captured by a camera of the current vehicle, and identifying original hand images of at least two users from the in-vehicle image;

[0008] Segmenting various types of finger images from the original hand image, and combining the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image, wherein each finger image in the combined hand image corresponds to a different type;

[0009] Performing gesture recognition on the combined hand image to obtain a control gesture corresponding to the combined hand image;

[0010] The vehicle computer that controls the current vehicle executes the vehicle computer control instruction corresponding to the control gesture.

[0011] In an embodiment of the present application, an in-vehicle image captured by a camera of the current vehicle can be obtained, and original hand images of at least two users can be identified from the in-vehicle image, and then various types of finger images can be segmented from the original hand image, and the finger images segmented from different original hand images can be combined based on the types corresponding to the finger images to obtain a combined hand image, and then gesture recognition is performed on the combined hand image to obtain a control gesture corresponding to the combined hand image, and then the vehicle computer of the current vehicle is controlled to execute the vehicle computer control instruction corresponding to the control gesture, thereby realizing the vehicle computer control function; by combining the finger images in the original hand images of different users and performing gesture recognition on the combined hand image, the gestures of multiple users can be processed at the same time, and by capturing the overall features and action patterns of the combined hand image, the control gesture can be determined according to the gestures of most users, and the noise and interference introduced by the inaccurate gestures of individual users in the multi-user environment are reduced, thereby improving the accuracy and reliability of gesture recognition in the multi-user environment, solving the problem of low recognition accuracy when multiple gestures are performed simultaneously in the prior art, thereby improving the accuracy and reliability of vehicle-computer linkage, and providing users with a smoother and more natural interactive experience.

[0012] In a second aspect, an embodiment of the present application provides a vehicle control device, the device comprising:

[0013] A hand recognition module is used to obtain an in-vehicle image captured by a camera of the current vehicle and recognize original hand images of at least two users from the in-vehicle image;

[0014] A combination module, used for segmenting various types of finger images from the original hand image, and combining the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image, wherein each finger image in the combined hand image corresponds to a different type;

[0015] A gesture recognition module, used to perform gesture recognition on the combined hand image to obtain a control gesture corresponding to the combined hand image;

[0016] The control module is used to control the vehicle computer of the current vehicle to execute the vehicle computer control instruction corresponding to the control gesture.

[0017] In a third aspect, an embodiment of the present application provides a vehicle, the vehicle comprising:

[0018] at least one processor; and a memory communicatively coupled to the at least one processor;

[0019] Among them, the memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the vehicle control method of any embodiment of the present application.

[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, a vehicle control method as in any embodiment of the present application is implemented.

[0021] The description of the second, third and fourth aspects in this application can refer to the detailed description of the first aspect; and the beneficial effects described in the second, third and fourth aspects can refer to the beneficial effect analysis of the first aspect, which will not be repeated here.

[0022] In this application, the name of the above-mentioned vehicle control device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of this application, they fall within the scope of the claims of this application and their equivalent technologies.

[0023] These and other aspects of the present application will become more apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 It is a flow chart of the vehicle control method provided by the embodiment of the present application;

[0026] Figure 2 is another flowchart of the vehicle control method provided by an embodiment of the present application;

[0027] Figure 3 It is another flowchart diagram of the vehicle control method provided in the embodiment of the present application;

[0028] Figure 4 is a structural schematic diagram of a vehicle control device provided in an embodiment of the present application;

[0029] Figure 5 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0031] It should be noted that the terms "first", "second", "target", and "original" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including", "having", and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0032] Figure 1 This is a flow chart of the vehicle control method provided by the embodiment of the present application. This embodiment can be applied to the scenario where the gestures of the driver and passengers in the vehicle need to be recognized and the corresponding vehicle control instructions need to be executed. The vehicle control method provided by this embodiment can be executed by the vehicle control device provided by the embodiment of the present application, and the device can be implemented by software and / or hardware. In a specific embodiment, the vehicle control device can be integrated in the vehicle.

[0033] In a specific implementation, the vehicle may include a vehicle controller, a vehicle computer, and a camera, etc. The camera may be installed on a dashboard or an A-pillar of the vehicle to capture images of locations within the cabin.

[0034] The execution subject of this method may be the vehicle controller of the vehicle (i.e. the vehicle controller of the current vehicle), see Figure 1 The vehicle control method of this embodiment includes but is not limited to the following steps:

[0035] S110: Acquire an in-vehicle image captured by a camera of the current vehicle, and identify original hand images of at least two users from the in-vehicle image.

[0036] Among them, the in-vehicle image is a visual representation of the in-vehicle environment captured in real time by a camera installed inside the vehicle; the in-vehicle image may include users, seats, dashboards, steering wheels, and any other visible elements in the vehicle.

[0037] The original hand image is extracted from the in-car image through image recognition and segmentation technology, and is an image that directly reflects the visual information of the user's hand.

[0038] Specifically, when it is necessary to control the vehicle computer to perform a control operation corresponding to the gesture of the user in the vehicle, the in-vehicle image captured by the camera of the current vehicle can be obtained, that is, the in-vehicle video captured by the camera can be obtained, and when the user in the vehicle makes a gesture, the frame image corresponding to that moment is captured from the in-vehicle video to obtain the in-vehicle image. At this time, the in-vehicle image may include multiple users.

[0039] Afterwards, the original hand images of at least two users can be identified from the in-car image, that is, the in-car image can be preprocessed, for example, the in-car image can be denoised to eliminate noise interference in the in-car image and improve image quality; or, the contrast of the in-car image can be adjusted to make the difference between the hand area and other areas more obvious, so as to facilitate subsequent hand detection; or, if the current vehicle is in a driving state, the in-car image can be stabilized to reduce image blur caused by vehicle vibration; then, the target detection algorithm (such as YOLO (You Only Look Once) algorithm, etc.) can be used to identify the user's hand area from the preprocessed in-car image. For example, the in-car image can be input into a pre-trained target detection model (such as YOLOv8 (You Only Look Once version 8) model), after being processed by the target detection model, at least two detection frames can be marked in the in-vehicle image, one of which covers the hand area of ​​a user, and the hand area at this time may include the wrist area, the palm area and the finger area; then, based on the at least two detection frames, the hand areas of the users can be cropped out from the in-vehicle image respectively to obtain the original hand images of at least two users.

[0040] S120, segmenting various types of finger images from the original hand image, and combining the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image.

[0041] The finger image is an image segmented from the original hand image and contains only the visual information of a single finger, which can more clearly show the shape, outline and details of the finger, providing a basis for subsequent gesture recognition. Optionally, the types corresponding to the finger image may include the thumb, index finger, middle finger, ring finger and little finger.

[0042] The combined hand image is an image obtained by combining different types of finger images segmented from different original hand images according to certain rules or algorithms. During the combination process, each finger image comes from a different original hand image, but they form a complete and meaningful hand posture in the combined hand image; and each finger image in the combined hand image corresponds to a different type.

[0043] Specifically, after obtaining the original hand images of at least two users, various types of finger images can be segmented from the original hand images. For example, for each of the at least two original hand images, the current original hand image can be further preprocessed, such as smoothing filtering and edge enhancement, to highlight the contours and details of the fingers. Then, the semantic segmentation technology is used to extract the hand boundary from the current original hand image, and the contour of each finger is further refined. Then, based on the contour of each finger, the current original hand image is segmented to obtain multiple finger images. Each finger image in the multiple finger images is classified according to features such as the length, width, and shape of the fingers, and the type corresponding to each finger image is determined, such as the thumb, index finger, middle finger, ring finger, and little finger, so as to obtain various types of finger images.

[0044] Afterwards, the finger images segmented from different original hand images can be combined based on the types corresponding to the finger images. For example, any type can be selected as the current type, and the finger images of the current type from the finger images segmented from different original hand images can be combined into a finger image set corresponding to the current type. Then, feature extraction is performed on each finger image in the finger image set to obtain the posture features corresponding to each finger image. Then, the number corresponding to each posture feature is determined, and the finger images corresponding to the posture features with the largest number are selected from the finger image set to obtain the target finger images corresponding to the current type. Then, other types are selected as the current type, and the above process is repeated to select the target finger images corresponding to each type respectively.

[0045] Afterwards, the selected target finger images can be combined into a complete hand using image stitching technology to obtain a combined hand image, and each finger image in the combined hand image corresponds to a different type, that is, the combined hand image includes a finger image corresponding to the thumb, a finger image corresponding to the index finger, a finger image corresponding to the middle finger, a finger image corresponding to the ring finger, and a finger image corresponding to the little finger.

[0046] S130: Perform gesture recognition on the combined hand image to obtain a control gesture corresponding to the combined hand image.

[0047] The control gesture is used to characterize the specific gesture category corresponding to the combined hand image.

[0048] Specifically, after obtaining the combined hand image, gesture recognition can be performed on the combined hand image to obtain the control gesture corresponding to the combined hand image, that is, a pre-trained gesture recognition model can be used to perform gesture recognition on the combined hand image to obtain the control gesture corresponding to the combined hand image, wherein the gesture recognition model can be a yolov8 model, a key point-based recognition model, or a temporal network-based recognition model, etc. Specifically, the combined hand image can be input into a pre-trained gesture recognition model, and after being processed by the gesture recognition model, the gesture category corresponding to the combined hand image can be output, and the gesture category is determined as a control gesture.

[0049] S140, controlling the vehicle computer of the current vehicle to execute the vehicle computer control instruction corresponding to the control gesture.

[0050] Among them, the vehicle control command is an operation command generated by the vehicle controller according to the user's gesture recognition result, which is used to represent the control command corresponding to the control gesture; through the vehicle control command, the user can achieve non-contact control of the vehicle's internal environment through gestures, improving the convenience and safety of driving and riding. For example, the vehicle control command may include adjusting the volume, switching the playback mode, and adjusting the air conditioning temperature.

[0051] Specifically, after obtaining the control gesture, the identified control gesture can be mapped to the corresponding vehicle control instruction. For example, the preset gesture instruction relationship table can be queried based on the control gesture to obtain the vehicle control instruction corresponding to the control gesture, and the vehicle control instruction is sent to the vehicle. After that, the vehicle can receive the vehicle control instruction and execute the control operation corresponding to the vehicle control instruction, and then send the execution result to the vehicle controller. Among them, the preset gesture instruction relationship table is a pre-set relationship table used to store the mapping relationship between the control gesture and the vehicle control instruction; the preset gesture instruction relationship table can include multiple control gestures and the vehicle control instruction corresponding to each control gesture.

[0052] Afterwards, the vehicle controller can receive the execution result, and when the execution result is execution failure, generate execution failure information and display the execution failure information to remind the user to handle the exception in time.

[0053] Optionally, the user's control gesture can be a digital gesture. The characteristic of a digital gesture is that the hand features are clear and the judgment of the control gesture will not be affected by the size of the hand. Therefore, by combining the finger images segmented from different original hand images to obtain a combined hand image, and performing gesture recognition on the combined hand image to obtain a control gesture, the judgment of the control gesture can be avoided due to the difference in hand size of different users, thereby improving the accuracy of gesture recognition.

[0054] Exemplarily, when the control gesture is "1", and the vehicle computer control instruction corresponding to "1" is to switch to the next song, the vehicle computer can be controlled to perform the "switch to the next song" operation.

[0055] The technical solution of the embodiment of the present application can obtain the in-vehicle image captured by the camera of the current vehicle, and identify the original hand images of at least two users from the in-vehicle image, then segment various types of finger images from the original hand image, and combine the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image, and then perform gesture recognition on the combined hand image to obtain the control gesture corresponding to the combined hand image, and then control the vehicle computer of the current vehicle to execute the vehicle computer control instruction corresponding to the control gesture, thereby realizing the vehicle computer control function; by combining the finger images in the original hand images of different users and performing gesture recognition on the combined hand image, the gestures of multiple users can be processed at the same time, and by capturing the overall features and action patterns of the combined hand image, the control gesture can be determined according to the gestures of most users, and the noise and interference introduced by the inaccurate gestures of individual users in the multi-user environment are reduced, thereby improving the accuracy and reliability of gesture recognition in the multi-user environment, solving the problem of low recognition accuracy when multiple gestures are performed simultaneously in the prior art, thereby improving the accuracy and reliability of vehicle-computer linkage, and providing users with a smoother and more natural interactive experience.

[0056] The following further describes a vehicle control method provided by an embodiment of the present application. Figure 2 1 is another flow chart of the vehicle control method provided by the embodiment of the present application. The embodiment of the present application is optimized on the basis of the above embodiments. Figure 2 The method of this embodiment includes but is not limited to the following steps:

[0057] S201. Obtain an in-vehicle image captured by a camera of the current vehicle, and identify original hand images of at least two users from the in-vehicle image.

[0058] S202: Segment various types of finger images from the original hand image.

[0059] Afterwards, S203 or S207 may be executed.

[0060] Optionally, the type corresponding to the finger image may be determined in the following manner: determining a pixel area corresponding to the finger image; determining a relative position of the finger image according to the pixel area; and determining the type corresponding to the finger image according to the relative position.

[0061] The pixel area is the specific position area of ​​the finger in the original hand image, and the relative position is the position relationship between the finger image and the palm in the original hand image.

[0062] Specifically, the original hand image can be processed using speech segmentation technology to segment each finger from the original hand image to obtain multiple finger images (i.e., five finger images), and then the specific position area of ​​each finger image in the original hand image is determined to obtain the pixel area corresponding to each finger image.

[0063] Afterwards, a two-dimensional coordinate system with the center point of the palm in the original hand image as the origin can be established, and any finger image can be selected as the current finger image, and then the position of the center point of the pixel area corresponding to the current finger image in the two-dimensional coordinate system is determined, and the relative distance and angle between the current finger image and the palm are calculated based on the position to obtain the relative position of the current finger image, and then the preset position type relationship table is queried based on the relative position to obtain the finger type corresponding to the relative position, and the finger type is determined as the type corresponding to the current finger image, and then other finger images can be selected as the current finger image, and the above process is repeated to obtain the type corresponding to each finger image. Among them, the preset position type relationship table is a pre-set relationship table used to store the mapping relationship between relative positions and finger types; the preset position type relationship table can include multiple relative positions and the finger type corresponding to each relative position.

[0064] In an embodiment of the present application, by analyzing the relative position relationship between the fingers and the palm, the type corresponding to the finger image can be more accurately identified, and multiple finger images can be automatically processed to determine the type corresponding to the finger image without human intervention, thereby providing data support for the subsequent determination of the combined hand image, thereby improving the efficiency and accuracy of gesture recognition.

[0065] S203, respectively obtaining different types of finger images segmented from different original hand images.

[0066] S204: Combine different types of finger images to obtain at least two combined hand images.

[0067] Exemplarily, it is known that the in-vehicle image includes user 1, user 2 and user 3, the original hand image of user 1 is {thumb image 1, index finger image 1, middle finger image 1, ring finger image 1, little finger image 1}, the original hand image of user 2 is {thumb image 2, index finger image 2, middle finger image 2, ring finger image 2, little finger image 2}, and the original hand image of user 3 is {thumb image 3, index finger image 3, middle finger image 3, ring finger image 3, little finger image 3}; the process of combining different types of finger images segmented from different original hand images to obtain at least two combined hand images can be: any user's thumb image, any user's index finger image, any user's middle finger image, any user's ring finger image and any user's little finger image can be randomly selected from the original hand image of user 1, the original hand image of user 2 and the original hand image of user 3, and these finger images are combined to obtain a combined hand image, and then the above process can be repeated to obtain at least two combined hand images. For example, finger images randomly selected from the original hand image of user 1, the original hand image of user 2 and the original hand image of user 3 are thumb image 1, index finger image 2, middle finger image 1, ring finger image 3 and little finger image 3, and then thumb image 1, index finger image 2, middle finger image 1, ring finger image 3 and little finger image 3 are combined to obtain a combined hand image.

[0068] S205 . Perform gesture recognition on each of the at least two combined hand images to obtain a predicted gesture corresponding to each combined hand image.

[0069] The predicted gesture is used to characterize the specific gesture category corresponding to the combined hand image.

[0070] Specifically, after obtaining at least two combined hand images, any one of the at least two combined hand images can be selected as the current combined hand image, and the current combined hand image can be input into a pre-trained gesture recognition model, and then the output value of the gesture recognition model can be determined as the predicted gesture corresponding to the current combined hand image; thereafter, other combined hand images can be selected as the current combined hand image, and the above process can be repeated to obtain the predicted gesture corresponding to each combined hand image.

[0071] S206: Determine a control gesture according to the predicted gesture corresponding to each combined hand image.

[0072] Specifically, after obtaining the predicted gesture corresponding to each combined hand image, in one implementation, the number of each predicted gesture can be determined according to the predicted gesture corresponding to each combined hand image, and the predicted gesture with the largest number can be determined as the control gesture.

[0073] In another implementation, the number of each predicted gesture can be determined according to the predicted gesture corresponding to each combined hand image, and the predicted gestures whose number exceeds a set threshold are determined as control gestures. The set threshold is a preset value used to determine the control gesture, and the user can adjust and set the set threshold according to actual usage requirements. The embodiment of the present application does not specifically limit this. Exemplarily, the set threshold can be the product of the number of combined hand images and a preset multiple, and the preset multiple can be 2 / 3.

[0074] Afterwards, S211 may be executed.

[0075] S207 , respectively obtaining the same type of finger images segmented from different original hand images to obtain multiple types of finger images.

[0076] Specifically, after various types of finger images are segmented from the original hand image, any type can be selected from the various types as the current type, and the finger image corresponding to the current type can be obtained from the finger images segmented from different original hand images to obtain a set of finger images corresponding to the current type; thereafter, other types can be selected as the current type, and the above process can be repeated to obtain a set of finger images corresponding to the various types, that is, multiple categories of finger images.

[0077] Exemplarily, it is known that the in-car image includes user 1, user 2 and user 3, the original hand image of user 1 is {thumb image 1, index finger image 1, middle finger image 1, ring finger image 1, little finger image 1}, the original hand image of user 2 is {thumb image 2, index finger image 2, middle finger image 2, ring finger image 2, little finger image 2}, and the original hand image of user 3 is {thumb image 3, index finger image 3, middle finger image 3, ring finger image 3, little finger image 3}; respectively obtain the same type of finger images segmented from different original hand images, and the process of obtaining multiple types of finger images is as follows: the thumb can be selected as the current type, and the thumb images can be segmented from different original hand images. From the finger images segmented from the initial hand image, the finger images corresponding to the thumb are obtained, and a set of finger images corresponding to the thumb is obtained {thumb image 1, thumb image 2, thumb image 3}; then, other types are selected as the current type, and the above process is repeated to obtain a set of finger images corresponding to the index finger {index finger image 1, index finger image 2, index finger image 3}, a set of finger images corresponding to the middle finger {middle finger image 1, middle finger image 2, middle finger image 3}, a set of finger images corresponding to the ring finger {ring finger image 1, ring finger image 2, ring finger image 3}, and a set of finger images corresponding to the little finger {little finger image 1, little finger image 2, little finger image 3}.

[0078] S208 , fusing each type of finger image in the multiple types of finger images respectively to obtain multiple types of fused finger images.

[0079] The fused finger image is a new finger image obtained by performing feature fusion on finger images of the same type.

[0080] Specifically, after obtaining multiple categories of finger images, any type can be selected as the current type, and then feature extraction is performed on each finger image in the finger image set corresponding to the current type to obtain the posture features of each finger image, and the image fusion technology is used to perform feature fusion on the posture features of each finger image to obtain a fused finger image corresponding to the current type. At this time, the number of finger images with the same posture in the finger image set can be determined, and the posture with the largest number is the same as the posture of the fused finger image, which can eliminate or weaken the noise and interference caused by inaccurate gestures of individual users; thereafter, other types can be selected as the current type, and the above process can be repeated to obtain fused finger images corresponding to various types, that is, multiple categories of fused finger images.

[0081] Exemplarily, given a set of finger images corresponding to the thumb {thumb image 1, thumb image 2, thumb image 3}, a set of finger images corresponding to the index finger {index finger image 1, index finger image 2, index finger image 3}, a set of finger images corresponding to the middle finger {middle finger image 1, middle finger image 2, middle finger image 3}, a set of finger images corresponding to the ring finger {ring finger image 1, ring finger image 2, ring finger image 3}, and a set of finger images corresponding to the little finger {little finger image 1, little finger image 2, little finger image 3}; a process of fusing each type of finger image in the multiple types of finger images to obtain multiple types of fused finger images That is: thumb image 1, thumb image 2 and thumb image 3 can be fused to obtain a fused finger image corresponding to the thumb; index finger image 1, index finger image 2 and index finger image 3 can be fused to obtain a fused finger image corresponding to the index finger; middle finger image 1, middle finger image 2 and middle finger image 3 can be fused to obtain a fused finger image corresponding to the middle finger; ring finger image 1, ring finger image 2 and ring finger image 3 can be fused to obtain a fused finger image corresponding to the ring finger; little finger image 1, little finger image 2 and little finger image 3 can be fused to obtain a fused finger image corresponding to the little finger.

[0082] S209: Combine multiple types of fused finger images to obtain a combined hand image.

[0083] Specifically, after obtaining multiple types of fused finger images, the multiple types of fused finger images can be combined into a complete hand using image stitching technology to obtain a combined hand image. Exemplarily, the fused finger image corresponding to the thumb, the fused finger image corresponding to the index finger, the fused finger image corresponding to the middle finger, the fused finger image corresponding to the ring finger, and the fused finger image corresponding to the little finger can be combined into a complete hand to obtain a combined hand image.

[0084] S210: Perform gesture recognition on the combined hand image to obtain a control gesture corresponding to the combined hand image.

[0085] Specifically, the implementation process and technical principle of S210 are the same as those of S130, and reference may be made to the description of S130 above, which will not be repeated here. Afterwards, S211 may be executed.

[0086] It should be noted that S203 to S206 and S207 to S210 are performed in parallel and can be selected according to actual conditions; and there is no obvious order of execution of S203 to S206 and S207 to S210, and the execution order can be determined according to actual conditions, which is not specifically limited in this embodiment.

[0087] S211, controlling the vehicle computer of the current vehicle to execute a vehicle computer control instruction corresponding to the control gesture.

[0088] The technical solution of the embodiment of the present application can obtain the in-car image captured by the camera of the current vehicle, and identify the original hand images of at least two users from the in-car image, and then segment various types of finger images from the original hand image, and then combine the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image, and perform gesture recognition on the combined hand image to obtain a control gesture corresponding to the combined hand image, that is, different types of finger images segmented from different original hand images can be obtained respectively, and different types of finger images can be combined to obtain at least two combined hand images, and then gesture recognition can be performed on each of the at least two combined hand images to obtain a predicted gesture corresponding to each combined hand image, and a control gesture can be determined according to the predicted gesture corresponding to each combined hand image. By combining different types of finger images, more diverse combined hand images can be created, and by performing independent gesture recognition on multiple combined hand images, multiple predicted gestures can be obtained, and then Mutual verification between multiple predicted gestures can reduce misrecognition caused by inaccurate gestures of a single user, thereby improving the accuracy of determining control gestures, thereby improving the accuracy and reliability of gesture recognition in a multi-user environment; alternatively, finger images of the same type segmented from different original hand images can be obtained separately to obtain multiple categories of finger images, and each category of finger images in the multiple categories of finger images can be fused separately to obtain multiple categories of fused finger images, and then the multiple categories of fused finger images are combined to obtain a combined hand image, and gesture recognition is performed on the combined hand image to obtain the control gesture corresponding to the combined hand image. By fusing multiple categories of finger images, feature information of fingers of the same type from different users can be integrated, which helps to eliminate or reduce noise and interference caused by inaccurate gestures of individual users, thereby improving the accuracy and reliability of gesture recognition in a multi-user environment; through the above two methods, the control gestures in a multi-user environment can be accurately determined, thereby improving the accuracy and efficiency of vehicle control, thereby enhancing the smoothness of interaction and user satisfaction.

[0089] The following further describes a vehicle control method provided by an embodiment of the present application. Figure 3 This is another flow chart of the vehicle control method provided by the embodiment of the present application. The embodiment of the present application is a detailed explanation of determining the wrong gestures of at least two users and reminding them of the wrong gestures after the vehicle control unit of the current vehicle executes the vehicle control instruction corresponding to the control gesture. Figure 3 The method of this embodiment includes but is not limited to the following steps:

[0090] Optionally, after the vehicle computer controlling the current vehicle executes the vehicle computer control instruction corresponding to the control gesture, the posture score corresponding to the original hand image can be determined according to the control gesture, which provides a basis for the subsequent determination of the erroneous hand image, thereby improving the accuracy of determining the erroneous hand image.

[0091] S310: Acquire a combined hand image corresponding to a control gesture to obtain a standard hand image.

[0092] The standard hand image is a combined hand image whose gesture category is a control gesture.

[0093] Specifically, when there are multiple combined hand images, the combined hand image in which the gesture is predicted to be a control gesture can be determined as a standard hand image; when there is one combined hand image, the combined hand image can be determined as a standard hand image.

[0094] S320: Determine a posture score corresponding to each finger image in the original hand image according to the standard hand image, and determine a posture score corresponding to the original hand image according to the posture score corresponding to each finger image.

[0095] Among them, the posture score is used to characterize the difference between the posture of each finger in the original hand image and the posture of the corresponding finger in the standard hand image; the higher the posture score, the smaller the difference between the corresponding original hand image and the standard hand image, that is, the more standard the corresponding original hand image is.

[0096] Specifically, for each original hand image of at least two users, any finger image can be selected from the current original hand image as the current finger image, and the posture score corresponding to the current finger image can be determined based on the standard hand image. Then other finger images can be selected as the current finger images, and the above process can be repeated to obtain the posture score corresponding to each finger image in the current original hand image. Thereafter, the sum of the posture scores corresponding to each finger image can be calculated to obtain the posture score corresponding to the current original hand image.

[0097] Furthermore, the posture score corresponding to each finger image in the original hand image is determined according to the standard hand image, including Sa1-Sa4:

[0098] Sa1. For each finger image, determine the type corresponding to the current finger image and obtain the target type.

[0099] The target type is the finger type corresponding to the current finger image.

[0100] Specifically, any one finger image may be selected from the finger images segmented from the original hand image as the current finger image, and the type corresponding to the current finger image may be determined as the target type.

[0101] Sa2. Replace the finger image of the target type in the standard hand image with the current finger image to obtain a reconstructed hand image.

[0102] The reconstructed hand image is a new hand image obtained by replacing a finger image of a specific type (ie, target type) with another finger image (ie, current finger image) based on an existing standard hand image.

[0103] Specifically, after obtaining the target type, semantic segmentation technology can be used to segment the standard hand image, and finger images of the target type can be segmented from the standard hand image. Then, image fusion technology is used to smoothly merge the current finger image and the rest of the standard hand image to reduce the unnatural feeling after replacement and obtain a reconstructed hand image.

[0104] Exemplarily, it is known that the original hand image of user 1 is {thumb image 1, index finger image 1, middle finger image 1, ring finger image 1, little finger image 1}, and the standard hand image is {standard thumb image 1, standard index finger image 1, standard middle finger image 1, standard ring finger image 1, standard little finger image 1}; if the current finger image is thumb image 1, and the type corresponding to the current finger image is thumb, then the standard thumb image 1 in the standard hand image can be replaced with thumb image 1 to obtain a reconstructed hand image, i.e., {thumb image 1, standard index finger image 1, standard middle finger image 1, standard ring finger image 1, standard little finger image 1}.

[0105] Sa3. Perform gesture recognition on the reconstructed hand image to obtain a predicted gesture corresponding to the reconstructed hand image.

[0106] Specifically, after the reconstructed hand image is obtained, the reconstructed hand image may be input into a pre-trained gesture recognition model, and the output value of the gesture recognition model is determined as the predicted gesture corresponding to the reconstructed hand image.

[0107] Sa4. Determine the posture score corresponding to the current finger image based on the predicted gesture and control gesture corresponding to the reconstructed hand image.

[0108] Specifically, after obtaining the predicted gesture corresponding to the reconstructed hand image, it can be determined whether the predicted gesture corresponding to the reconstructed hand image is the same as the control gesture; when the predicted gesture corresponding to the reconstructed hand image is the same as the control gesture, the posture score corresponding to the current finger image is determined to be the first set score; when the predicted gesture corresponding to the reconstructed hand image is different from the control gesture, the posture score corresponding to the current finger image is determined to be the second set score. The first set score and the second set score are pre-set values, which can be adjusted and set according to actual usage requirements, and the first set score is greater than the second set score.

[0109] Afterwards, other finger images can be selected as current finger images from the finger images segmented from the original hand image, and Sa1-Sa4 can be repeatedly executed to obtain the posture score corresponding to each finger image in the original hand image.

[0110] Exemplarily, it is known that the original hand image of user 1 is {thumb image 1, index finger image 1, middle finger image 1, ring finger image 1, little finger image 1}, the standard hand image is {standard thumb image 1, standard index finger image 1, standard middle finger image 1, standard ring finger image 1, standard little finger image 1}, and the reconstructed hand image 1 is {thumb image 1, standard index finger image 1, standard middle finger image 1, standard ring finger image 1, standard little finger image 1}. The first setting score is -1, and the second setting score is -2; if the predicted gesture corresponding to the reconstructed hand image 1 is the same as the control gesture, then the posture score corresponding to the thumb image 1 is -1; if the predicted gesture corresponding to the reconstructed hand image 1 is different from the control gesture, then the posture score corresponding to the thumb image 1 is -2.

[0111] In an embodiment of the present application, replacing the current finger image with the finger image of the same type in the standard hand image can generate a more accurate reconstructed hand image, and then performing gesture recognition can accurately determine the posture score corresponding to each finger image, which provides data support for the subsequent determination of the posture score of the original hand image, thereby improving the accuracy of determining the erroneous hand image; and, the posture scoring mechanism can quantitatively evaluate the degree of deviation between each finger image and the correct gesture, thereby making the gesture evaluation more objective and specific.

[0112] S330: Determine an original hand image whose posture score is less than a preset score threshold as an erroneous hand image.

[0113] The preset scoring threshold is a preset threshold used to determine an erroneous hand image, and the user can adjust and set the preset scoring threshold according to actual usage requirements, and the present application embodiment does not specifically limit this. For example, the preset scoring threshold may be -5.

[0114] The erroneous hand image is an original hand image whose gesture category is different from the control gesture.

[0115] Specifically, after obtaining the posture score corresponding to the original hand image, the relationship between the posture score corresponding to the original hand image and the preset score threshold can be determined. If the posture score is less than the preset score threshold, the original hand image corresponding to the posture score can be determined as an incorrect hand image; if the posture score is not less than the preset score threshold, the original hand image corresponding to the posture score can be determined as a correct hand image.

[0116] S340: Generate gesture adjustment information based on the erroneous hand image and the control gesture.

[0117] Among them, the gesture adjustment information is a series of feedback information guiding the user on how to adjust the gesture to achieve the correct gesture, which is generated by analyzing the difference between the user's original hand image and the combined hand image corresponding to the control gesture (i.e., the image corresponding to the correct gesture); the gesture adjustment information can be presented in the form of graphics, text, animation or sound, etc., aiming to help users quickly and intuitively understand and correct incorrect gestures.

[0118] Specifically, after obtaining the erroneous hand image, the erroneous hand image can be compared with the combined hand image corresponding to the control gesture to analyze the differences between the two, including aspects such as finger position and joint bending angle. Then, based on the comparison results, targeted gesture adjustment information is generated. The gesture adjustment information at this time may include: which fingers need to be moved and how the joints should be bent, etc.

[0119] S350: Display gesture adjustment information to prompt the user to adjust the incorrect gesture.

[0120] Specifically, after the gesture adjustment information is generated, the gesture adjustment information can be displayed in an appropriate form, for example, in the form of intuitive animation demonstration, detailed text description or voice prompt, so as to prompt the user to adjust the incorrect gesture; among them, the animation demonstration can show the transformation process from an incorrect gesture to a correct gesture, helping the user to intuitively understand the adjustment method; the text description can describe each adjustment step in detail for the user to read and refer to; the voice prompt can broadcast each adjustment step in detail to provide prompts when the user cannot view the screen.

[0121] Optionally, after the user adjusts the gesture according to the displayed gesture adjustment information, the vehicle controller can prompt the user to re-execute the gesture and recognize it again to verify the adjustment effect; if the adjustment is successful, positive feedback (such as "the gesture is correct") can be given; if it is still incorrect, gesture adjustment information can continue to be provided.

[0122] In the technical solution provided in the embodiment of the present application, a combined hand image corresponding to a control gesture can be obtained to obtain a standard hand image, which provides a unified and standardized benchmark for gesture evaluation and can eliminate errors caused by different evaluation criteria, thereby improving the accuracy and consistency of gesture evaluation results; then, the posture score corresponding to each finger image in the original hand image can be determined based on the standard hand image, and the posture score corresponding to the original hand image can be determined based on the posture score corresponding to each finger image. By performing a separate posture score on each finger image in the original hand image, the posture accuracy of each finger can be evaluated more finely, which helps to discover specific finger posture problems. , providing more targeted guidance for subsequent gesture adjustments, and improving the accuracy of determining the posture score corresponding to the original hand image; then, the original hand image with a posture score less than a preset score threshold can be determined as an erroneous hand image, and the erroneous gesture can be quickly identified, thereby improving the efficiency and accuracy of erroneous gesture recognition, thereby improving the accuracy of gesture recognition in a multi-user environment; thereafter, gesture adjustment information can be generated based on the erroneous hand image and the control gesture, and the gesture adjustment information can be displayed to prompt the user to adjust the erroneous gesture, which can provide the user with personalized gesture adjustment guidance and help the user quickly adjust the gesture posture, thereby improving the user experience.

[0123] Figure 4 is a structural diagram of a vehicle control device provided in an embodiment of the present application, referring to Figure 4 , the vehicle control device may include:

[0124] The hand recognition module 410 is used to obtain the in-vehicle image captured by the camera of the current vehicle, and recognize the original hand images of at least two users from the in-vehicle image;

[0125] A combining module 420, configured to segment various types of finger images from the original hand image, and to combine the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image, wherein each finger image in the combined hand image corresponds to a different type;

[0126] A gesture recognition module 430 is used to perform gesture recognition on the combined hand image to obtain a control gesture corresponding to the combined hand image;

[0127] The control module 440 is used to control the vehicle computer of the current vehicle to execute the vehicle computer control instruction corresponding to the control gesture.

[0128] In one embodiment, the combining module 420 combines the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image, including:

[0129] Respectively obtain different types of finger images segmented from different original hand images;

[0130] Finger images of different types are combined to obtain at least two combined hand images.

[0131] In one embodiment, the gesture recognition module 430 is specifically used for:

[0132] Performing gesture recognition on each of the at least two combined hand images respectively to obtain a predicted gesture corresponding to each combined hand image;

[0133] The control gesture is determined according to the predicted gesture corresponding to each combined hand image.

[0134] In one embodiment, the combining module 420 combines the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image, including:

[0135] Respectively obtain the same type of finger images segmented from different original hand images to obtain multiple types of finger images;

[0136] Each type of finger image in the multiple types of finger images is fused respectively to obtain multiple types of fused finger images;

[0137] Multiple types of fused finger images are combined to obtain a combined hand image.

[0138] In one embodiment, the vehicle control device further includes an error image determination module, and the error image determination module is specifically used to:

[0139] Determine a posture score corresponding to the original hand image according to the control gesture;

[0140] An original hand image whose posture score is less than a preset score threshold is determined as an erroneous hand image.

[0141] In one embodiment, the error image determination module determines the posture score corresponding to the original hand image according to the control gesture, including:

[0142] Acquire the combined hand image corresponding to the control gesture to obtain a standard hand image;

[0143] The posture score corresponding to each finger image in the original hand image is determined according to the standard hand image, and the posture score corresponding to the original hand image is determined according to the posture score corresponding to each finger image.

[0144] In one embodiment, the error image determination module determines the posture score corresponding to each finger image in the original hand image according to the standard hand image, including:

[0145] For each finger image, determine the type corresponding to the current finger image and obtain the target type;

[0146] Replace the finger image of the target type in the standard hand image with the current finger image to obtain a reconstructed hand image;

[0147] Performing gesture recognition on the reconstructed hand image to obtain a predicted gesture corresponding to the reconstructed hand image;

[0148] The posture score corresponding to the current finger image is determined according to the predicted gesture and the control gesture corresponding to the reconstructed hand image.

[0149] In one embodiment, the error image determination module determines a posture score corresponding to the current finger image according to the predicted gesture and the control gesture corresponding to the reconstructed hand image, including:

[0150] When the predicted gesture corresponding to the reconstructed hand image is the same as the control gesture, determining the posture score corresponding to the current finger image as a first set score;

[0151] When the predicted gesture corresponding to the reconstructed hand image is different from the control gesture, determining the posture score corresponding to the current finger image as a second set score;

[0152] Among them, the first setting score is greater than the second setting score.

[0153] In one embodiment, the vehicle control device further includes an adjustment prompt module, and the adjustment prompt module is specifically used for:

[0154] generating gesture adjustment information based on the erroneous hand image and the control gesture;

[0155] Display gesture adjustment information to prompt users to adjust incorrect gestures.

[0156] In one embodiment, the gesture recognition module 430 determines the control gesture according to the predicted gesture corresponding to each combined hand image, including:

[0157] determining the number of each predicted gesture according to the predicted gesture corresponding to each combined hand image;

[0158] The predicted gesture with the largest number is determined as the control gesture.

[0159] In one embodiment, the type corresponding to the finger image in the combination module 420 is determined by:

[0160] Determine the pixel area corresponding to the finger image;

[0161] Determine the relative position of the finger image according to the pixel area, the relative position being the positional relationship between the finger image and the palm in the original hand image;

[0162] The type corresponding to the finger image is determined according to the relative position.

[0163] In one embodiment, the types corresponding to the finger images in the combination module 420 include thumb, index finger, middle finger, ring finger and little finger.

[0164] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0165] The vehicle control device provided in this embodiment can be applied to the vehicle control method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0166] Figure 5 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. Figure 5 A block diagram of an exemplary vehicle 11 suitable for implementing embodiments of the present application is shown. Figure 5 The vehicle 11 shown is merely an example and should not bring any limitation to the function and scope of use of the present embodiment.

[0167] like Figure 5 As shown, the vehicle 11 is embodied in the form of a general purpose computing electronic device. The components of the vehicle 11 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting various system components (including the system memory 28 and the processing unit 16).

[0168] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0169] The vehicle 11 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the vehicle 11, including volatile and non-volatile media, removable and non-removable media.

[0170] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The vehicle 11 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical medium) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.

[0171] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0172] The vehicle 11 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the vehicle 11, and / or any device that enables the vehicle 11 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the vehicle 11 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20.

[0173] like Figure 5 As shown, the network adapter 20 communicates with other modules of the vehicle 11 via the bus 18. It should be understood that although Figure 5 Not shown, other hardware and / or software modules may be used in conjunction with the vehicle 11, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0174] The processing unit 16 executes various functional applications and page displays by running the programs stored in the system memory 28, for example, implementing a vehicle control method provided by any embodiment of the present application.

[0175] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, a vehicle control method provided by any embodiment of the present application is implemented, for example.

[0176] The computer storage medium of this embodiment can adopt any combination of one or more computer-readable media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing programs, which can be used by instruction execution systems, devices or devices or used in combination with them.

[0177] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0178] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0179] Computer program code for performing the operation of the present application can be written in one or more programming languages ​​or combinations thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0180] It should be understood by those skilled in the art that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0181] In addition, the acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0182] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the inventive concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A vehicle control method, characterized in that: The method comprises: Acquire an in-vehicle image captured by a camera of the current vehicle, and identify original hand images of at least two users from the in-vehicle image; Segmenting various types of finger images from the original hand image, and combining the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image, wherein each finger image in the combined hand image corresponds to a different type; Performing gesture recognition on the combined hand image to obtain a control gesture corresponding to the combined hand image; The vehicle computer of the current vehicle is controlled to execute the vehicle computer control instruction corresponding to the control gesture.

2. The vehicle control method according to claim 1, characterized in that: The combining the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image includes: Respectively obtaining different types of finger images segmented from different original hand images; The finger images of the different types are combined to obtain at least two combined hand images.

3. The vehicle control method according to claim 2, characterized in that: The performing gesture recognition on the combined hand image to obtain a control gesture corresponding to the combined hand image includes: Performing gesture recognition on each of the at least two combined hand images respectively to obtain a predicted gesture corresponding to each combined hand image; The control gesture is determined according to the predicted gesture corresponding to each combined hand image.

4. The vehicle control method according to claim 1, characterized in that: The combining the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image includes: Respectively acquiring the finger images of the same type segmented from different original hand images to obtain multiple types of finger images; fusing each type of finger image in the multiple types of finger images respectively to obtain multiple types of fused finger images; The multiple types of fused finger images are combined to obtain the combined hand image.

5. The vehicle control method according to claim 1, characterized in that: The method further comprises: Determining a posture score corresponding to the original hand image according to the control gesture; An original hand image whose posture score is less than a preset score threshold is determined as an erroneous hand image.

6. The vehicle control method according to claim 5, characterized in that: The step of determining a posture score corresponding to the original hand image according to the control gesture comprises: Acquire a combined hand image corresponding to the control gesture to obtain a standard hand image; The posture score corresponding to each finger image in the original hand image is determined according to the standard hand image, and the posture score corresponding to the original hand image is determined according to the posture score corresponding to each finger image.

7. The vehicle control method according to claim 6, characterized in that: The step of determining a posture score corresponding to each finger image in the original hand image according to the standard hand image includes: For each of the finger images, determine the type corresponding to the current finger image to obtain a target type; Replacing the finger image of the target type in the standard hand image with the current finger image to obtain a reconstructed hand image; Performing gesture recognition on the reconstructed hand image to obtain a predicted gesture corresponding to the reconstructed hand image; A posture score corresponding to the current finger image is determined according to the predicted gesture corresponding to the recombined hand image and the control gesture.

8. The vehicle control method according to claim 7, characterized in that: The step of determining a posture score corresponding to the current finger image according to the predicted gesture corresponding to the reconstructed hand image and the control gesture comprises: When the predicted gesture corresponding to the recombined hand image is the same as the control gesture, determining the posture score corresponding to the current finger image to be a first set score; When the predicted gesture corresponding to the recombined hand image is different from the control gesture, determining the posture score corresponding to the current finger image to be a second set score; The first setting score is greater than the second setting score.

9. The vehicle control method according to claim 5, characterized in that: The method further comprises: generating gesture adjustment information based on the erroneous hand image and the control gesture; The gesture adjustment information is displayed to prompt the user to adjust the incorrect gesture.

10. The vehicle control method according to claim 3, characterized in that: The determining the control gesture according to the predicted gesture corresponding to each combined hand image includes: determining the number of each predicted gesture according to the predicted gesture corresponding to each combined hand image; The predicted gesture with the largest number is determined as the control gesture.

11. The vehicle control method according to claim 1, characterized in that: The type corresponding to the finger image is determined in the following manner: Determine a pixel area corresponding to the finger image; Determining a relative position of the finger image according to the pixel area, the relative position being a positional relationship between the finger image and a palm in the original hand image; The type corresponding to the finger image is determined according to the relative position.

12. The vehicle control method according to claim 1, characterized in that: The types corresponding to the finger images include thumb, index finger, middle finger, ring finger and little finger.

13. A vehicle control device, characterized in that: The device comprises: A hand recognition module, used to obtain an in-vehicle image captured by a camera of the current vehicle, and recognize original hand images of at least two users from the in-vehicle image; A combining module, configured to segment various types of finger images from the original hand image, and to combine the finger images segmented from different original hand images based on the types corresponding to the finger images to obtain a combined hand image, wherein each finger image in the combined hand image corresponds to a different type; A gesture recognition module, used to perform gesture recognition on the combined hand image to obtain a control gesture corresponding to the combined hand image; The control module is used to control the vehicle computer of the current vehicle to execute the vehicle computer control instruction corresponding to the control gesture.

14. A vehicle, characterized in that: The vehicle comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the vehicle control method described in any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the vehicle control method as claimed in any one of claims 1 to 12 is implemented.

Images