Gesture interaction method and device, vehicle and storage medium

Abstract

The invention provides a gesture interaction method and device, a vehicle and a storage medium, and relates to the technical field of vehicles. If it is detected that the target image comprises at least two gesture regions, determining whether a target face region exists in the target image; if the target face region exists, determining a target gesture region in the at least two gesture regions according to a distance relationship between the target face region and each gesture region; and identifying the target gesture area to obtain a gesture type so as to control the vehicle cabin. According to the scheme, the vehicle cabin can be effectively controlled under the condition that at least two gestures exist, so that the use experience of a user is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and more specifically, to a gesture interaction method, device, vehicle and storage medium in the field of vehicle technology. Background Art

[0002] Gesture interaction, also known as gesture control or gesture recognition, is a technical means of controlling a device by capturing the position, direction, and speed of a user's gesture. At present, gesture interaction technology plays an important role in vehicles. By recognizing the user's gestures, various devices in the cabin can be controlled without a touch screen or physical buttons. For example, by recognizing gestures, you can interact with the windows and sunroof shades in the cabin. Therefore, the directionality of gestures is crucial in gesture interaction.

[0003] However, the related art can only identify the direction of the gesture. However, if there are at least two gestures, there may be a control disorder, so that the cabin cannot be effectively controlled, affecting the user experience.

[0004] Therefore, how to effectively control the vehicle cabin in the presence of at least two gestures is a technical problem that urgently needs to be solved. Summary of the invention

[0005] The present application provides a gesture interaction method, device, vehicle and storage medium. The method can achieve effective control of the vehicle cabin in the presence of at least two gestures to improve the user experience.

[0006] In a first aspect, a gesture interaction method is provided, the method comprising:

[0007] Acquire the target image in the cabin;

[0008] If it is detected that the target image includes at least two gesture areas, determining whether there is a target face area in the target image;

[0009] If there is a target face region, determining a target gesture region in at least two gesture regions according to a distance relationship between the target face region and each gesture region;

[0010] The target gesture area is identified and the gesture type is obtained to control the vehicle cabin.

[0011] In an embodiment of the present application, by determining whether a target face area exists in the target image, the target face area can be used as a reference point for determining a target gesture area when the target face area exists. Since the target face area is used to determine effective information of at least two gesture areas, the target face area can be used as a reference point to determine the target gesture area from at least two gesture areas based on the distance relationship between the target face area and each gesture area. Since the cabin can be controlled based on the target gesture area, the embodiment of the present application can achieve effective control of the cabin when at least two gestures exist, thereby improving the user experience.

[0012] In combination with the first aspect, in certain implementations of the first aspect, a target gesture area among at least two gesture areas is determined based on the target face area, including: determining the distance between each gesture area among the at least two gesture areas and the target face area to obtain at least two distance values; and determining the gesture area corresponding to the smallest distance value among the at least two distance values ​​as the target gesture area.

[0013] In the embodiments of the present application, since the distances between different gesture areas and the target face area are different, by determining the distance between each gesture area and the target face area, it is possible to ensure that the target gesture area for controlling the cabin can be obtained from at least two gesture areas, thereby avoiding control disorder caused by too many gesture areas.

[0014] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, determining whether there is a target face area in the target image includes: if there is a face area in the target image, determining the face area as the target face area; if there are at least two face areas in the target image, determining the target face area from the at least two face areas according to preset information; wherein the preset information is the face area or a preset face image.

[0015] In an embodiment of the present application, the target face area is determined by preset information, which can provide a reference point for later ensuring that a valid gesture area is identified from at least two gesture areas, thereby avoiding control disorder caused by too many gesture areas.

[0016] In combination with the first aspect and the above-mentioned implementation methods, in certain implementation methods of the first aspect, a target face area is determined from at least two face areas based on preset information, including: if the preset information is the face area, the face area with the largest area in at least two face areas is determined as the target face area; if the preset information is a preset face image, the face area in at least two face areas that matches the preset face image is determined as the target face area.

[0017] In the embodiment of the present application, two methods for determining the target face region are provided to ensure that the target face region is identified from at least two face regions, so as to provide a reference point for identifying the effective gesture region.

[0018] In combination with the first aspect and the above-mentioned implementation methods, in certain implementation methods of the first aspect, if the target face area does not exist, the method provided in the embodiment of the present application also includes: determining the target gesture area from at least two gesture areas based on the area of ​​the gesture area; or, determining the target gesture area whose image information in at least two gesture areas matches the preset gesture image based on the preset gesture image and the image information of at least two gesture areas.

[0019] In an embodiment of the present application, by providing two methods for determining the target gesture area, it is possible to ensure that a gesture area for controlling the vehicle cabin can be obtained from at least two gesture areas in the absence of a target face area, thereby avoiding control disorder caused by too many gesture areas.

[0020] In combination with the first aspect and the above-mentioned implementation methods, in certain implementation methods of the first aspect, if there is no target face area, the method provided in the embodiment of the present application also includes: identifying the gesture type corresponding to each gesture area in at least two gesture areas; determining whether the valid gesture types in all gesture types are the same; wherein the valid gesture type is the gesture type for controlling the cabin; if all valid gesture types are the same, the cabin is controlled according to the valid gesture type; if there are different valid gesture types among all valid gesture types, the cabin is not controlled.

[0021] In an embodiment of the present application, a method for controlling a vehicle cabin when a target face area does not exist is provided, thereby achieving diversity in recognizing effective gestures.

[0022] In combination with the first aspect and the above-mentioned implementation methods, in certain implementation methods of the first aspect, the method provided in the embodiment of the present application also includes: obtaining sound source information and signal instructions in the target voice information; wherein the sound source information is used to indicate the user's position in the cabin, and the signal instructions are used to identify the control method of the cabin; identifying the target gesture area to obtain the gesture type to control the cabin, including: identifying the target gesture area to obtain the gesture type; determining the target object to be controlled in the cabin according to the gesture type and the sound source information; and controlling the target object according to the signal instruction.

[0023] In the embodiments of the present application, the efficiency of controlling the vehicle cabin is further improved through target voice information and target gesture areas.

[0024] In a second aspect, a gesture interaction device is provided, the device comprising:

[0025] An acquisition module, used for acquiring a target image in the cabin;

[0026] A determination module, configured to determine whether a target face region exists in the target image if it is detected that the target image includes at least two gesture regions;

[0027] The determination module is further used to determine, if there is a target face region, a target gesture region in at least two gesture regions according to a distance relationship between the target face region and each gesture region;

[0028] The recognition module is used to recognize the target gesture area and obtain the gesture type to control the vehicle cabin.

[0029] In a possible implementation, the determination module is further used to determine the distance between each of the at least two gesture areas and the target face area to obtain at least two distance values; and determine the gesture area corresponding to the smallest distance value among the at least two distance values ​​as the target gesture area.

[0030] In one possible implementation, the determination module is also used to determine the face area as the target face area if there is a face area in the target image; if there are at least two face areas in the target image, determine the target face area from the at least two face areas according to preset information; wherein the preset information is the face area or a preset face image.

[0031] In one possible implementation, the determination module is also used to, if the preset information is the face area, determine the face area with the largest area among at least two face areas as the target face area; if the preset information is a preset face image, determine the face area among at least two face areas that matches the preset face image as the target face area.

[0032] In one possible implementation, the determination module is further used to determine a target gesture area from at least two gesture areas based on the area of ​​the gesture area; or, based on a preset gesture image and image information of at least two gesture areas, determine a target gesture area whose image information matches a preset gesture image.

[0033] In a possible implementation, the identification module is further configured to identify a gesture type corresponding to each of the at least two gesture areas.

[0034] In a possible implementation, the determination module is further used to determine whether the valid gesture types among all gesture types are the same; wherein the valid gesture type is a gesture type for controlling the vehicle cabin.

[0035] In a possible implementation, the gesture interaction device also includes: a control module, which is used to control the vehicle cabin according to the valid gesture type if all valid gesture types are the same; if different valid gesture types exist among all valid gesture types, the vehicle cabin is not controlled.

[0036] In a possible implementation, the acquisition module is also used to acquire sound source information and signal instructions in the target voice information; wherein the sound source information is used to indicate the position of the user in the cabin, and the signal instructions are used to identify the control method of the cabin.

[0037] In a possible implementation, the control module is further used to identify the target gesture area to obtain the gesture type; determine the target object to be controlled in the cabin according to the gesture type and sound source information; and control the target object according to the signal instruction.

[0038] In a third aspect, a vehicle is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the vehicle executes the method in the first aspect or any possible implementation of the first aspect.

[0039] In a fourth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect.

[0040] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a schematic diagram of a scenario provided by an embodiment of the present application;

[0042] Figure 2 This is a scenario diagram of a technical problem provided by an embodiment of the present application;

[0043] Figure 3 is a flowchart of a gesture interaction method provided in an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of a hand detection algorithm provided in an embodiment of the present application;

[0045] Figure 5 is a schematic diagram of gesture type recognition provided by an embodiment of the present application;

[0046] Figure 6 is a schematic diagram of a target image provided in an embodiment of the present application;

[0047] Figure 7 This is a schematic diagram of the first gesture area provided in an embodiment of the present application;

[0048] Figure 8 is a schematic diagram of a second gesture area provided in an embodiment of the present application;

[0049] Fig. 9 This is a schematic diagram of the third gesture area provided in an embodiment of the present application;

[0050] Fig.10 This is a schematic diagram of a multi-mode voice gesture interaction process provided by an embodiment of the present application;

[0051] Fig.11 is a flowchart of a gesture interaction method provided in an embodiment of the present application;

[0052] Fig.12 is a structural schematic diagram of a gesture interaction device provided in an embodiment of the present application;

[0053] Fig.13 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0055] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0056] Gesture interaction, also known as gesture control or gesture recognition, is a technical means of controlling a device by capturing the position, direction, and speed of a user's gestures.

[0057] Gesture recognition is a technology that uses images to identify the position and category of hands in an image. Its algorithmic technology route is to first detect the gesture area in the image, then crop the gesture area separately, and use a classifier model to identify the gesture type.

[0058] The advantages of spatial gesture interaction in the cockpit are mainly reflected in the following aspects: First, intuitiveness and naturalness: Gesture interaction is an intuitive and natural way of interaction. It is based on human daily gestures, allowing users to interact with the cockpit system more intuitively. This way of interaction reduces the user's learning cost and improves the convenience of use. Second, adapt to different scenarios: Gesture interaction is suitable for different driving scenarios. Whether in the process of driving or in the parking state, users can easily control various functions in the cockpit through gestures. Third, improve interaction efficiency: Gesture interaction is usually faster than voice interaction, especially in scenarios that require quick response. In addition, gesture interaction also has the advantage of cross-space communication and can achieve interaction at a longer distance. Fourth, flexibility and scalability: The gesture interaction system can continuously update and expand the functions of the gesture interaction system through firmware upgrades to meet the changing needs of users. At the same time, the flexibility of gesture interaction also enables it to adapt to the needs of different models and cockpit designs.

[0059] With the continuous advancement of technology and the continuous expansion of applications, gesture interaction will play a more important role in future cockpit systems. By recognizing the user's gestures, various devices in the cabin can be controlled without touch screens or physical buttons. For example, spatial perception gesture interaction technology based on IR infrared sensors can be used to recognize the user's gestures to control various functions in the cabin, bringing users a more convenient and intelligent interactive experience.

[0060] For example, Figure 1 As shown, Figure 1 A schematic diagram of a gesture interaction scenario provided in an embodiment of the present application. A user may be sitting in a vehicle cabin. For example, in the cabin, the user may interact with the windows and sunroof shades in the cabin by recognizing the user's gestures.

[0061] like Figure 1 As shown in (a) in FIG. 1 , when the user's gesture is directed upward, it can be determined that the sunroof sunshade in the cabin is interacted with. For example, when the sunroof sunshade is closed, if it is recognized that the user's gesture is directed upward, it can be determined that the sunroof sunshade is opened; when the sunroof sunshade is open, if it is recognized that the user's gesture is directed upward, it can be determined that the sunroof sunshade is closed.

[0062] like Figure 1As shown in (b) in FIG. 1 , when the user's gesture is directed to the left, it can be determined that the left window in the cabin is interacted with. For example, when the left window is closed, if it is recognized that the user's gesture is directed to the left, it can be determined that the left window is opened; when the left window is open, if it is recognized that the user's gesture is directed to the left, it can be determined that the left window is closed.

[0063] like Figure 1 As shown in (c) in FIG. 1 , when the user's gesture is directed to the right, it can be determined that the right window in the cabin is interacted with. For example, when the right window is closed, if it is recognized that the user's gesture is directed to the right, it can be determined that the right window is opened; when the right window is open, if it is recognized that the user's gesture is directed to the left, it can be determined that the right window is closed.

[0064] However, the related art can only identify the direction of the gesture. If there are at least two hands in the image, there may be control disorder, which makes it impossible to effectively control the cabin, affecting the user experience. For example, there may be two hands of a user in the image. Whether the user makes two gestures at the same time or the user makes a gesture with only one hand, it may lead to the recognition of two gestures, and it is impossible to know which gesture is used for control; or, there may be two users in the image, or the hands of at least two users may be recognized, which may lead to the recognition of at least two hands, and the same problem exists.

[0065] For example, Figure 2 As shown, Figure 2 It is a scenario diagram of a technical problem provided in an embodiment of the present application. Figure 2 Take the case where there is a user in the image as an example.

[0066] For example, Figure 2 In (a), there are two hands of the user. One hand of the user is naturally placed on the armrest of the cockpit, which is recognized as a gesture; the other hand makes a gesture pointing upward, which leads to control disorder and it is impossible to know which gesture is used for control.

[0067] For example, Figure 2 In (b), there are two hands of the user. One hand of the user makes a gesture pointing upward, and the other hand makes a gesture pointing rightward, which also leads to control disorder and it is impossible to know which gesture is used for control.

[0068] For example, Figure 2 In (c), there are two hands of the user. Both hands of the user make gestures pointing upward. However, this still causes control disorder, and it is impossible to know which gesture is used for control.

[0069] Therefore, how to effectively control the vehicle cabin in the presence of at least two gestures is a technical problem that urgently needs to be solved.

[0070] Based on this, the solution provided by the present application is a gesture interaction method, device, vehicle and storage medium. The solution determines a target gesture area in at least two gesture areas according to a target face area in a target image, and then identifies the target gesture area to obtain a gesture type to control the vehicle cabin. The solution can effectively control the vehicle cabin in the presence of at least two gestures to improve the user experience.

[0071] It should be understood that the embodiments of the present application do not specifically limit the execution subject of the gesture interaction method, as long as the program that records the code of the gesture interaction method of the embodiments of the present application can be run to communicate according to the gesture interaction method of the embodiments of the present application. For example, the execution subject of the gesture interaction method provided by the embodiments of the present application can be the vehicle described in the above embodiments, or a device in the vehicle; or a gesture interaction device used in the vehicle, such as a chip.

[0072] like Figure 3 As shown, Figure 3 A flowchart of a gesture interaction method provided for the implementation of the present application, the method includes S301 to S304.

[0073] S301. Acquire a target image in the vehicle cabin.

[0074] Exemplarily, the gesture area may be an area determined by determining the position of the hand in the image through image recognition.

[0075] In actual application scenarios, the hand detection algorithm can be used to identify the gesture area in the target image. Figure 4 As shown, Figure 4 Schematic diagram of a hand detection algorithm. The hand detection algorithm may be a YOLO (You Only LookOnce) algorithm, such as the yoloV5 algorithm.

[0076] It should be explained that the YOLO algorithm is a real-time object detection system that simultaneously predicts the target bounding box and its category probability in the target image through a single convolutional neural network (CNN).

[0077] Optionally, the hand detection algorithm may also be other detection algorithms, such as SSD (Single Shot MultiBox Detector), R-CNN (Region-based Convolutional Neural Network) and other detection algorithms, and the embodiments of the present application do not impose specific restrictions on this.

[0078] Exemplarily, the gesture type of the gesture area can be determined by a gesture type recognition algorithm, and then the direction of the gesture can be determined. The gesture type can include a gesture pointing to the right, a gesture pointing to the left, a gesture pointing upward, a gesture pointing downward, etc. The gesture type can be a valid gesture type, which can realize the control of the vehicle cabin; the gesture type can also be an invalid gesture type, which cannot realize the control of the vehicle cabin.

[0079] In the embodiment of the present application, the description may be made by taking the example that the valid gesture types include a gesture pointing to the right, a gesture pointing to the left, and a gesture pointing upwards. Correspondingly, a gesture pointing downwards is an invalid gesture type.

[0080] In actual application scenarios, the gesture type recognition algorithm can be a classification network algorithm. For example, the classification network algorithm can be based on a lightweight convolutional neural network model such as MobileNet, and specifically use the backbone network algorithm (Backbone Network) in the neural network model to extract features to determine the gesture type; it can also determine the user's gaze point or gaze direction through the eye gaze estimation network (Eye Gaze Estimation Network), further improving the accuracy of determining the gesture type.

[0081] For example, Figure 5 As shown, Figure 5 The figure is a schematic diagram of gesture type recognition. After the target gesture area is identified by the hand detection algorithm, the gesture type recognition algorithm can be performed on the gesture area in the target image based on the convolutional neural network model. Through the backbone network algorithm module, the convolutional layer module (Convolutional Layer) and the fully connected layer (Fully Connected Layer, FC), the gesture area is identified as a string of machine codes that can be executed by the computer, such as "0100".

[0082] For example, the camera installed in the vehicle cabin can capture the user's gesture and obtain the target image. The camera can be installed in a position that can capture each seat in the vehicle cabin, and thus can capture the gesture of the user sitting in each seat, such as the roof console, instrument panel, etc. Alternatively, the camera can also be installed in a position that can capture the preset seat in the vehicle cabin, and thus can capture the gesture of the user sitting in the preset seat. The embodiment of the present application does not impose specific restrictions on this.

[0083] S302. If it is detected that the target image includes at least two gesture areas, determine whether there is a target face area in the target image.

[0084] Exemplarily, the target face region may also be a region determined by determining the position of the face in the image through image recognition.

[0085] In actual application scenarios, similar to the hand detection algorithm, the face detection algorithm can be used to identify the target face area in the target image. The face detection algorithm can also be the YOLO algorithm.

[0086] Optionally, the face detection algorithm may also be other detection algorithms, such as SSD, R-CNN and other detection algorithms, and the embodiments of the present application do not impose specific restrictions on this.

[0087] S303. If there is a target face region, determine a target gesture region among the at least two gesture regions according to a distance relationship between the target face region and each gesture region.

[0088] Exemplarily, the distance relationship may refer to the size relationship between the distances between at least two gesture areas and the target face area. For example, the distance value between the target face area and gesture area A is X1, and the distance value between the target face area and gesture area B is X2, and X1 is greater than X2; then the distance relationship between the target face area and gesture area A and gesture area B is: the distance between gesture area A and the target face area is greater than the distance between gesture area B and the target face area.

[0089] It should be noted that in the related art, the line between the hand and the face can be directed toward the display; then, the face that is not facing the display is eliminated to obtain the second face and the second hand; then the second hand is compared with the preset hand gesture to determine the hand with the highest similarity as the valid hand, and the number of valid hands can be multiple. In other words, the related art determines the valid hand by the angle between the line between the hand and the face and the display, and the face needs to be facing the target object to control the target object through gestures. The present application can determine the target gesture area through the distance relationship between the target face area and each gesture area; there is no need to judge the positional relationship between the face and the hand and the display. Compared with the related art, the present application can determine the target gesture area more conveniently. Moreover, the related art does not limit the number of valid hands; there is still a problem of control disorder.

[0090] For a detailed description of S303, please refer to the following embodiments, which will not be described in detail here.

[0091] S304. Identify the target gesture area and obtain the gesture type to control the vehicle cabin.

[0092] Exemplarily, the gesture type can be used to indicate the direction of the gesture. For example, the gesture type can be used to indicate that the direction of the gesture is upward, the direction of the gesture is left, the direction of the gesture is right, etc.

[0093] Exemplarily, the embodiment of the present application can control the cabin by combining signal instructions and target gesture areas. For example, the signal instruction can be a voice instruction output by the user through voice, and the voice instruction may not contain the target object to be controlled. For example, "open this", "close this". The target gesture area can determine the target object to be controlled. For example, if the gesture type corresponding to the gesture type area is pointing upward, the sunroof shade is controlled.

[0094] For example, if the signal instruction is: "Open this", and the gesture type in the target gesture area is pointing to the left, the left window is opened. If the signal instruction is: "Close this", and the gesture type in the target gesture area is pointing upward, the sunroof blind is closed.

[0095] In an embodiment of the present application, by determining whether there is a target face area in the target image, the target face area can be used as a reference point for determining a target gesture area when there is a target face area; since the target face area is used to determine valid information of at least two gesture areas, the target face area can be used as a reference point to determine the target gesture area from at least two gesture areas based on the distance relationship between the target face area and each gesture area; since the vehicle cabin can be controlled based on the target gesture area, the embodiment of the present application can identify valid gestures from at least two gestures, thereby improving the user experience.

[0096] In a possible embodiment of the present application, the above S303 includes: determining the distance between each of at least two gesture areas and the target face area to obtain at least two distance values; and determining the gesture area corresponding to the smallest distance value among the at least two distance values ​​as the target gesture area.

[0097] Exemplarily, the distance between two areas (such as a gesture area and a target face area) can be determined by the center coordinates of each area or by the angular coordinates (lower left corner coordinates) of each area. The embodiments of the present application do not impose any specific restrictions on this.

[0098] Exemplarily, the distance between two regions (such as a gesture region and a target face region) can determine the two-dimensional distance between the two regions in the target image; the three-dimensional distance between the two regions can also be determined by computer vision and three-dimensional reconstruction, which is not specifically limited in the present application. As for how to determine the three-dimensional distance between two regions by computer vision and three-dimensional reconstruction, reference can be made to the prior art, which will not be elaborated here.

[0099] For example, take the distance determined by the center coordinates of each area as an example, Figure 6 As shown, Figure 6 A schematic diagram of a target image provided in an embodiment of the present application, the target image includes a target face area 610, a gesture area 620 and a gesture area 630:

[0100] like Figure 6 As shown in (a), the two-dimensional coordinates of the target face region 610 are (28, 83); the two-dimensional coordinates of the gesture region 620 are (18, 65); and the two-dimensional coordinates of the gesture region 630 are (35, 65). Among them, the distance value a1 between the target face region 610 and the gesture region 620 is 20.59; and the distance value a2 between the target face region 610 and the gesture region 630 is 19.31. Since the distance value a2 is less than the distance value a1, the gesture region 630 corresponding to the distance value a2 is determined as the target gesture region.

[0101] like Figure 6 As shown in (b), the two-dimensional coordinates of the target face region 610 are (28, 83, 340); the two-dimensional coordinates of the gesture region 620 are (18, 65, 310); and the two-dimensional coordinates of the gesture region 630 are (35, 65, 305). Among them, the distance value a1 between the target face region 610 and the gesture region 620 is 10.98; and the distance value a2 between the target face region 610 and the gesture region 630 is 15.32. Since the distance value a1 is less than the distance value a2, the gesture region 630 corresponding to the distance value a1 is determined as the target gesture region.

[0102] In the embodiments of the present application, since the distances between different gesture areas and the target face area are different, by determining the distance between each gesture area and the target face area, it is possible to ensure that the target gesture area for controlling the cabin can be obtained from at least two gesture areas, thereby avoiding control disorder caused by too many gesture areas.

[0103] In a possible embodiment of the present application, the above S302 includes: if there is a face area in the target image, determining the face area as the target face area; if there are at least two face areas in the target image, determining the target face area from the at least two face areas according to preset information.

[0104] The preset information is the face area or the preset face image. Thus, by providing two methods for determining the target face area, it is ensured that the target face area is identified from at least two face areas, so as to provide a reference point for identifying the effective gesture area.

[0105] As an example, if the preset information is the area of ​​the face region, the face region with the largest area among at least two face regions is determined as the target face region.

[0106] Optionally, the face region with the smallest face area may be determined as the target face region from at least two face regions. Optionally, the face region with the face area closest to the preset face region may be determined as the target face region from at least two face regions.

[0107] As another example, when the preset information is a preset face image, the preset face image and at least two face regions can be compared respectively, and a face region in the at least two face regions that matches the face in the preset face image is determined as the target face region. The number of preset faces can be one or more.

[0108] Exemplarily, in the case where there are multiple preset faces, a priority can be set for each preset face. For example, if there are two face regions among at least two face regions that match the face in the preset face image, the target face region is determined according to the priority of the preset face image corresponding to each face region. For example, face region 1 matches preset face image 1, and face region 2 matches preset face image 2. If the priority of preset face image 1 is greater than that of preset face image 2, face region 1 can be determined as the target face region.

[0109] It is understandable that the presence of at least two face regions in the target image may be a situation where there are at least two people in the vehicle cabin. At this time, it is necessary to accurately identify which face region is a valid face region, thereby providing a reference point for later identification of a valid gesture region.

[0110] In an embodiment of the present application, the target face area is determined by preset information, which can provide a reference point for later ensuring that a valid gesture area is identified from at least two gesture areas, thereby avoiding control disorder caused by too many gesture areas.

[0111] In a possible embodiment of the present application, if there is no target face area, the method provided in the embodiment of the present application also includes: determining the target gesture area from at least two gesture areas based on the area of ​​the gesture area; or, determining the target gesture area whose image information in at least two gesture areas matches the preset gesture image based on the preset gesture image and the image information of at least two gesture areas.

[0112] As an example, when determining the target gesture area based on the area of ​​the gesture area, the gesture area with the largest area among at least two gesture areas can be used as the target gesture area; optionally, the gesture area with the smallest area among at least two gesture areas can also be used as the target gesture area; of course, the gesture area among at least two gesture areas whose area is closest to the preset gesture area can also be used as the target gesture area.

[0113] As another example, when determining the target gesture area according to a preset gesture image, the image information in at least two gesture areas can be compared with the preset gesture images respectively, and the gesture area whose image information in at least two gesture areas matches the preset gesture images can be determined as the target gesture area. The number of preset gesture images can be one or more.

[0114] Exemplarily, when there are multiple preset gesture images, a priority can be set for each preset gesture image. For example, if there are image information in at least two gesture areas and the image information in two gesture areas matches the preset gesture image, the target gesture area is determined according to the priority of the preset gesture image corresponding to the image information in each gesture area. For example, image information 1 of the gesture area matches preset gesture image 1, and image information 2 of the gesture area matches preset gesture image 2. If the priority of preset gesture image 1 is greater than that of preset gesture image 2, gesture area 1 can be determined as the target gesture area.

[0115] In an embodiment of the present application, by providing two methods for determining the target gesture area, it is possible to ensure that a gesture area for controlling the vehicle cabin can be obtained from at least two gesture areas in the absence of a target face area, thereby avoiding control disorder caused by too many gesture areas.

[0116] In a possible embodiment of the present application, if there is no target face area, the method provided in the embodiment of the present application also includes: identifying the gesture type corresponding to each gesture area in at least two gesture areas; determining whether the valid gesture types in all gesture types are the same; if all valid gesture types are the same, controlling the cabin according to the valid gesture types; if different valid gesture types exist among all valid gesture types, not controlling the cabin.

[0117] Among them, the valid gesture type is the gesture type for controlling the vehicle cabin.

[0118] It is understandable that, as described in the above embodiments, when there is no target face region, the target gesture region can be determined from at least two gesture regions according to the gesture region area or the preset gesture image. In this embodiment, another method is provided to solve the technical problem of how to determine the target gesture region when there is no target face region.

[0119] Exemplarily, the valid gesture type may be a gesture type that can control the vehicle cabin, for example: the valid gesture type may be a gesture type pointing upwards (to control the sunroof shade in the vehicle cabin), a gesture type pointing to the left (to control the left window in the vehicle cabin), or a gesture type pointing to the right (to control the right window in the vehicle cabin).

[0120] It is not difficult to understand that an invalid gesture type may refer to a gesture type that cannot control the cabin. For example, if the gesture type is a gesture pointing downward, or a gesture type that does not point in any direction (e.g., 1 Fig. 9 ), or other gesture types.

[0121] Exemplarily, if there is an invalid gesture type among the gesture types corresponding to at least two gesture areas, a valid gesture type is obtained from the gesture types corresponding to the at least two gesture areas, and then the gesture recognition result is determined according to the valid gesture type.

[0122] Exemplarily, for example, if the gesture types corresponding to the two valid gesture areas are the same gesture type (for example, a gesture type pointing upward), the cabin is controlled (for example, the sunroof sunshade is controlled) according to the gesture type corresponding to the valid gesture area; if the gesture types corresponding to the two gesture areas are different gesture types (for example, a gesture type pointing upward and a gesture type pointing to the left), the recognition results of at least two gesture areas are currently invalid, and the cabin is not controlled. Of course, in the case of only one valid gesture type, the gesture recognition result indicates the valid gesture type.

[0123] For example, Figure 7 As shown, the target image includes a gesture area 710 and a gesture area 720, and the gesture type corresponding to the gesture area 710 and the gesture type corresponding to the gesture area 720 are valid gesture types. Since the gesture type corresponding to the gesture area 710 is a gesture type pointing upward, and the gesture type corresponding to the gesture area 720 is a gesture type pointing upward, the two are the same gesture type, so the sunroof shade is controlled according to the gesture type pointing upward.

[0124] For example, Figure 8 As shown, the target image includes a gesture area 810 and a gesture area 820, and the gesture type corresponding to the gesture area 810 and the gesture type corresponding to the gesture area 820 are valid gesture types. Since the gesture type corresponding to the gesture area 810 is a gesture type pointing to the left, and the gesture type corresponding to the gesture area 820 is a gesture type pointing upward, the two are different gesture types, and the cabin is not controlled.

[0125] It should be noted that if it is identified that there is an invalid gesture type in the gesture type corresponding to each of the at least two gesture areas, a valid gesture type is screened out from the at least two gesture types, and the vehicle cabin is controlled according to the valid gesture type.

[0126] For example, Fig. 9 As shown, the target image includes gesture area 910 and gesture area 920. The gesture type corresponding to gesture area 910 is an invalid gesture type, and the gesture type corresponding to gesture area 920 is an upward-pointing gesture type. A valid gesture type is screened out from gesture area 910 and gesture area 920, that is, the gesture type corresponding to gesture area 920 (the upward-pointing gesture type); then, the sunroof shade is controlled according to the upward-pointing gesture type.

[0127] For a detailed description of the gesture recognition result, please refer to the following scenario embodiment.

[0128] In actual application scenarios, it can be described by the following code logic. The gesture type is determined by placeholders, including 001, 010, and 100. 001 can be used to represent a gesture type pointing to the left, 010 to represent a gesture type pointing upward, and 100 to represent a gesture type pointing to the right. The preset gesture information can include 001, 010, and 100.

[0129] The specific code logic is as follows:

[0130] (1) Initialize m_gesture_result=0.

[0131] It should be noted that the following is a circular structure logic.

[0132] (2) Traverse each gesture area and proceed to 3)-5).

[0133] (3) Use gesture type recognition algorithm to identify the label of gesture type.

[0134] Exemplarily, the labels include: 1, 2, 3; the number of the label is used to indicate the position where 1 is located. Among them, 1 (001) can be used to indicate a gesture type pointing to the left, 2 (010) can be used to indicate a gesture type pointing upward, and 3 (100) can be used to indicate a gesture type pointing to the right.

[0135] (4) Initialize curr_gesture_result = 0. If the current gesture recognition result is a valid gesture type, max_index_cls∈[1,3], then update the curr_gesture_result value and calculate it through the placeholder.

[0136] For example, curr_gesture_result may indicate the type of gesture currently recognized. m_gesture_result=0 may be used to indicate the gesture recognition result described below.

[0137] (5) Bitwise OR m_gesture_result with curr_gesture_result to update m_gesture_result.

[0138] (6) Return the m_gesture_result value as the output result.

[0139] It needs to be explained that Bitwise OR is a binary operation that operates on the bits of two numbers, and the result is 1 if at least one of the two corresponding bits is 1, otherwise it is 0. This operation is often used in computer science and programming, especially low-level programming or processing bitmaps.

[0140] For the detailed description of (1) to (6) above, please refer to the following two examples.

[0141] As an example, the gesture area includes gesture area 1 to gesture area 3. Among them, gesture area 1 corresponds to a gesture type pointing upward, gesture area 2 corresponds to a gesture type pointing upward, and gesture area 3 corresponds to a gesture type pointing downward. Initialize m_gesture_result = 0, and initialize curr_gesture_result = 0. First, identify the label of the gesture type corresponding to gesture area 1 as 010, update the curr_gesture_result value to 010 (i.e., step (4)), at this time m_gesture_result = 0; bitwise OR m_gesture_result with curr_gesture_result, update m_gesture_result = 010 (i.e., step (5)). Then, the label of the gesture type corresponding to the gesture area 2 is identified as 010, and the curr_gesture_result value is updated to 010 (i.e., step (4)). At this time, m_gesture_result = 010; m_gesture_result is bitwise ORed with curr_gesture_result, and m_gesture_result = 010 is updated (i.e., step (5)); then, the gesture type corresponding to the gesture area 3 is identified as an invalid gesture type, and the curr_gesture_result value is not updated (i.e., step (4)). Return m_gesture_result = 010 as the output result (i.e., the gesture recognition result) (i.e., step (6)). Since the gesture recognition result is 010, which matches the preset gesture information, it can be determined that the gesture recognition result is a gesture type pointing upward.

[0142] As another example, the gesture area includes gesture area 1 to gesture area 3. Among them, gesture area 1 corresponds to a gesture type pointing upward, gesture area 2 corresponds to a gesture type pointing upward, and gesture area 3 corresponds to a gesture type pointing to the left. Initialize m_gesture_result = 0, and initialize curr_gesture_result = 0. First, identify the label of the gesture type corresponding to gesture area 1 as 010, update the value of curr_gesture_result to 010 (i.e., step (4)), at this time m_gesture_result = 0; bitwise OR m_gesture_result with curr_gesture_result, update m_gesture_result = 010 (i.e., step (5). Then, identify the label of the gesture type corresponding to gesture area 2 as 010, update the value of curr_gesture_result to 010 (i.e., step (4)), at this time m_gesture_result = 0; bitwise OR m_gesture_result with curr_gesture_result, update m_gesture_result = 010 (i.e., step (5)). The result is bitwise ORed with curr_gesture_result, and m_gesture_result is updated to 010 (i.e., step (5). Next, the label of the gesture type corresponding to the gesture area 3 is identified as 001, and the value of curr_gesture_result is updated to 001 (i.e., step (4)). At this time, m_gesture_result = 010; m_gesture_result is bitwise ORed with curr_gesture_result, and m_gesture_result is updated to 011 (i.e., step (5). Since the gesture recognition result is 011, which does not match the preset gesture information, the cabin is not controlled.

[0143] It should be explained that the preset gesture information may include 001, 010, and 100. If the gesture types indicated by at least two gesture areas are the same gesture type, then no matter how many times m_gesture_result is bitwise ORed with curr_gesture_result, the final updated m_gesture_result will match the preset gesture information. If there are different gesture types in the gesture types indicated by at least two gesture areas, when m_gesture_result is bitwise ORed with curr_gesture_result, that is, when any two different placeholders are bitwise ORed, the result obtained must have at least two bits that are 1, such as 110, 011, 101, etc., which does not match the preset gesture information.

[0144] Optionally, in an embodiment of the present application, when there is a target face area in the target image, a gesture recognition result can be determined according to at least two gesture types, and the vehicle cabin can be controlled according to the gesture type corresponding to the gesture recognition result.

[0145] In an embodiment of the present application, a method for controlling a vehicle cabin when a target face area does not exist is provided, thereby achieving diversity in recognizing effective gestures.

[0146] In a possible embodiment of the present application, the method provided in the embodiment of the present application also includes: obtaining sound source information and signal instructions in the target voice information; the above 304 includes: identifying the target gesture area to obtain the gesture type; determining the target object to be controlled in the cabin according to the gesture type and sound source information; and controlling the target object according to the signal instruction.

[0147] Among them, the sound source information is used to indicate the user's position in the cabin, and the signal command is used to identify the control method of the cabin.

[0148] Exemplarily, the vehicle is a hatchback as an example: specifically, the front row includes two seats (seat A and seat B), and the back row includes two seats (seat C and seat D). Seat A is adjacent to the left front window, seat B is adjacent to the right front window, seat C is adjacent to the left rear window, and seat D is adjacent to the right rear window. The sound source information can be used to determine which seat the user is in, and then further determine the specific window in the cabin to be controlled. For example, if the sound source information is used to indicate that the user is in seat C in the cabin, it can be determined that the left rear window, the right rear window, or the sunroof curtain can be controlled; if the sound source information is used to indicate that the user is in seat A in the cabin, it can be determined that the left front window, the right front window, or the sunroof curtain can be controlled. Of course, in order to further determine the target object, the target object can be determined according to the gesture type and the sound source information. For details, please refer to the following.

[0149] Exemplarily, the target object may be a sunshade of a skylight or a car window; optionally, the target object may be other equipment, such as an air conditioner, a car light, etc., and the embodiments of the present application do not impose specific limitations on this.

[0150] As an example, take the vehicle as a hatchback: if the gesture type corresponding to the received target gesture area is a gesture type pointing to the left, and the sound source information indicates that the user is near the left front window, then based on the gesture type and the sound source information, it is determined that the target object to be controlled in the cabin is the left front window.

[0151] As another example, take the vehicle as a hatchback: if the gesture type corresponding to the received target gesture area is a gesture type pointing to the right, and the sound source information indicates that the user is near the left front window, then based on the gesture type and the sound source information, it is determined that the target object to be controlled in the cabin is the right front window.

[0152] It can be understood that controlling the cabin according to the target gesture area and the sound source position can also be called: multi-modal voice gesture interaction. Specifically: by tracking the position of the hand in space, determine the gesture area; identify the gesture type corresponding to the gesture area; and then combine the voice information to control the cabin.

[0153] It is understandable that, since the signal instruction can be used to further determine whether the operation performed on the target object is open or closed, the accuracy of controlling the window can be improved. Of course, if the signal instruction is: "Open this", and the target object (such as the window) itself is in an open state, the window can be kept in an open state. At the same time, the user can also be prompted by voice that the window is currently in an open state.

[0154] As an example, if the target object is the right rear window, if the gesture type is a gesture pointing to the right, and the voice command is "open this", the right rear window can be opened. If the gesture type is a gesture pointing to the right, and the voice command is "close this", the right rear window can be closed.

[0155] It is understandable that the effect experience of multimodal voice gesture interaction depends on the accuracy and robustness of voice recognition and directional gesture (gesture type) recognition.

[0156] For example, Fig.10 As shown, Fig.10 The figure is a schematic diagram of a multi-modal voice gesture interaction process.

[0157] S1001. The gesture recognition module captures the user's gestures through a camera to obtain a target image.

[0158] Exemplarily, the gesture may be a gesture pointing upward.

[0159] S1002. The voice module obtains target voice information.

[0160] Exemplarily, the target voice information may be “turn this on” or “turn this off”.

[0161] S1003. The gesture recognition module recognizes the target image and determines the gesture type of the gesture.

[0162] Exemplarily, in the case where the gesture may be a gesture pointing upward, the gesture type is an upward direction.

[0163] S1004. The voice module determines the sound source information and signal instructions according to the target voice information.

[0164] Exemplarily, the sound source information can determine the user's position in the vehicle cabin, such as the left rear seat.

[0165] S1005. The gesture recognition module sends the gesture type to the in-vehicle service module.

[0166] S1006. The service module determines the user status according to the gesture type.

[0167] For example, during driving, the driver may accidentally make a waving gesture, which may be misunderstood by the system as an operation request. Therefore, the service module determines the user status according to the gesture type.

[0168] S1007. The service module sends the user status and gesture type to the voice module.

[0169] S1008. The voice module determines whether the user status is valid.

[0170] S1009. The voice module sends the gesture type to the control module.

[0171] S1010. The control module controls the target object according to the gesture type.

[0172] Exemplarily, when the gesture type is in the upward direction, the sunroof sunshade is controlled, that is, the target object is the sunroof sunshade.

[0173] For detailed description of S1001 to S1010, please refer to Figures 3 to 9 As well as the relevant descriptions above, I will not repeat them here.

[0174] In the embodiments of the present application, the efficiency of controlling the vehicle cabin is further improved through target voice information and target gesture areas.

[0175] like Fig.11 As shown, Fig.11 A flowchart of a gesture interaction method provided by an embodiment of the present application is described by taking a gesture interaction method as an example:

[0176] Exemplarily, the execution subject of S1101 to S1113 below may be a vehicle, and of course may also be a gesture interaction device used in a vehicle, such as a chip, and the embodiments of the present application do not impose specific limitations on this.

[0177] S1101. Acquire a target image.

[0178] The target image includes at least one gesture frame.

[0179] Exemplarily, the gesture frame may refer to the gesture area mentioned above.

[0180] S1102. Determine whether the number of gesture frames is greater than 1; if so, execute 1104; if not, execute 1103.

[0181] S1103. Determine that the gesture frame is a target gesture frame.

[0182] S1104. Determine whether there is a face frame; if so, execute S1105; if not, execute S1109.

[0183] Exemplarily, the face frame may refer to the face region mentioned above.

[0184] S1105. Determine whether the number of face frames is greater than 1; if so, execute S1106; if not, execute S1107.

[0185] S1106. Determine the face frame with the largest area from among the at least two face frames as the target face frame.

[0186] S1107. Determine that the face frame is a target face frame.

[0187] S1108. Determine the gesture frame that is closest to the target face frame among the at least two gesture frames as the target gesture frame.

[0188] S1109. Determine the gesture frame with the largest area among the at least two gesture frames as the target gesture frame.

[0189] S1110. Identify the target gesture frame and obtain the gesture type.

[0190] S1111. Obtain sound source information and signal instructions.

[0191] Exemplarily, the sound source information and the signal instruction may be information included in the target voice information.

[0192] Exemplarily, the sound source information is used to indicate the user's position in the vehicle cabin, and the signal instruction is used to identify the control method for the vehicle cabin.

[0193] S1112. Determine the target object to be controlled in the vehicle cabin according to the target gesture frame and the sound source position.

[0194] Exemplarily, the target object may be a sunshade of a skylight or a car window; optionally, the target object may also be other equipment, such as an air conditioner, a car light, etc.

[0195] S1113. Control the target object according to the gesture type and signal instruction.

[0196] It should be noted that the above Fig.11The specific implementation of S1101 to S1113 can be found in Figures 3 to 10 ; no further description is given here.

[0197] In an embodiment of the present application, the method for determining the target gesture frame is determined by determining the number of gesture frames and target face frames, so that the diversity of identifying effective gestures can be enriched; when there are at least two gesture frames, if there is a target face frame, the target gesture frame can be determined based on the target face frame, and then the target face frame can be used as a reference point to ensure that a valid target gesture frame can be obtained; if there is no target face frame, the target gesture frame can also be directly determined, thereby ensuring that the cabin can be effectively controlled; based on this, the present scheme can recognize effective gestures from at least two gestures to improve the user experience.

[0198] Combination of the above Figures 1 to 11 The gesture interaction method provided by the embodiment of the present application is described in detail; Fig.12 and Fig.13 The device embodiments of the present application are described in detail. It should be understood that the device in the embodiments of the present application can execute the various methods of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0199] Combine the following Fig.12 The gesture interaction device provided in the embodiment of the present application is described in detail.

[0200] like Fig.12 As shown, Fig.12 It is a structural schematic diagram of a gesture interaction device provided in an embodiment of the present application.

[0201] For example, Fig.12 As shown, the device comprises:

[0202] An acquisition module 1210 is used to acquire a target image in the vehicle cabin;

[0203] A determination module 1220, configured to determine whether there is a target face region in the target image if it is detected that the target image includes at least two gesture regions;

[0204] The determination module 1220 is further configured to determine, if there is a target face region, a target gesture region among the at least two gesture regions according to a distance relationship between the target face region and each gesture region;

[0205] The recognition module 1230 is used to recognize the target gesture area and obtain the gesture type to control the vehicle cabin.

[0206] In a possible implementation, the determination module 1220 is further used to determine the distance between each of the at least two gesture areas and the target face area to obtain at least two distance values; and determine the gesture area corresponding to the smallest distance value among the at least two distance values ​​as the target gesture area.

[0207] In one possible implementation, the determination module 1220 is also used to determine the face region as the target face region if there is a face region in the target image; if there are at least two face regions in the target image, determine the target face region from the at least two face regions according to preset information; wherein the preset information is the face region area or a preset face image.

[0208] In one possible implementation, the determination module 1220 is also used to, if the preset information is the face area, determine the face area with the largest area among at least two face areas as the target face area; if the preset information is a preset face image, determine the face area among at least two face areas that matches the preset face image as the target face area.

[0209] In one possible implementation, the determination module 1220 is further used to determine a target gesture area from at least two gesture areas based on the area of ​​the gesture area; or, based on a preset gesture image and image information of at least two gesture areas, determine a target gesture area whose image information matches a preset gesture image.

[0210] In a possible implementation, the identification module 1230 is further configured to identify a gesture type corresponding to each of the at least two gesture areas.

[0211] In a possible implementation, the determination module 1220 is further used to determine whether the valid gesture types among all gesture types are the same; wherein the valid gesture type is a gesture type for controlling the vehicle cabin.

[0212] In a possible implementation, the gesture interaction device also includes: a control module, which is used to control the vehicle cabin according to the valid gesture type if all valid gesture types are the same; if different valid gesture types exist among all valid gesture types, the vehicle cabin is not controlled.

[0213] In a possible implementation, the acquisition module 1210 is also used to obtain sound source information and signal instructions in the target voice information; wherein the sound source information is used to indicate the user's position in the cabin, and the signal instructions are used to identify the control method of the cabin.

[0214] In a possible implementation, the determination module 1220 is further configured to determine a target object to be controlled in the vehicle cabin according to the target gesture area and the sound source position;

[0215] In a possible implementation, the control module is also used to identify the target gesture area to obtain the gesture type; determine the target object to be controlled in the cabin according to the gesture type and sound source information; and control the target object according to the gesture type and signal instructions.

[0216] It should be noted that the gesture interaction device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when executing the gesture interaction method. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0217] In addition, the gesture interaction device and gesture interaction method embodiments provided in the above embodiments belong to the same concept, so for details not disclosed in the device embodiments of this specification, please refer to the gesture interaction method embodiments described above in this specification, and no further details will be given here.

[0218] like Fig.13 As shown, Fig.13 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application.

[0219] Exemplarily, the vehicle 1300 includes: a processor 1310 , a memory 1320 , and an executable program code 1330 ;

[0220] Exemplarily, the memory 1320 is used to store executable program code; the processor 1310 is used to call and execute the executable program code; and the executable program code 1330 is used to implement a gesture interaction method.

[0221] The present application can divide the functional modules of the vehicle according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0222] In the case of dividing each functional module according to each function, the vehicle may include: an acquisition module, a determination module, an identification module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0223] The vehicle provided in the present application is used to execute the above-mentioned gesture interaction method, and thus can achieve the same effect as the above-mentioned implementation method.

[0224] In the case of an integrated unit, the vehicle may further include a storage module, wherein the storage module may be used to support the vehicle to execute relevant program codes and data.

[0225] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present application. The processor may also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a digital signal processor (DSP) and a microprocessor, etc. The storage module may be a memory.

[0226] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the aforementioned embodiments. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a digital versatile disk (DVD), a compact disc read-only memory (CD-ROM), a microdrive, and a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a dynamic random access memory (DRAM), a video random access memory (VRAM), a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0227] The present application also provides a computer program product. When the computer program product runs on a computer, the computer is enabled to execute the above-mentioned related steps to implement a gesture interaction method in the above-mentioned embodiment.

[0228] In addition, the vehicle provided in the embodiments of the present application may specifically be a chip, component or module, and the vehicle may include a connected processor and memory; wherein the memory is used to store instructions, and when the vehicle is running, the processor may call and execute instructions so that the chip executes a gesture interaction method in the above-mentioned embodiments.

[0229] Among them, the vehicle, computer-readable storage medium, computer program product or chip provided in this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0230] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0231] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0232] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A gesture interaction method, characterized in that: The method comprises: Acquire the target image in the cabin; If it is detected that the target image includes at least two gesture areas, determining whether there is a target face area in the target image; If the target face region exists, determining a target gesture region among the at least two gesture regions according to a distance relationship between the target face region and each of the gesture regions; The target gesture area is identified to obtain a gesture type so as to control the vehicle cabin.

2. The method according to claim 1, characterized in that: The determining, according to the distance relationship between the target face region and each of the gesture regions, a target gesture region among the at least two gesture regions comprises: Determine the distance between each of the at least two gesture areas and the target face area to obtain at least two distance values; A gesture area corresponding to a minimum distance value among the at least two distance values ​​is determined as the target gesture area.

3. The method according to claim 1, characterized in that The determining whether there is a target face area in the target image includes: If there is a face region in the target image, determining the face region as the target face region; If the at least two face regions exist in the target image, the target face region is determined from the at least two face regions according to preset information; wherein the preset information is the face region area or a preset face image.

4. The method according to claim 3, characterized in that The step of determining the target face region from at least two face regions according to preset information includes: If the preset information is the area of ​​the face region, determining the face region with the largest area among the at least two face regions as the target face region; If the preset information is the preset face image, a face region among the at least two face regions that matches the preset face image is determined as the target face region.

5. The method according to any one of claims 1 to 4, characterized in that If the target face area does not exist, the method further includes: Determine the target gesture area from the at least two gesture areas according to the area of ​​the gesture area; or, According to the preset gesture image and the image information of the at least two gesture areas, a target gesture area in the at least two gesture areas whose image information matches the preset gesture image is determined.

6. The method according to any one of claims 1 to 4, characterized in that If the target face area does not exist, the method further includes: Identifying a gesture type corresponding to each of the at least two gesture areas; Determining whether the valid gesture types among all the gesture types are the same; wherein the valid gesture type is a gesture type for controlling the vehicle cabin; If all the valid gesture types are the same, controlling the vehicle cabin according to the valid gesture type; If different valid gesture types exist among all the valid gesture types, the vehicle cabin is not controlled.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Acquire sound source information and signal instructions in the target voice information; wherein the sound source information is used to indicate the position of the user in the vehicle cabin, and the signal instructions are used to identify a control method for the vehicle cabin; The identifying the target gesture area and obtaining the gesture type to control the vehicle cabin includes: Identify the target gesture area to obtain a gesture type; determining a target object to be controlled in the vehicle cabin according to the gesture type and the sound source information; The target object is controlled according to the signal instruction.

8. A gesture interaction device, characterized in that: The device comprises: An acquisition module, used for acquiring a target image in the cabin; A determination module, configured to determine whether there is a target face area in the target image if it is detected that the target image includes at least two gesture areas; The determination module is further configured to determine, if the target face region exists, a target gesture region among the at least two gesture regions according to a distance relationship between the target face region and each of the gesture regions; The recognition module is used to recognize the target gesture area and obtain the gesture type to control the vehicle cabin.

9. A vehicle, characterized in that: The vehicle comprises: A memory for storing executable program codes; A processor is used to call and run the executable program code from the memory so that the vehicle executes the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a vehicle, the vehicle is caused to perform the method according to any one of claims 1 to 7.

Images