Vehicle control method, vehicle control device, vehicle and storage medium

By identifying user gestures to automatically adjust components in the vehicle, the driving risk problem caused by manual operation is solved, precise control of components in the vehicle is achieved, and driving safety and user experience are improved.

CN119078708APending Publication Date: 2024-12-06GREAT WALL MOTOR CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411213406.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the prior art, the adjustment of components in the vehicle depends on manual operation, which causes the driver to distract his attention during complex road conditions or high-speed driving, which increases driving risks and makes it difficult to achieve automatic and precise control of components in the vehicle.

Method used

By identifying the spatial position and gesture type of user gestures, the control instructions and control directions of components in the vehicle are determined, and automatic and accurate adjustments of components in the vehicle are achieved to avoid manual operations.

Benefits of technology

Improve driving safety, reduce the risk of drivers manually adjusting vehicle components during driving, and improve driving safety and user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119078708A_ABST
    Figure CN119078708A_ABST
Patent Text Reader

Abstract

The invention provides a vehicle control method, a vehicle control device, a vehicle and a storage medium, the method is applied to the field of vehicles, and the method comprises the steps that if a target gesture in the vehicle is detected, the gesture type and the gesture position of the target gesture are determined; wherein the gesture position is used for indicating the spatial position of the target gesture in the vehicle; based on the gesture type, determining a target component in the vehicle and a target control instruction of the target component; determining a control direction of the control instruction based on the gesture position; and controlling the target component based on the control instruction and the control direction. According to the method, the target state to which the target component needs to be adjusted by the user can be determined according to the type and the position of the target gesture, automatic and accurate adjustment of the component in the vehicle is achieved, the driving risk caused by the fact that a driver needs to manually adjust the vehicle component in the driving process is avoided, and the driving safety is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a vehicle control method, a vehicle control device, a vehicle and a storage medium in the field of vehicle technology. Background Art

[0002] With the continuous development of intelligent automobile technology, the functions of vehicles are becoming more and more abundant. Among them, the layout and design of vehicle components directly affect the user's driving experience.

[0003] In the related art, the adjustment method of vehicle components mainly relies on manual operation by the user; for example, to adjust the air outlet direction of the air conditioner in the vehicle, it is necessary to rely on the user to manually adjust the guide plate of the air outlet to change the air outlet direction. This manual operation method may cause the driver's attention to be distracted when the vehicle is driving; especially when driving in complex road conditions or at high speeds, it increases the risk of driving.

[0004] Therefore, how to achieve automatic and precise control of vehicle components and improve driving safety is a problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides a vehicle control method, a vehicle control device, a vehicle and a storage medium. The method can realize automatic and precise adjustment of components in the vehicle by identifying the spatial position and gesture type of the user's gesture, thereby improving driving safety.

[0006] In a first aspect, a vehicle control method is provided, the method comprising:

[0007] If a target gesture in the vehicle is detected, the gesture type and gesture position of the target gesture are determined; wherein the gesture position is used to indicate the spatial position of the target gesture in the vehicle; based on the gesture type, a target component in the vehicle and a target control instruction for the target component are determined; based on the gesture position, a control direction of the control instruction is determined; based on the control instruction and the control direction, the target component is controlled.

[0008] In an embodiment of the present application, when a user's target gesture is detected in a vehicle, the gesture type and the location of the gesture can be determined through the acquired target gesture; based on the gesture type, the target component in the vehicle and the control instruction of the target component can be determined; based on the gesture position, the control direction of the control instruction can be determined; because when the target component in the vehicle is controlled by gesture, the control direction of the control instruction of the target component by the target gesture can be determined; it can be understood that in the above scheme, not only the control instruction of the target component can be determined based on the gesture, but also the control direction of the control instruction can be determined according to the gesture position of the target gesture; therefore, through the above control instructions and control directions, automatic and precise control of the target component in the vehicle can be achieved; the driving risks brought by the driver's need to manually adjust the vehicle components during driving are avoided, thereby improving driving safety.

[0009] In combination with the first aspect, in some possible implementations, determining a control direction of a control instruction based on a gesture position includes:

[0010] Based on the relative position of the gesture position and the target component, the adjustment angle of the target component is determined; based on the adjustment angle, the control direction of the control instruction is determined.

[0011] In the embodiments of the present application, based on the relative position of the user's gesture position and the target component, the position of the user's gesture position relative to the target component can be determined; based on its position, the angle at which the target component needs to be adjusted can be determined; and the control direction can be determined by adjusting the angle. Since the accurate control direction is determined, the target component can be precisely controlled based on the control instructions and control direction, avoiding the driving risks caused by manual adjustment that cannot adjust the vehicle components to the required state, thereby improving driving safety.

[0012] In combination with the first aspect and the above implementations, in some possible implementations, the present invention further includes:

[0013] Based on the gesture type and mapping relationship of the target gesture, the target component type and the target control instruction in the vehicle are determined; wherein the mapping relationship is used to represent the mapping relationship between the preset gesture type, the component type of the vehicle and the control instruction; the target component type is determined as the target component; or, based on the gesture position, the target area in the vehicle is determined, and the target component type in the target area is determined as the target component.

[0014] In an embodiment of the present application, in one implementation, by identifying the gesture type of the target gesture, the gesture type and the mapping relationship are matched to determine the component type and the target control instruction in the vehicle; all determined component types in the vehicle are determined as target components; because the target component and the target control instruction can be determined based on the gesture type of the target gesture, the user can flexibly control the components in the vehicle, thereby improving the user experience. In another implementation, after determining the component type and the target control instruction in the vehicle, the target area is determined based on the gesture position, and the component type in the target area is determined as the target component, which can control the vehicle components in the area where the user is located, thereby saving the vehicle's computing power.

[0015] In combination with the first aspect and the above implementations, in some possible implementations, if a target gesture in a vehicle is detected, determining a gesture type and a gesture position of the target gesture includes:

[0016] Acquire a first image and a second image; wherein the first image and the second image are images captured by different cameras in the vehicle; determine the gesture type based on the first image and / or the second image; determine the gesture position based on the first image, the second image and the target parameters of the camera; wherein the target parameters include the intrinsic parameters and extrinsic parameters of the camera.

[0017] In an embodiment of the present application, the gesture position of the target gesture can be obtained through the images captured by two cameras; the relative position with respect to the target component can be determined through the gesture position, that is, the control direction of the control instruction can be determined, thereby achieving precise control of the target component based on the control instruction and the control direction, thereby avoiding the driving risks caused by the driver's need to manually adjust the vehicle components during driving, and improving driving safety.

[0018] In combination with the first aspect and the above implementations, in some possible implementations, the present invention further includes:

[0019] If the image captured by the camera does not include the complete image area of ​​the target gesture, obtain the target distance and the seat position of the target user; wherein the target distance is used to represent the distance between the head of the target user and the top of the vehicle's cabin; based on the target distance and / or the seat position, adjust the shooting angle of the camera so that the image captured by the camera includes the complete image area of ​​the target gesture.

[0020] In an embodiment of the present application, when the camera fails to completely capture the image of the target gesture, the shooting angle of the camera is intelligently adjusted by obtaining the distance between the user's head and the top of the vehicle cabin and the user's seat position; therefore, when the camera fails to capture the complete gesture image area, the camera angle can be adjusted in time to ensure the integrity of the gesture image. Since the integrity of the gesture image is ensured, the gesture type can be accurately identified, thereby improving the accuracy of control.

[0021] In combination with the first aspect and the above implementations, some possible implementations include:

[0022] If a target gesture in the vehicle is detected, determine whether the target gesture is a preset gesture; determine the gesture type and gesture position of the target gesture, including: if the target gesture is a preset gesture, determine the gesture type and gesture position of the target gesture.

[0023] In an embodiment of the present application, when a target gesture is detected in a vehicle, it is first determined whether the target gesture belongs to a preset gesture. Only when it is determined that the target gesture is a preset gesture, the gesture type of the target gesture and its spatial position in the vehicle are further determined; therefore, recognition and subsequent component control functions can be performed only when the user's gesture is a preset gesture, thereby avoiding erroneous triggering of control gestures during normal driving of the user, resulting in misjudgment of the user's intentions and erroneous vehicle component control operations.

[0024] In combination with the first aspect and the above implementations, in some possible implementations, controlling the target component based on the control instruction and the control direction includes:

[0025] Based on the control instruction, the target component is controlled to operate in the control direction.

[0026] In an embodiment of the present application, after determining the target component, control instruction and control direction, the target component is controlled to run in the control direction based on the content of the control instruction, which can realize automatic component control, avoid the user's manual adjustment steps, and improve the accuracy of control.

[0027] In a second aspect, a vehicle control device is provided, the device comprising:

[0028] A detection module, used to detect a target gesture in a vehicle;

[0029] A processing module is used to determine the gesture type and gesture position of the target gesture if a target gesture is detected in the vehicle; wherein the gesture position is used to indicate the spatial position of the target gesture in the vehicle; based on the gesture type, determine the target component in the vehicle and the target control instruction of the target component; based on the gesture position, determine the control direction of the control instruction; based on the control instruction and the control direction, control the target component.

[0030] It should be understood that the expansion, limitation, explanation and description of the relevant contents in the above-mentioned first aspect also apply to the same contents in the second aspect.

[0031] In a third aspect, a vehicle is provided, comprising a memory and a processor; the memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the vehicle executes the vehicle control method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0032] In a fourth aspect, a computer program product is provided, which includes: a computer program code, which, when executed on a computer, enables the computer to execute the vehicle control method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0033] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the vehicle control method in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a schematic diagram of an application scenario of a vehicle control method provided in an embodiment of the present application;

[0035] Figure 2 is a schematic flow chart of a vehicle control method provided in an embodiment of the present application;

[0036] Figure 3 is a schematic flow chart of another vehicle control method provided in an embodiment of the present application;

[0037] Figure 4 is a schematic flow chart of another vehicle control method provided in an embodiment of the present application;

[0038] Figure 5 is a schematic diagram of a scenario of a vehicle control method provided in an embodiment of the present application;

[0039] Figure 6 is a schematic diagram of another vehicle control method provided in an embodiment of the present application;

[0040] Figure 7 is a schematic diagram of a scenario of another vehicle control method provided in an embodiment of the present application;

[0041] Figure 8 is a structural schematic diagram of a vehicle control device provided in an embodiment of the present application;

[0042] Fig. 9 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0044] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0045] Combine the following Figure 1 The schematic diagram of the application scenario of this solution is described with examples.

[0046] Figure 1 It is a schematic diagram of an application scenario of a vehicle control method provided in an embodiment of the present application.

[0047] For example, Figure 1 The scene in FIG. 1 includes a camera 110 , a camera 120 , an air conditioner 130 , and a user's gesture 140 .

[0048] In the related art, the adjustment method of vehicle components mainly relies on manual operation by the user; for example, when the user wants to adjust the air outlet direction of the air conditioner in the vehicle, the user needs to manually adjust the guide plate of the air outlet to change the air outlet direction. This manual operation method may cause the driver's attention to be distracted when the vehicle is driving, especially when driving in complex road conditions or at high speeds, which increases the risk of driving.

[0049] In view of this, the embodiments of the present application provide a vehicle control method, a vehicle control device, a vehicle and a storage medium; through the vehicle control method provided by the embodiments of the present application, when a user's target gesture is detected in the vehicle, the gesture type and the location of the gesture can be determined through the acquired target gesture; for example, the gesture type and the gesture location are determined through the image of the user gesture 140 captured by the camera 110 and the camera 120; based on the gesture type, the target component in the vehicle and the control instruction of the target component can be determined; for example, through the mapping relationship between the preset gesture type corresponding to the gesture type and the target component and the control instruction, it is determined that the target component is an air conditioner and the control instruction is to adjust the wind direction; based on the gesture location, the control direction of the control instruction can be determined; because the control direction is determined based on the gesture location when the user's target gesture is acquired, the user does not need to manually adjust the target component when controlling the target component, and the target state to which the user needs to adjust the target component can be determined through the type and location of the target gesture, thereby realizing automatic and precise adjustment of the components in the vehicle, avoiding the driving risks caused by the driver's need to manually adjust the vehicle components during driving, and improving driving safety.

[0050] Combine the following Figure 2 A vehicle control method provided in an embodiment of the present application is described in detail.

[0051] Figure 2 is a schematic flow chart of a vehicle control method provided in an embodiment of the present application. Figure 2 As shown, the method 200 includes S210 to S240, and S210 to S240 are described in detail below.

[0052] For example, Figure 2 The illustrated vehicle control method 200 may be executed by a vehicle; or, executed by a processor in a vehicle; or, executed by a chip mounted in a processor in a vehicle.

[0053] S210: If a target gesture in the vehicle is detected, determine the gesture type and gesture position of the target gesture.

[0054] The gesture position is used to indicate the spatial position of the target gesture in the vehicle. It can be understood that the gesture position refers to the three-dimensional spatial position of the target gesture in the vehicle cabin.

[0055] In one implementation, the method includes:

[0056] Acquire a first image and a second image; determine a gesture type based on the first image and / or the second image; and determine a gesture position based on the first image, the second image, and a target parameter of the camera.

[0057] It should be noted that the first image and the second image are images captured by different cameras in the vehicle.

[0058] In an embodiment of the present application, hand detection is performed on the first image and / or the second image based on the first image captured by the first camera and / or the second image captured by the second camera in the vehicle to obtain a target image area in the first image and / or the second image, and gesture recognition is performed on the target image area to determine the user's gesture type; based on parameters such as the first image, the second image, and the intrinsic and extrinsic parameters of the camera, the gesture position is determined according to a stereoscopic vision algorithm and a trigonometric function relationship.

[0059] Exemplarily, the vehicle is an electric vehicle, a hybrid vehicle or a fuel vehicle; the embodiments of the present application are not limited to this; the target gesture refers to the gesture movement of the user's hand; for example, the gesture movement includes an OK gesture or a number 1 gesture, or other gesture movements.

[0060] For example, the camera may be an occupant monitoring system (OMS) camera, a binocular camera or a wide-angle camera of the vehicle. The number of cameras may be two or more, and images from multiple perspectives may be captured. The camera parameters include intrinsic parameters and extrinsic parameters. The intrinsic parameters refer to parameters such as focal length and optical center position. The extrinsic parameters refer to parameters such as the relative position, rotation angle, and relative posture between cameras. These parameters may be obtained through calibration when the camera is installed, or may be obtained based on the current position and adjustment angle of the camera.

[0061] It should be noted that the camera needs to be set at different positions so that it can capture the user's hand image in the vehicle; for example, for the driver's area, it can be set above the dashboard, slightly tilted downward at an angle of 30 degrees, or it can be set on the top of the center console, offset 45 degrees toward the driver's seat, so that the driver's hand image can be captured; the camera should have the characteristics of high resolution, high frame rate and low noise; for example, a camera with a resolution of 1920×1080 pixels and a frame rate of 60 frames / second is selected to ensure the clarity and real-time nature of the image.

[0062] Among them, hand detection is an image segmentation algorithm used to determine the image area where the user's hands are located from the image.

[0063] Exemplarily, feature extraction is performed on the acquired first image and / or second image to obtain image features, and multiple feature areas of the image features are classified to obtain a target image area. Hand recognition is an image classification algorithm used to determine the gesture type corresponding to the hand; for example, based on the sub-image features corresponding to the target image area obtained after hand detection, the target image area is classified to obtain a reference gesture, and the user's gesture type is determined based on a preset reference gesture set; or, key point recognition is performed on the target image area to obtain multiple hand key points, and the gesture posture composed of the multiple hand key points is determined, and the user's gesture type is obtained according to the reference gesture corresponding to the gesture posture. Stereoscopic vision algorithm refers to shooting the same scene from different perspectives through different cameras, and obtaining three-dimensional information using the parallax information between the captured images; for example, after obtaining images collected by different cameras, an image block matching or feature matching algorithm is used to obtain corresponding points of the same object in different images, and parallax information is obtained according to the coordinate difference of the corresponding points in different images, and three-dimensional information is obtained according to the calculated parallax information, the focal length of different cameras, and the distance between different cameras, and finally the three-dimensional structure of the scene is reconstructed according to the three-dimensional information.

[0064] Exemplarily, after the first image captured by the first camera and / or the second image captured by the second camera, the acquired images are transmitted to the graphics processing unit (GPU) for preprocessing operations to improve image quality and improve the efficiency and accuracy of subsequent gesture recognition. After the preprocessing operations, the GPU performs gesture recognition and gesture detection to determine the gesture type. Among them, the GPU is responsible for rendering and drawing the content to be displayed on the display screen.

[0065] For example, preprocessing operations include graying, filtering denoising and image enhancement algorithms. Graying is the process of converting a color image into a grayscale image, which is used to reduce the complexity of the image, reduce the amount of data, and facilitate further processing and analysis. Filtering denoising is an algorithm for reducing noise by smoothing the image, with the aim of retaining useful information in the image while removing irrelevant noise as much as possible. Among them, noise is interference information in the image that does not belong to the original image, caused by sensor errors, transmission problems or environmental interference. For example, commonly used filtering methods include mean filtering, median filtering and Gaussian filtering, etc., which are not limited in the embodiments of the present application. Image enhancement algorithms are used to improve the visual effect of the image or highlight certain features, making the image more suitable for human eye observation or further computer analysis, and generally include histogram equalization, contrast stretching, edge enhancement and gamma correction, etc.

[0066] Through the above scheme, the gesture position of the target gesture can be obtained based on the images collected by the two cameras; based on the gesture position, the relative position with the target component can be determined, that is, the control direction of the control instruction can be determined, so as to realize precise control of the target component based on the control instruction and the control direction, thereby avoiding the driving risks brought by the driver's need to manually adjust the vehicle components during driving, and improving driving safety.

[0067] In one implementation, the method further includes:

[0068] If the image captured by the camera does not include the complete image area of ​​the target gesture, obtain the target distance and the seat position of the target user; based on the target distance and / or seat position, adjust the shooting angle of the camera so that the image captured by the camera includes the complete image area of ​​the target gesture.

[0069] The target distance is used to indicate the distance between the head of the target user and the top of the vehicle's cabin; the seat position of the target user refers to the position of the user's seat in the vehicle or the seat position after longitudinal adjustment.

[0070] In an embodiment of the present application, when the camera fails to completely capture the image of the target gesture, the shooting angle of the camera is intelligently adjusted by obtaining the distance between the user's head and the top of the vehicle cabin and the user's seat position.

[0071] Exemplarily, when the placement and rotation angle of the camera cannot capture a complete image of the user's hand, the height position where the user's hand may be located is determined by the distance between the user's head and the top of the vehicle cabin. The longitudinal position where the user's hand may be located is determined by the user's seat position, and the camera is controlled to rotate according to the mapping relationship between the above position and the preset camera angle rotation amplitude, so that the image captured by the camera includes the image area of ​​the complete target gesture; or, the camera detects the user's hand movements in real time through a hand recognition and tracking algorithm; when it is tracked that the user's hand is about to leave the camera range, the position and angle of the camera are adjusted. For example, use image segmentation, hand detection and hand recognition algorithms to continuously track the hand position in the acquired image or video sequence. The specific algorithm can be implemented based on traditional optical flow methods, Kalman filters or convolutional neural networks combined with time series models.

[0072] Through the above scheme, when the camera fails to capture the complete gesture image area, the camera angle can be adjusted in time to ensure the integrity of the gesture image. Since the integrity of the gesture image is ensured, the gesture type can be accurately identified, thereby improving the accuracy of control.

[0073] In one implementation, the method further includes:

[0074] If a target gesture in the vehicle is detected, determine whether the target gesture is a preset gesture; determine the gesture type and gesture position of the target gesture, including: if the target gesture is a preset gesture, determine the gesture type and gesture position of the target gesture.

[0075] The preset gesture refers to a set of pre-set reference gestures, which usually include a fist gesture, an OK gesture, and other gestures.

[0076] Exemplarily, when a target gesture is detected in a vehicle, a deep learning algorithm is used to determine whether it is a preset gesture. For example, a pre-trained convolutional neural network model is used to extract and classify features of the gesture image to determine whether the target gesture matches the features of the preset gesture. When the features of the target gesture match the features of the preset gesture, the gesture type of the target gesture and its spatial position in the vehicle are further determined.

[0077] In one implementation, whether the target gesture is a preset gesture is determined based on a deep learning algorithm; the deep learning algorithm includes the processes of gesture detection, feature extraction, gesture recognition and classification. First, the hand area is segmented from the image for further gesture recognition. This step usually uses a convolutional neural network model, and a pre-trained deep learning model can be used. For example, deep learning models such as YOLO and SSD, or bounding box detection algorithms; after gesture detection, the features of the hand area are extracted through a convolutional neural network model, or the key points of the hand are extracted through a special network model, such as the OpenPose network model; after extracting the features or key points of the gesture, gesture recognition is performed, and the identified gesture type is compared with the preset gesture set to determine whether it is a preset gesture.

[0078] It should be noted that the training dataset of the convolutional neural network model should contain a large number of gesture samples with different angles, lighting conditions, hand sizes, and skin colors to improve the generalization ability of the model.

[0079] Through the above scheme, recognition and subsequent component control functions can be performed only when the user's gesture is a preset gesture, thereby avoiding the user's mistaken triggering of control gestures during normal driving, resulting in misjudgment of the user's intention and incorrect vehicle component control operations.

[0080] S220: Determine a target component in the vehicle and a target control instruction of the target component based on the gesture type.

[0081] Among them, the target components include air conditioning, sunroof, car windows, seats, etc.; the target control instructions include turning on the air conditioning, adjusting the air conditioning wind speed, opening the sunroof, adjusting the opening and closing degree of the car windows, adjusting the seat tilt angle, etc.

[0082] In one implementation, the method further includes:

[0083] Based on the gesture type and mapping relationship of the target gesture, the target component type and the target control instruction in the vehicle are determined; the target component type is determined as the target component; or, based on the gesture position, the target area in the vehicle is determined, and the target component type in the target area is determined as the target component.

[0084] The mapping relationship is used to represent the mapping relationship between the preset gesture type, the component type of the vehicle and the control instruction; wherein the preset gesture type includes the gesture type of the target gesture.

[0085] Implementation method 1:

[0086] In an embodiment of the present application, by identifying the gesture type of the target gesture, the gesture type and the mapping relationship are matched to determine the component type and the target control instruction in the vehicle; and all determined component types in the vehicle are determined as target components.

[0087] For example, taking the OK gesture corresponding to controlling the turning on of the air conditioner in the vehicle as an example, after determining that the user's gesture type is the OK gesture, it can be determined based on the mapping relationship that the component type corresponding to the gesture is the air conditioner, and the control instruction is to turn on the air conditioner. All the air conditioners in the vehicle are determined as target components, and the air conditioners in the entire vehicle are controlled to be turned on.

[0088] Through the above solution, since the target component and the target control instruction can be determined based on the gesture type of the target gesture, the user can flexibly control the components in the vehicle, thereby improving the user experience.

[0089] Implementation method 2:

[0090] In an embodiment of the present application, by identifying the gesture type of the target gesture, matching the gesture type and the mapping relationship, determining the component type and the target control instruction in the vehicle, determining the target area based on the gesture position, and determining the component type in the target area as the target component.

[0091] For example, taking the example of the number 1 gesture corresponding to controlling the air conditioning wind speed in the vehicle to be set to level 1, after determining that the user's gesture type is the number 1 gesture, it can be determined based on the mapping relationship that the component type corresponding to the gesture is the air conditioning, and the control instruction is to set the air conditioning wind speed to level 1. According to the user's gesture position and the position information of the camera in the vehicle, the area where the user is in the vehicle is determined, and the air conditioning in the area is determined as the target component, and the air conditioning wind speed of the target component is controlled to be set to level 1.

[0092] Through the above solution, it is possible to control the vehicle components in the area where the user is located and save the computing power of the vehicle.

[0093] Table 1

[0094] Preset Gesture Types Part Type Control instructions OK air conditioner Turn on the air conditioner Numbers 1-4 air conditioner Set the air conditioner speed to 1-4 make a fist Skylight Open the skylight forward Seats Reduce seat tilt angle backward Seats Increase seat tilt angle

[0095] Exemplarily, the mapping relationship between preset gesture types, vehicle component types and control instructions is shown in Table 1; the component type corresponding to the OK gesture is air conditioning, and the corresponding control instruction is to turn on the air conditioning; the component type corresponding to the number 1-4 gestures is air conditioning, and the corresponding control instruction is to set the air conditioning wind speed to 1-4 gears; the component type corresponding to the fist gesture is sunroof, and the corresponding control instruction is to open the sunroof; the component type corresponding to the forward gesture is seat, and the corresponding control instruction is to reduce the seat tilt angle; the component type corresponding to the backward gesture is air conditioning, and the corresponding control instruction is to increase the seat tilt angle.

[0096] It should be understood that the above is an example of the mapping relationship between preset gesture types, vehicle component types and control instructions, and this application does not impose any limitation on this.

[0097] S230: Determine a control direction of the control instruction based on the gesture position.

[0098] In one implementation, the method includes:

[0099] Based on the relative position of the gesture position and the target component, the adjustment angle of the target component is determined; based on the adjustment angle, the control direction of the control instruction is determined.

[0100] The relative position refers to the relative spatial position relationship between the user's gesture position and the target component.

[0101] In an embodiment of the present application, the relative position of the spatial position of the user's gesture and the position of the target component is calculated, and the orientation of the gesture position relative to the position of the target component can be determined based on the relative position. Based on the orientation, an adjustment angle for adjusting the operating direction of the target component to toward the gesture position can be determined, and based on the adjustment angle, the control direction is determined.

[0102] For example, if the gesture position is to the left of the position of the target component, the adjustment angle of the target component is adjusted to the left in the horizontal direction, that is, the control instruction is an instruction to control the target component to the left. If the gesture position is above the position of the target component, the adjustment angle of the target component is adjusted upward in the vertical direction, that is, the control instruction is an instruction to control the target component upward. If the gesture position is to the lower right of the position of the target component, the adjustment angle of the target component is adjusted upward in the vertical direction and to the right in the horizontal direction, that is, the control instruction is an instruction to control the target component to the upper right.

[0103] Exemplarily, based on the spatial position of the user's gesture in the vehicle, the spatial position is transmitted to the central controller of the entire vehicle system in the form of a digital signal, and the control direction of the control instruction is determined according to the received gesture position information and sent to the target component control module through the vehicle's controller area network bus (CAN).

[0104] Among them, the central controller is preset with algorithms and logic that can implement the method provided by the embodiments of the present application; the CAN bus is a communication network widely used in modern vehicles, allowing different electronic control units or controllers to exchange information within the vehicle.

[0105] In the above scheme, since the accurate control direction is determined, precise control of the target components can be achieved based on the control instructions and control directions, avoiding driving risks caused by manual adjustment of vehicle components to the required state, thereby improving driving safety.

[0106] S240: Control the target component based on the control instruction and the control direction.

[0107] In one implementation, the method includes:

[0108] Based on the control instruction, the target component is controlled to operate in the control direction.

[0109] For example, taking controlling the air conditioner wind direction as an example, after detecting the target gesture, the air conditioner control module receives the instruction to adjust the air conditioner running direction, and adjusts the blade angle and wind direction of the air conditioner outlet through the driving motor to align it with the target gesture.

[0110] Exemplarily, the vehicle-mounted terminal sends the received control instruction and control direction to the target component, so that the target component changes state based on the control instruction and control direction.

[0111] In one implementation, in order to achieve precise and smooth adjustment, a stepper motor can be used to adjust the angle of the air-conditioning outlet blades; wherein the stepper motor rotates according to the received pulse signal, and each pulse corresponds to a slight adjustment of the blade angle. Based on the required control direction, the number of pulses to be sent is calculated to ensure that the air-conditioning outlet blades can be aligned with the gesture position.

[0112] For example, control parameters such as step size and speed can be set for the stepper motor to achieve smooth adjustment; the step size parameter is used to control the angle adjustment amplitude, usually set to a smaller value to ensure the fineness of the adjustment; the speed parameter is used to control the speed of adjustment to ensure the speed of the response adjustment process.

[0113] In the above scheme, after determining the target component, control instruction and control direction, the target component is controlled to run in the control direction based on the content of the control instruction, which can realize automatic control of vehicle components, avoid the user's manual adjustment steps, and improve the accuracy of control.

[0114] In the above embodiment, when a user's target gesture is detected in the vehicle, the gesture type and the location of the gesture can be determined through the acquired target gesture; based on the gesture type, the target component in the vehicle and the control instruction of the target component can be determined; based on the gesture position, the control direction of the control instruction can be determined; because the control direction is determined based on the gesture position when the user's target gesture is acquired, the user does not need to manually adjust the target component when controlling it, and the target state to which the user needs to adjust the target component can be determined through the type and position of the target gesture, thereby realizing automatic and precise adjustment of the components in the vehicle, avoiding the driving risks brought about by the driver's need to manually adjust the vehicle components during driving, and improving driving safety.

[0115] Combine the following Figure 3 Another vehicle control method provided in an embodiment of the present application is illustrated by way of example.

[0116] Figure 3 FIG. 1 is a schematic flow chart of a specific application of a vehicle control method provided in an embodiment of the present application. Figure 3 As shown, the method 300 includes S310 to S370, and S310 to S370 are described in detail below.

[0117] S310: Acquire a first image captured by a first camera and a second image captured by a second camera.

[0118] Among them, the first image and the second image are images captured by different cameras in the vehicle; among them, the camera can be the vehicle's OMS camera, binocular camera or wide-angle camera, etc. The number of cameras is two or more, and images from multiple perspectives can be captured.

[0119] It should be noted that the cameras need to be set at different positions to be able to capture the user's hand image in the vehicle. For example, the first camera is set above the dashboard facing the driver area; for example, tilted downward at an angle of 30 degrees, and the second camera is set on the top of the center console, offset 45 degrees towards the driver's seat, so that the driver's hand image can be captured.

[0120] Optionally, the camera should have the characteristics of high resolution, high frame rate and low noise; for example, a camera with a resolution of 1920×1080 pixels and a frame rate of 60 frames per second is selected to ensure the clarity and real-time nature of the image.

[0121] For example, an image captured by a camera located above the dashboard and the top of the center console in the vehicle is obtained, and the image includes the complete hand image area of ​​the driver. If the captured image does not include the complete hand image area of ​​the driver, the shooting angles of the first camera and the second camera can be adjusted so that images of multiple perspectives of the complete hand image area can be captured.

[0122] Through the above scheme, when the camera fails to capture the complete gesture image area, the camera angle can be adjusted in time to ensure the integrity of the gesture image. Since the integrity of the gesture image is ensured, the gesture type can be accurately identified, thereby improving the accuracy of control.

[0123] S320: Determine a gesture type of the target gesture based on the first image and / or the second image.

[0124] Exemplarily, based on a first image captured by a first camera and / or a second image captured by a second camera in the vehicle, hand detection is performed on the first image and / or the second image to obtain a target image area in the first image and / or the second image, gesture recognition is performed on the target image area, and the user's gesture type is determined; based on parameters such as the first image, the second image, and the intrinsic and extrinsic parameters of the camera, the gesture position is determined according to a stereoscopic vision algorithm and a trigonometric function relationship.

[0125] Optionally, the above implementation can refer to Figure 2 The relevant description of S210 is not repeated here.

[0126] S330: Determine whether the gesture type of the target gesture is a preset gesture. If yes, execute S340 to S360; if no, execute S310.

[0127] The preset gesture refers to a set of pre-set reference gestures, which usually include a fist gesture, an OK gesture, etc.

[0128] Exemplarily, when a target gesture is detected in the vehicle, gesture recognition is used to determine whether the target gesture belongs to a preset gesture. If the target gesture is a preset gesture, the gesture type of the target gesture and its spatial position in the vehicle are determined.

[0129] In the above scheme, when the user's gesture is a preset gesture, the subsequent component control function is implemented, which avoids the user's mistaken triggering of the control gesture during normal driving, resulting in misjudgment of the user's intention and incorrect vehicle component control operation.

[0130] When the target gesture is of a preset gesture type, the control instruction of the target component and the control direction of the target component can be determined according to the target gesture; among which, the implementation method of determining the control instruction of the target component refers to S360; the implementation method of determining the control direction of the target component refers to S340 and S350.

[0131] S340: Determine a gesture position based on the first image, the second image, and target parameters of the camera.

[0132] Among them, the target parameters of the camera refer to internal parameters and external parameters. The internal parameters refer to parameters such as focal length and optical center position, and the external parameters refer to parameters such as the relative position, rotation angle, and relative posture between cameras. These parameters can be obtained through calibration when the camera is installed, or can be obtained based on the current position of the camera and the adjustment angle.

[0133] Exemplarily, based on a first image captured by a first camera, a second image captured by a second camera, and parameters such as intrinsic parameters and extrinsic parameters of the cameras, the gesture position is determined according to a stereoscopic vision algorithm and a trigonometric function relationship.

[0134] Among them, the stereoscopic vision algorithm refers to using different cameras to shoot the same scene from different perspectives, and using the parallax information between the captured images to obtain three-dimensional information. For example, after obtaining images collected by different cameras, the corresponding points of the same object in different images are obtained using image block matching or feature matching methods, and the parallax information is obtained based on the coordinate difference of the corresponding points in different images. The three-dimensional information is obtained based on the calculated parallax information, the focal length of different cameras, and the distance between different cameras, and finally the three-dimensional structure of the scene is reconstructed based on the three-dimensional information.

[0135] Exemplarily, after reconstructing the three-dimensional structure of the scene in the area where the user's hand is located based on the three-dimensional information, the three-dimensional coordinates of the user's hand in the scene are determined to obtain the gesture position.

[0136] S350: Determine a control direction of the control instruction based on the gesture position.

[0137] Exemplarily, the vehicle-mounted terminal determines the control direction of the control instruction based on the spatial position of the user's gesture in the vehicle and the spatial position of the target component to be controlled in the vehicle.

[0138] Optionally, the implementation of S350 can refer to Figure 2 The relevant description of the implementation method in S230 will not be repeated here.

[0139] S360: Determine a target component and a control instruction of the target component based on the gesture type of the target gesture.

[0140] Among them, the target components include air conditioners, sunroofs, windows, seats, etc.; the target control instructions include turning on the air conditioner, adjusting the air conditioner wind speed, opening the sunroof, etc.

[0141] Exemplarily, based on determining the gesture type corresponding to the user's gesture, matching is performed according to the gesture type and the mapping relationship to determine the target component that the user intends to control and the target control instruction.

[0142] Optionally, the implementation of S360 can refer to Figure 2 The relevant description of the implementation method in S220 will not be repeated here.

[0143] S370: Based on the control instruction, control the target component to run in the control direction.

[0144] Exemplarily, the vehicle-mounted terminal sends the received control instruction and control direction to the target component, so that the target component changes state based on the control instruction and control direction.

[0145] Optionally, the implementation of S370 can refer to Figure 2 The relevant description of the implementation method in S240 will not be repeated here.

[0146] In an embodiment of the present application, when a target gesture of a user in a vehicle is detected, the gesture type and gesture position can be determined by identifying the gesture. Based on the gesture type, the target component in the vehicle and its corresponding control instruction can be determined; based on the gesture position, the execution direction of the control instruction can be determined. Therefore, the control instruction and control direction can be accurately determined by gestures, thereby achieving automatic and precise control of the target component of the vehicle. This avoids the need for drivers and passengers to manually adjust the operation of vehicle components, thereby improving the user experience.

[0147] Combine the following Figures 4 to 7 A control method for controlling the air direction of an air conditioner by using an OK gesture provided in an embodiment of the present application is illustrated by way of example.

[0148] Figure 4 FIG. 1 is a schematic flow chart of another vehicle control method provided in an embodiment of the present application. Figure 4 As shown, the method 400 includes S410 to S460, and S410 to S460 are described in detail below.

[0149] S410: Acquire a first image captured by a first camera and a second image captured by a second camera.

[0150] For example, an image captured by a camera located above the dashboard and the top of the center console in the vehicle is obtained, and the image includes the complete hand image area of ​​the driver. If the captured image does not include the complete hand image area of ​​the driver, the shooting angles of the first camera and the second camera can be adjusted so that images of multiple perspectives of the complete hand image area can be captured.

[0151] S420: Determine a gesture type of the target gesture based on the first image and / or the second image.

[0152] Exemplarily, based on a first image captured by a first camera and / or a second image captured by a second camera in the vehicle, hand detection is performed on the first image and / or the second image to obtain a target image area in the first image and / or the second image, gesture recognition is performed on the target image area, and the user's gesture type is determined to be an OK gesture; based on parameters such as the first image, the second image, and the intrinsic and extrinsic parameters of the camera, the gesture position is determined according to a stereoscopic vision algorithm and a trigonometric function relationship.

[0153] S430: Determine whether the gesture type of the target gesture is a preset gesture. If yes, execute S440; if no, execute S410.

[0154] Exemplarily, when a target gesture is detected in the vehicle, gesture recognition is used to determine whether the target gesture belongs to a preset gesture. When the target gesture is determined to be a preset gesture, the gesture type of the target gesture and its spatial position in the vehicle are further determined.

[0155] Optionally, the implementation of S410 to S430 can refer to Figure 3 The relevant descriptions of S310 to S330 are not repeated here.

[0156] S440: Determine that the target component is an air conditioner, and determine a control instruction for the air conditioner.

[0157] Exemplarily, based on the OK gesture and the mapping relationship between the preset gesture type, the component type of the vehicle and the control instruction, it is determined that the target component type is an air conditioner, and the control instruction is determined to turn on the air conditioner.

[0158] Implementation method 1:

[0159] In an embodiment of the present application, by identifying the gesture type of the target gesture, matching the gesture type and the mapping relationship, determining that the component type in the vehicle is an air conditioner and a target control instruction; and determining the air conditioner in the vehicle as the target component.

[0160] For example, Figure 5As shown in (a), the gesture 510 is recognized as an OK gesture. After matching according to the gesture type and the mapping relationship, the component type is determined to be an air conditioner; wherein, the air conditioners in the vehicle cabin include air conditioners 130 and 150; the target air conditioners are determined to be air conditioners 130 and 150. In addition, the control instruction obtained according to the OK gesture is a turn-on instruction, and the air conditioners 130 and 150 are controlled by the gesture 510 to blow air in the direction where the gesture 510 is located; that is, the air conditioner 130 is controlled to blow air in the lower left direction and the air conditioner 150 is controlled to blow air in the lower right direction.

[0161] For example, Figure 5 As shown in (b), the gesture 510 is recognized as an OK gesture. After matching according to the gesture type and the mapping relationship, the component type is determined to be an air conditioner; wherein, the air conditioner in the vehicle cabin includes the air conditioner 130 and the air conditioner 150; the target air conditioner is determined to be the air conditioner 130 and the air conditioner 150. In addition, the control instruction obtained according to the OK gesture is a turn-on instruction, and the air conditioner 130 and the air conditioner 150 are controlled by the gesture 510 to blow air in the direction where the gesture 510 is located; that is, the air conditioner 130 is controlled to blow air in the upper left direction and the air conditioner 150 is controlled to blow air in the upper right direction.

[0162] In the above-mentioned implementation method one, by identifying the gesture type of the target gesture, the gesture type and the mapping relationship are matched, and it is determined that the component type in the vehicle is air conditioning, and the target control instruction is to turn on the air conditioning; all air conditioners in the vehicle are determined as target components; because the target component and the target control instruction can be determined based on the gesture type of the target gesture, the user can flexibly control the air conditioner, thereby improving the user experience.

[0163] Implementation method 2:

[0164] In an embodiment of the present application, by identifying the gesture type of the target gesture, matching the gesture type and the mapping relationship, it is determined that the component type in the vehicle is the air conditioner and the target control instruction; the target area is determined based on the gesture position, and the air conditioner in the target area is determined as the target component.

[0165] For example, Figure 6 As shown in (a), the gesture 510 is recognized as an OK gesture. After matching according to the gesture type and the mapping relationship, the component type is determined to be an air conditioner. The air conditioners in the vehicle cabin include air conditioners 130 and 150. The target air conditioner is determined according to the target area where the gesture 510 is currently located, that is, the distance between the gesture 510 and the air conditioner 130, and the distance between the gesture 510 and the air conditioner 150. Since the distance between the gesture 510 and the air conditioner 130 is less than the distance between the gesture 510 and the air conditioner 150. Therefore, Figure 6As shown in (a) in FIG. 1 , the target air conditioner is determined to be the air conditioner 130. In addition, the control instruction obtained according to the OK gesture is the turn-on instruction, and the air conditioner 130 is controlled to blow air in the direction of the gesture 510 through the gesture 510; that is, the air conditioner 130 is controlled to blow air in the lower left direction.

[0166] For example, Figure 6 As shown in (b), the gesture 510 is recognized as an OK gesture. After matching according to the gesture type and the mapping relationship, the component type is determined to be an air conditioner. The air conditioners in the vehicle cabin include air conditioners 130 and 150. The target air conditioner is determined according to the target area where the gesture 510 is currently located, that is, the distance between the gesture 510 and the air conditioner 130, and the distance between the gesture 510 and the air conditioner 150. Therefore, as Figure 6 As shown in (a) in FIG. 1 , the target air conditioner is determined to be the air conditioner 130. In addition, the control instruction obtained according to the OK gesture is the turn-on instruction, and the air conditioner 130 is controlled to blow air in the direction of the gesture 510 through the gesture 510; that is, the air conditioner 130 is controlled to blow air in the upper left direction.

[0167] For example, in the above implementations 1 and 2, the air outlet speed of the air conditioner is a preset air outlet speed, and the air outlet speed changes when the gesture position changes. Optionally, in another implementation, the air outlet speed can be determined according to the gesture type of the target gesture; it can be understood that different gesture types at the same gesture position correspond to different air outlet speeds.

[0168] For example, Figure 7 As shown in (a), the gesture 520 is recognized as a digital 1 gesture, and after matching according to the gesture type and the mapping relationship, the component type is determined to be an air conditioner; wherein the air conditioner in the vehicle cabin includes the air conditioner 130; and the target air conditioner is determined to be the air conditioner 130. In addition, the control instruction obtained according to the OK gesture is to set the air conditioner wind speed to level 1, and the wind speed of the air conditioner 130 is controlled to be set to level 1 through the gesture 520.

[0169] Similarly, if Figure 7 As shown in (b) in FIG. 5 , the control instruction corresponding to gesture 530 is to set the air-conditioning wind speed to level 2; Figure 7 As shown in (c) in FIG. 5 , the control instruction corresponding to gesture 540 is to set the air-conditioning wind speed to level 3; Figure 7 As shown in (d) in the figure, the control instruction corresponding to gesture 550 is to set the air-conditioning wind speed to level 4; the specific control method can be found in the above embodiment and will not be described again here.

[0170] It should be understood that Figure 7In the embodiment of the present application, different gesture types at the same gesture position are used as examples for illustration; in the embodiment of the present application, the gesture position is not limited. For different gesture positions, the air outlet direction of the air outlet is controlled differently.

[0171] In one implementation, the air conditioner in the vehicle is controlled by two gestures. After detecting that the first target gesture is an OK gesture, the air conditioner is controlled to turn on. Then, when the second target gesture is detected as a number 1 gesture, the air conditioner wind speed is controlled to be set to level 1. The air conditioner is controlled to turn on and blow air at level 1.

[0172] Optionally, when it is detected that the OK gesture and the number 1 gesture are at the same position, the air conditioner is controlled to turn on and blow air in the direction of the position at a wind speed of level 1.

[0173] Optionally, when it is detected that the OK gesture and the number 1 gesture are at different positions, the air conditioner is controlled to turn on and blow air at a wind speed of level 1 in the direction of the number 1 gesture position.

[0174] In the above-mentioned second implementation method, after determining that the component type in the vehicle is an air conditioner and the target control instruction is to turn on the air conditioner, the target area is determined based on the gesture position, and the air conditioner in the target area is determined as the target component, which can realize the control of the air conditioner in the area where the user is located and save the computing power of the vehicle.

[0175] It should be understood that the above embodiments can be arbitrarily combined to form optional embodiments of the present application, which will not be described in detail here.

[0176] S450: Determine a gesture position based on the first image, the second image, and the target parameters of the camera, and determine a control direction of an air outlet of the air conditioner based on a relative position of the gesture position and the air conditioner.

[0177] For example, Figure 5 As shown, the parameter configuration of camera 110 and camera 120, such as internal parameters and external parameters, is read. Based on the in-car images captured by camera 110 and camera 120, the corresponding pixel points in the two images are found through a matching algorithm. These corresponding points are usually those feature points that are clearly visible and close in position in both images, such as gestures, seats, etc. After finding the corresponding pixel points, the depth calculation is performed using the internal parameters and external parameters. Finally, the depth value of each matching point, that is, the distance from the point to the camera, can be calculated through a triangulation algorithm or the principle of similar triangles. These depth values ​​are used to describe the three-dimensional structure of the in-car scene. After the structure is constructed, the spatial position coordinates of the user's gesture 510 are determined. After the in-vehicle terminal calculates the gesture position, based on the relative position of the gesture position and the air conditioner 130, the control direction of the air outlet of the air conditioner 130 is determined according to the relative position.

[0178] Among them, the depth calculation is based on the disparity algorithm, that is, the three-dimensional coordinates of the point in the scene are estimated by using the baseline distance between the two cameras and the horizontal displacement of the corresponding pixel points in the two images, where the baseline distance refers to the distance between the two cameras; the matching algorithm can be based on the similarity of the feature description, such as finding the best matching pair by calculating the Euclidean distance or cosine similarity between them.

[0179] In the above scheme, the gesture position of the target gesture can be obtained through the images collected by the two cameras; the relative position with the air conditioner can be determined through the gesture position, that is, the control direction of the control instruction can be determined, so as to realize precise control of the target component based on the control instruction and the control direction, thereby avoiding the driving risks brought by the driver's need to manually adjust the air conditioner during driving and improving driving safety.

[0180] S460, control the air conditioner to run in the control direction.

[0181] Exemplarily, the air conditioner 130 is controlled to operate in the direction of the gesture 510 .

[0182] Optionally, the implementation of S460 can refer to Figure 2 The relevant description of the implementation method in S240 will not be repeated here.

[0183] In an embodiment of the present application, after the user's target gesture is detected, the gesture type and gesture position of the target gesture are determined, and based on the mapping relationship between the gesture type and the vehicle component type and the control instruction, it is determined that the vehicle component to be controlled is the air conditioner, and the control direction of the air conditioner is determined based on the gesture position, so that the air conditioner runs toward the user's gesture position, which is convenient for the user to automatically adjust the air conditioner and avoids the user's manual adjustment of the air conditioner multiple times, thereby distracting the driver's attention and improving driving safety.

[0184] Combination of the above Figures 1 to 7 A vehicle control method provided by an embodiment of the present application is described in detail; Figure 8 and Fig. 9 The device embodiments of the present application are described in detail. It should be understood that the device in the embodiments of the present application can execute the various methods of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0185] Figure 8 800 is a schematic diagram of a vehicle control device provided in an embodiment of the present application, wherein the device 800 includes a detection module 810 and a processing module 820 .

[0186] Among them, the detection module 810 is used to detect the target gesture in the vehicle; the processing module 820 is used to determine the gesture type and gesture position of the target gesture if the target gesture in the vehicle is detected; wherein the gesture position is used to indicate the spatial position of the target gesture in the vehicle; based on the gesture type, the target component in the vehicle and the target control instruction of the target component are determined; based on the gesture position, the control direction of the control instruction is determined; based on the control instruction and the control direction, the target component is controlled.

[0187] Optionally, as an embodiment, the processing module 820 is specifically configured to:

[0188] Based on the relative position of the gesture position and the target component, the adjustment angle of the target component is determined; based on the adjustment angle, the control direction of the control instruction is determined.

[0189] Optionally, as an embodiment, the processing module 820 is further configured to:

[0190] Based on the gesture type and mapping relationship of the target gesture, the target component type and the target control instruction in the vehicle are determined; wherein the mapping relationship is used to represent the mapping relationship between the preset gesture type, the component type of the vehicle and the control instruction; the target component type is determined as the target component; or, based on the gesture position, the target area in the vehicle is determined, and the target component type in the target area is determined as the target component.

[0191] Optionally, as an embodiment, the processing module 820 is specifically configured to:

[0192] Acquire a first image and a second image; wherein the first image and the second image are images captured by different cameras in the vehicle; determine the gesture type based on the first image and / or the second image; determine the gesture position based on the first image, the second image and the target parameters of the camera; wherein the target parameters include the intrinsic parameters and extrinsic parameters of the camera.

[0193] Optionally, as an embodiment, the processing module 820 is further configured to:

[0194] If the image captured by the camera does not include the complete image area of ​​the target gesture, obtain the target distance and the seat position of the target user; wherein the target distance is used to represent the distance between the head of the target user and the top of the vehicle's cabin; based on the target distance and / or the seat position, adjust the shooting angle of the camera so that the image captured by the camera includes the complete image area of ​​the target gesture.

[0195] Optionally, as an embodiment, the processing module 820 is specifically configured to:

[0196] If a target gesture in the vehicle is detected, determine whether the target gesture is a preset gesture; determine the gesture type and gesture position of the target gesture, including: if the target gesture is a preset gesture, determine the gesture type and gesture position of the target gesture.

[0197] Optionally, as an embodiment, the processing module 820 is specifically configured to:

[0198] Based on the control instruction, the target component is controlled to operate in the control direction.

[0199] It should be noted that the vehicle control device 900 is implemented in the form of a functional unit. The term "module" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0200] For example, a "module" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combined logic circuit, and / or other suitable components that support the described functions.

[0201] Therefore, the units of each example described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0202] Fig. 9 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application.

[0203] Exemplarily, vehicle 900 includes: a processor 910 , a memory 920 , and an executable program code 930 .

[0204] Exemplarily, the vehicle 900 includes one or more processors 910, which can support the vehicle 900 to implement the vehicle generation method in the method embodiment. The processor 910 can be a general-purpose processor or a special-purpose processor. For example, the processor 910 can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit, a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0205] Exemplarily, the processor 910 may be used to control the vehicle 900, execute software programs, and process data of the software programs. The vehicle 900 may also include a communication unit to implement input (reception) and output (transmission) of signals.

[0206] Exemplarily, the vehicle 900 may include one or more memories 920 on which executable program code 930 is stored. The executable program code 930 can be executed by the processor 910 to generate instructions, so that the processor 910 executes the generation method described in the above method embodiment according to the instructions.

[0207] Optionally, data may be stored in the memory 920. Optionally, the processor 910 may read data stored in the memory 920, which may be stored at the same storage address as the executable program code 930, or may be stored at a different storage address from the executable program code 930.

[0208] Exemplarily, the processor 910 and the memory 920 may be provided separately or integrated together, for example, integrated on a system on chip (System On Chip, SOC) of the terminal device.

[0209] Exemplarily, the memory 920 can be used to store relevant programs of the vehicle generation method provided in the embodiment of the present application, and the processor 920 can be used to call the executable program code 930 stored in the memory 920 when controlling the vehicle to execute the vehicle control method of the embodiment of the present application.

[0210] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the vehicle control method of any of the aforementioned embodiments.

[0211] Among them, computer-readable storage media may include but are not limited to any type of disk, including floppy disks, optical disks, digital versatile disks (Digital Video Disc, DVD), compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), microdrives and magneto-optical disks, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable programmable read only memory (Electrically Erasable Programmable read only memory, EEPROM), dynamic random access memory (Dynamic Random Access Memory, DRAM), video random access memory (Video Random Access Memory, VRAM), flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0212] The present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement a vehicle control method in the above-mentioned embodiment.

[0213] In addition, the electronic device provided in the embodiments of the present application may specifically be a chip, a component or a module, and the electronic device may include a connected processor and a memory; wherein the memory is used to store instructions, and when the electronic device is running, the processor may call and execute instructions so that the chip executes a vehicle control method in the above-mentioned embodiments.

[0214] Among them, the vehicle, computer-readable storage medium, computer program product or chip provided in this application are all used to execute the corresponding vehicle control method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding vehicle control method provided above, and will not be repeated here.

[0215] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0216] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0217] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A vehicle control method, characterized in that: The method comprises: If a target gesture in the vehicle is detected, determining a gesture type and a gesture position of the target gesture; wherein the gesture position is used to indicate a spatial position of the target gesture in the vehicle; Determining a target component in the vehicle and a target control instruction for the target component based on the gesture type; Based on the gesture position, determining a control direction of the control instruction; Based on the control instruction and the control direction, the target component is controlled.

2. The method according to claim 1, characterized in that The determining the control direction of the control instruction based on the gesture position includes: Determining an adjustment angle of the target component based on a relative position between the gesture position and the target component; Based on the adjustment angle, a control direction of the control instruction is determined.

3. The method according to claim 1, characterized in that The determining, based on the gesture type, a target component in the vehicle and a target control instruction for the target component includes: Based on the gesture type and mapping relationship of the target gesture, determine the target component type in the vehicle and the target control instruction; wherein the mapping relationship is used to represent the mapping relationship between the preset gesture type, the component type of the vehicle and the control instruction; The target component type is determined as the target component; or a target area in the vehicle is determined based on the gesture position, and the target component type in the target area is determined as the target component.

4. The method according to claim 1, characterized in that: If a target gesture in the vehicle is detected, determining the gesture type and gesture position of the target gesture includes: Acquire a first image and a second image; wherein the first image and the second image are images captured by different cameras in the vehicle; Determining the gesture type based on the first image and / or the second image; The gesture position is determined based on the first image, the second image, and target parameters of the camera; wherein the target parameters include intrinsic parameters and extrinsic parameters of the camera.

5. The method according to claim 4, characterized in that Also includes: If the image captured by the camera does not include a complete image area of ​​the target gesture, obtaining a target distance and a seat position of the target user; wherein the target distance is used to represent the distance between the head of the target user and the top of the cabin of the vehicle; Based on the target distance and / or the seat position, the shooting angle of the camera is adjusted so that the image captured by the camera includes a complete image area of ​​the target gesture.

6. The method according to any one of claims 1 to 5, characterized in that include: If a target gesture in the vehicle is detected, determining whether the target gesture is a preset gesture; The determining the gesture type and gesture position of the target gesture includes: If the target gesture is the preset gesture, the gesture type and the gesture position of the target gesture are determined.

7. The method according to any one of claims 1 to 5, characterized in that The controlling the target component based on the control instruction and the control direction includes: Based on the control instruction, the target component is controlled to operate in the control direction.

8. A vehicle control device, characterized in that: The device comprises: A detection module, used to detect a target gesture in a vehicle; A processing module, for determining a gesture type and a gesture position of the target gesture if a target gesture in the vehicle is detected; wherein the gesture position is used to indicate a spatial position of the target gesture in the vehicle; based on the gesture type, determining a target component in the vehicle and a target control instruction for the target component; based on the gesture position, determining a control direction of the control instruction; and controlling the target component based on the control instruction and the control direction.

9. A vehicle, characterized in that: The vehicle comprises: A memory for storing executable program codes; A processor is used to call and run the executable program code from the memory so that the vehicle executes the method as claimed in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a vehicle, the vehicle is caused to perform the method according to any one of claims 1 to 7.