Interactive control methods, devices, vehicles, media, and program products

Abstract

This application relates to the field of intelligent vehicle connectivity and interaction technology, and particularly to an interactive control method, device, vehicle, medium, and program product. The method includes: collecting vehicle driving status data, user gesture data, and human posture information; generating initial gesture commands for the vehicle based on the driving status data, gesture data, and human posture information; obtaining the verification result of the initial gesture commands; and, if the verification result meets preset verification conditions, determining the vehicle's gesture commands based on the initial gesture commands to control the vehicle. This solves the problems of relying solely on a single trajectory or simple gesture features for recognition, failing to consider children's physiological characteristics and behavioral patterns, lacking multi-dimensional differentiation criteria, hierarchical verification, and scene adaptation capabilities, failing to effectively identify children's accidental touch behavior, easily leading to misoperation, affecting normal device use, threatening driving safety, and struggling to balance anti-accidental touch effectiveness with adult ease of use.

Description

Technical Field

[0001] This application relates to the field of intelligent vehicle connectivity and interaction technology, and in particular to an interactive control method, device, vehicle, medium, and program product. Background Technology

[0002] In related technologies, continuous motion trajectory data such as hand movement path and displacement direction can be used to define the range of standard gesture movement routes. Gesture judgment and recognition can be completed by comparing whether the actual hand movement trajectory matches the preset trajectory. Alternatively, shallow static appearance features such as hand contour, finger posture, and hand orientation can be extracted to establish a basic feature comparison model. Gesture recognition can be achieved by relying on the degree of matching between measured features and preset simple features.

[0003] However, the relevant technologies rely solely on a single trajectory or simple gesture features for recognition, without considering children's physiological characteristics and behavioral patterns. They lack multi-dimensional differentiation criteria, graded verification, and scene adaptation capabilities, making it difficult to effectively identify children's accidental touch behavior. This can easily lead to misoperation, affecting the normal use of the device, threatening driving safety, and making it difficult to balance the anti-accidental touch effect with the convenience of adult interaction. Improvements are urgently needed. Summary of the Invention

[0004] This application provides an interactive control method, device, vehicle, medium, and program product to solve the problems in related technologies that rely solely on a single trajectory or simple gesture features for recognition, without combining children's physiological characteristics and behavioral patterns, lacking multi-dimensional differentiation criteria, graded verification, and scene adaptation capabilities, and are unable to effectively identify children's accidental touch behavior, which easily leads to misoperation, affecting the normal use of the device, threatening driving safety, and making it difficult to balance the effect of preventing accidental touch with the convenience of adult interactive use.

[0005] The first aspect of this application provides a vehicle interaction control method, comprising the following steps: collecting vehicle driving status data, and collecting gesture data and human posture information of a corresponding user inside the vehicle; generating an initial gesture command for the vehicle based on the driving status data, the gesture data, and the human posture information; obtaining a verification result of the initial gesture command, and, if the verification result meets a preset verification condition, determining a gesture command for the vehicle based on the initial gesture command, and controlling the vehicle according to the gesture command.

[0006] Optionally, in one embodiment of this application, generating the initial gesture command for the vehicle based on the driving state data, the gesture data, and the human posture information includes: extracting gesture features from the gesture data and extracting posture features from the human posture information; determining the user's identity recognition result based on the gesture features and the posture features; determining the user's gesture recognition result based on the driving state data and the gesture features; and generating the initial gesture command based on the identity recognition result and the gesture recognition result.

[0007] Optionally, in one embodiment of this application, determining the user's identity recognition result based on the gesture features and the posture features includes: obtaining target gesture features and target posture features corresponding to a target identity in a preset identity database; matching the gesture features with the target gesture features and matching the posture features with the target posture features, and determining the user's actual identity based on the matching results; and determining the actual identity as the identity recognition result if the matching results meet preset matching conditions.

[0008] Optionally, in one embodiment of this application, determining the user's gesture recognition result based on the driving state data and the gesture features includes: determining whether the gesture corresponding to the gesture features is an interfering action based on the gesture features; if the gesture is not an interfering action, extracting the vehicle speed features from the driving state data and dynamically determining a gesture recognition threshold based on the vehicle speed features; calculating the matching degree between the gesture features and the desired gesture features; if the matching degree is greater than the gesture recognition threshold, determining the gesture as a valid gesture and outputting the gesture recognition result based on the valid gesture.

[0009] Optionally, in one embodiment of this application, generating the initial gesture instruction based on the identity recognition result and the gesture recognition result includes: generating the initial gesture instruction based on the identity recognition result and the gesture recognition result when the identity recognition result meets preset authentication conditions and the gesture recognition result meets preset gesture verification conditions.

[0010] Optionally, in one embodiment of this application, the method further includes: if the verification result does not meet the preset verification conditions, controlling the vehicle to enter an interaction restriction mode.

[0011] A second aspect of this application provides a vehicle interaction control device, comprising: a data acquisition module for acquiring vehicle driving status data and acquiring gesture data and human posture information of a corresponding user inside the vehicle; a generation module for generating an initial gesture command for the vehicle based on the driving status data, the gesture data, and the human posture information; and a first control module for obtaining a verification result of the initial gesture command, and, if the verification result meets preset verification conditions, determining a gesture command for the vehicle based on the initial gesture command, and controlling the vehicle according to the gesture command.

[0012] Optionally, in one embodiment of this application, the generation module includes: an extraction unit, configured to extract gesture features from the gesture data and extract posture features from the human posture information; a first determination unit, configured to determine the user's identity recognition result based on the gesture features and the posture features; a second determination unit, configured to determine the user's gesture recognition result based on the driving state data and the gesture features; and a generation unit, configured to generate the initial gesture command based on the identity recognition result and the gesture recognition result.

[0013] Optionally, in one embodiment of this application, the first determining unit includes: an acquisition subunit, configured to acquire target gesture features and target posture features corresponding to a target identity in a preset identity database; a matching subunit, configured to match the gesture features with the target gesture features and match the posture features with the target posture features, and determine the user's actual identity based on the matching result; and a first determining subunit, configured to determine the actual identity as the identity recognition result if the matching result satisfies a preset matching condition.

[0014] Optionally, in one embodiment of this application, the second determining unit includes: a second determining subunit, configured to determine whether the gesture corresponding to the gesture feature is an interfering action based on the gesture feature; a third determining subunit, configured to extract the vehicle speed feature from the driving state data and dynamically determine the gesture recognition threshold based on the vehicle speed feature when the gesture is not an interfering action; a calculation subunit, configured to calculate the matching degree between the gesture feature and the desired gesture feature; and a fourth determining subunit, configured to determine the gesture as a valid gesture when the matching degree is greater than the gesture recognition threshold and output the gesture recognition result based on the valid gesture.

[0015] Optionally, in one embodiment of this application, the generation unit includes: a generation subunit, configured to generate the initial gesture instruction based on the identity recognition result and the gesture recognition result when the identity recognition result meets the preset authentication conditions and the gesture recognition result meets the preset gesture verification conditions.

[0016] Optionally, in one embodiment of this application, it further includes: a second control module, configured to control the vehicle to enter an interactive restriction mode if the verification result does not meet the preset verification conditions.

[0017] A third aspect of this application provides a vehicle, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle interaction control method as described in the above embodiments.

[0018] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle interaction control method described above.

[0019] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, implements the control method for vehicle interaction as described above.

[0020] This application embodiment can collect vehicle driving status data, user gesture data, and human posture information respectively. Then, based on the driving status data, gesture data, and human posture information, it obtains corresponding initial gesture commands. If the verification result of the initial gesture command meets preset verification conditions, the initial gesture command is used as the vehicle's gesture command to complete vehicle control. By collaboratively generating the initial gesture command based on multiple types of information such as driving status, gesture data, and human posture, and then verifying and filtering it to determine the final control command, the accuracy of gesture command recognition can be improved, invalid accidental touch operations can be effectively identified, and the safety of use in driving scenarios and the reliability of in-vehicle gesture interaction control can be considered. This solves the problems in related technologies that rely solely on a single trajectory or simple gesture features for recognition, without considering children's physiological characteristics and behavioral patterns, lacking multi-dimensional differentiation criteria, graded verification, and scenario adaptation capabilities, failing to effectively identify children's accidental touch behavior, easily leading to misoperation, affecting normal device use, threatening driving safety, and struggling to balance anti-accidental touch effectiveness with the convenience of adult interaction.

[0021] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0022] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a vehicle interaction control method provided according to an embodiment of this application; Figure 2 This is a flowchart illustrating data acquisition and transmission according to an embodiment of this application; Figure 3 This is a flowchart illustrating gesture and identity recognition according to one embodiment of this application; Figure 4 This is a flowchart illustrating gesture recognition threshold updating according to an embodiment of this application; Figure 5 This is a flowchart illustrating the secondary verification of an initial gesture command according to an embodiment of this application; Figure 6 A flowchart illustrating an unlocking interaction restriction mode according to an embodiment of this application; Figure 7 This is a block diagram of a vehicle interaction control device provided according to an embodiment of this application; Figure 8 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application. Detailed Implementation

[0023] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0024] The following description, with reference to the accompanying drawings, describes the interactive control method, device, vehicle, medium, and program product of embodiments of this application. Addressing the issues mentioned in the background art, which rely solely on single trajectory or simple gesture features for recognition, failing to consider children's physiological characteristics and behavioral patterns, lacking multi-dimensional differentiation criteria, graded verification, and scene adaptation capabilities, and thus unable to effectively identify children's accidental touches, easily leading to misoperations, affecting normal device use, threatening driving safety, and struggling to balance anti-accidental touch effectiveness with adult ease of use, this application provides a vehicle interaction control method. In this method, vehicle driving status data, user gesture data, and human posture information are collected separately. Then, based on the driving status data, gesture data, and human posture information, corresponding initial gesture commands are obtained. If the verification result of the initial gesture command meets preset verification conditions, the initial gesture command is used as the vehicle's gesture command to complete vehicle control. By collaboratively generating initial gesture commands based on multiple types of information—driving status, gesture data, and human posture—and then verifying and selecting the final control command, the accuracy of gesture command recognition can be improved, invalid accidental touch operations can be effectively identified, and the safety of use in driving scenarios and the reliability of in-vehicle gesture interaction control can be balanced. This solves the problems in related technologies that rely solely on a single trajectory or simple gesture features for recognition, without considering children's physiological characteristics and behavioral patterns, lacking multi-dimensional differentiation criteria, graded verification and scene adaptation capabilities, and are unable to effectively identify children's accidental touch behavior, which easily leads to misoperation, affecting the normal use of the device, threatening driving safety, and making it difficult to balance the effect of preventing accidental touch with the convenience of adult interaction.

[0025] Specifically, Figure 1 This is a flowchart of a vehicle interaction control method provided according to an embodiment of this application.

[0026] like Figure 1 As shown, the vehicle interaction control method includes the following steps: In step S101, the vehicle's driving status data is collected, as well as the gesture data and human posture information of the corresponding user inside the vehicle.

[0027] It is understood that, in the embodiments of this application, driving status data can be understood as various state information of the vehicle during driving, which may include, but is not limited to, changes in driving speed, magnitude of acceleration, vehicle speed, whether the vehicle body is stable, driving direction, etc.; gesture data may include, but is not limited to, the three-dimensional contour of the hand, such as the shape of the palm, the width of the palm, the proportion of the palm size, the position of the finger joints, the length of the fingers, the degree of finger bending, etc.; hand movement trajectory, such as the hand movement path, the starting and ending positions of the gesture, the direction of movement (such as from left to right, up or down, etc.), speed (displacement per unit time), acceleration, etc.; human posture information may include, but is not limited to, sitting posture, such as sitting upright, reclining or other postures, etc.; the angle of contact between the back and the seat; the pressure contact area, such as the size of the contact area between the buttocks and the seat, etc.; the pressure concentration area, such as whether the pressure is concentrated in the front, middle or rear of the seat, etc.; the pressure change trend, which is not specifically limited in this application.

[0028] Furthermore, embodiments of this application can collect vehicle driving status data through vehicle-mounted sensors, such as vehicle speed sensors, acceleration sensors, and vehicle posture sensors, to provide comprehensive basic data for subsequent judgment of driving scenarios and adjustment of gesture recognition thresholds; and use 3D ToF (3D Time of Flight Camera) cameras to collect gesture data, such as the three-dimensional contour of the hand and the trajectory of hand movement, and use seat pressure sensors to obtain human posture information.

[0029] As one possible approach, embodiments of this application can collect not only vehicle driving status data, but also gesture data and human posture information of the corresponding user inside the vehicle.

[0030] For example, in combination Figure 2 As shown, the data acquisition and transmission content of this application embodiment includes the following steps: Step S201: Activate the vehicle sensors, 3D ToF camera, and seat pressure sensor.

[0031] In this embodiment, vehicle driving status data is collected by onboard sensors, gesture data of rear passengers is collected by a 3D ToF camera, and human posture information of rear passengers is obtained by a seat pressure sensor.

[0032] Step S202: Collect vehicle driving status data.

[0033] Among them, driving status data can be obtained by onboard sensors.

[0034] Step S203: Collect user gesture data.

[0035] In this process, the 3D hand contour data is generated by the infrared LEDs inside the 3D ToF camera continuously emitting near-infrared light pulses towards the hand area. The hand surface reflects these light pulses, and the camera's sensor receives the reflected light signals. Based on the time-of-flight of the light pulses, the distance from each point on the hand surface to the camera is calculated, resulting in a series of depth values. Combining these depth values ​​with the camera's imaging principle, these depth values ​​are converted into 3D coordinate points in space. These numerous 3D coordinate points constitute the 3D hand contour data. For example, by continuously measuring the distances to different parts of the hand, information such as palm width and finger length can be determined.

[0036] The hand movement trajectory in the gesture data is continuously acquired by a 3D ToF camera at a certain frame rate (e.g., 30 or 60 frames per second). In each frame, the position of the same hand feature point at different times is determined by recognizing and matching three-dimensional coordinate points. Then, as time progresses, the continuous positional changes of the feature points are recorded, thereby generating hand movement trajectory data. Furthermore, the 3D ToF camera in this embodiment can accurately record whether the hand slides at a constant speed or waves at an accelerated speed within a certain period of time.

[0037] Step S204: Collect the user's human posture information.

[0038] The seat pressure sensor is a proprietary sensor installed in the seat, whose main function is to acquire the human posture information of rear passengers.

[0039] It should be noted that in this embodiment, multiple pressure sensors are evenly distributed on the seat surface, acting as "pressure detection points." When a rear passenger sits down, different parts of the body (such as the buttocks and thighs) exert different levels of pressure on the seat. Each sensor outputs a corresponding electrical signal based on the magnitude of the pressure. After amplification and filtering, these electrical signals are converted into pressure values ​​through data processing. Algorithms analyze areas of concentrated pressure and trends of pressure changes to determine whether the passenger is sitting upright, reclining, or in another posture, thereby generating corresponding human posture information.

[0040] Step S205: Data transmission.

[0041] In this embodiment, gesture data and human posture information can be synchronously transmitted to the central processing module via a vehicle-specific high-speed data bus (such as CAN (Controller Area Network) bus or Ethernet). During the transmission process, a data verification mechanism (such as checksum) is used to ensure data integrity and avoid data loss or corruption due to vehicle electromagnetic interference.

[0042] Step S206: Data processing.

[0043] In this embodiment, after receiving the data, the central processing module first performs noise reduction. For example, for gesture data, it filters out invalid coordinate points caused by changes in light or hand occlusion; for human posture information, it removes abnormal pressure values ​​caused by instantaneous fluctuations in the sensor. Then, it performs format unification, such as converting the two types of data into a format that the backend AI model can recognize (such as structured data tables, 3D point cloud files, etc.) to ensure that subsequent feature extraction and identity determination can be directly called, avoiding processing delays caused by data format incompatibility.

[0044] Step S207: Data caching.

[0045] In this embodiment, the processed data can be temporarily stored in the cache of the central processing module, waiting to enter the next stage (such as feature extraction). The entire process needs to be completed in milliseconds to meet the real-time interaction requirements in vehicle scenarios.

[0046] In step S102, the initial gesture command for the vehicle is generated based on driving status data, gesture data, and human posture information.

[0047] In some embodiments, the present application can generate corresponding initial gesture commands based on driving status data, gesture data, and human posture information.

[0048] Optionally, in one embodiment of this application, generating an initial gesture command for the vehicle based on driving status data, gesture data, and human posture information includes: extracting gesture features from the gesture data and extracting posture features from the human posture information; determining the user's identity recognition result based on the gesture features and posture features; determining the user's gesture recognition result based on the driving status data and gesture features; and generating an initial gesture command based on the identity recognition result and gesture recognition result.

[0049] It is understood that, in the embodiments of this application, gesture features may include, but are not limited to, hand shape features, such as the size ratio of the palm and the degree of finger bending; motion features, such as the starting and ending positions of the gesture and the direction of movement; posture features may include, but are not limited to, pressure distribution features, such as pressure concentration areas and pressure change trends; body posture features, such as the angle of contact between the back and the seat, etc., and this application does not impose specific limitations.

[0050] In some embodiments, this application can extract gesture features from gesture data, extract posture features from human posture information, and then determine the user's identity recognition result based on the gesture features and posture features, and determine the user's gesture recognition result based on driving status data and gesture features, thereby generating an initial gesture command based on the identity recognition result and gesture recognition result.

[0051] For example, embodiments of this application can be combined with Figure 3 As shown, gesture and identity recognition are performed.

[0052] Step S301: Receive gesture data and human posture information.

[0053] Step S302: Extract gesture features and pose features.

[0054] In this application embodiment, gesture features can be extracted from gesture data, and posture features can be extracted from human posture information.

[0055] Step S303: Perform identity verification.

[0056] In this embodiment, an AI model can be used to identify the user's identity, thereby determining whether the user is a child or an adult.

[0057] It should be noted that the AI ​​model in this embodiment is based on a Transformer encoder-decoder architecture, combined with AI technology and a specially constructed children's feature database. Specifically, the AI ​​model adopts a multi-layer Transformer architecture, including at least 6 encoder layers and 6 decoder layers. Each layer contains a multi-head self-attention mechanism with 8 attention heads. The hidden layer dimension is 512, and the intermediate layer dimension of the feedforward neural network is 2048. The model input receives structured feature vectors extracted from the children's feature database, and the output generates corresponding evaluation / recommendation results.

[0058] Furthermore, the total number of parameters in the AI ​​model in this embodiment is approximately 60M to 110M (millions), with the embedding layer having a dimension of 256. Layer normalization is used for normalization, and the activation function is GELU (Gaussian Error Linear Unit). The Adam optimizer is used during model training, with an initial learning rate set to... The Warmup strategy (Warmup Steps=4000) was adopted, the batch size was 32 to 64, the training rounds were 10 to 50, and Dropout (dropout rate of 0.1 to 0.3) was introduced to prevent overfitting.

[0059] In addition, the children’s feature database contains multi-dimensional feature data that is structured and stored by age group (0-3 years, 3-6 years, 6-12 years, etc.), physiological feature data (such as height, weight, etc.), and behavioral feature data (such as interaction frequency, etc.).

[0060] Step S304: Perform gesture recognition.

[0061] In this embodiment, gesture recognition can be performed using a pre-trained model, and the input data is gesture features.

[0062] Specifically, the pre-trained model is a deep learning model based on the fusion of CNN (Convolutional Neural Network) and RNN (Recurrent Neural Network). The overall architecture adopts a CNN-LSTM (Long Short-Term Memory) hybrid architecture, that is, the front-end CNN is responsible for extracting spatial features, and the back-end LSTM is responsible for extracting temporal features.

[0063] In terms of model structure, the CNN part uses a ResNet-18 backbone network, containing four residual blocks, each with two convolutional layers. The kernel size is 3×3, and the number of channels is 64, 128, 256, and 512 respectively. The LSTM part is connected to the back end of the CNN, containing two LSTM layers, each with 256 hidden units. The model ends with two fully connected layers with dimensions of 512, 256, and N, where N is the number of gesture categories, for example, N is between 20 and 50. The CNN part uses ReLU as the activation function, the LSTM part uses Tanh as the activation function, and the output layer uses Softmax. Each convolutional layer is followed by a Batch Normalization (BN) layer for normalization.

[0064] Regarding model parameters, the total number of parameters in a CNN-LSTM model is approximately 8M to 15M (millions). The input data consists of gesture features, with dimensions of 224×224×3 (RGB images) or 128-dimensional feature vectors (after feature extraction). The output is a probability distribution of gesture categories, with N classes. The Adam optimizer is used during model training. Set it to 0.9. Set the initial learning rate to 0.999. The learning rate decay strategy employs cosine annealing, with batch sizes ranging from 32 to 64 and training epochs from 50 to 100. Dropout (with a dropout rate of 0.3 to 0.5) and L2 weight decay (with a coefficient of...) are also introduced. To prevent overfitting, data augmentation methods include random rotation (±15 degrees), random cropping, horizontal flipping, and color dithering.

[0065] In terms of pre-training, the CNN-LSTM model is pre-trained using publicly available gesture datasets (such as LSA64). The pre-training task is gesture classification, and the cross-entropy loss function is used.

[0066] Step S305: Integrate the identity recognition result and gesture recognition result to generate initial gesture instructions.

[0067] Optionally, in one embodiment of this application, determining the user's identity recognition result based on gesture features and posture features includes: obtaining target gesture features and target posture features corresponding to the target identity in a preset identity database; matching the gesture features with the target gesture features and matching the posture features with the target posture features, and determining the user's actual identity based on the matching results; and determining the actual identity as the identity recognition result if the matching results meet preset matching conditions.

[0068] It is understood that, in the embodiments of this application, the preset identity database can be understood as a children's feature database, which may include, but is not limited to, children aged 0-3 years, 3-6 years, 6-12 years, etc. The specific settings can be made by those skilled in the art according to the actual situation, and this application does not impose specific limitations.

[0069] In some embodiments, this application can obtain target gesture features and target posture features corresponding to a target identity from a preset identity database, match the user-collected gesture features with the target gesture features, and match the posture features with the target posture features. Based on the matching results, the user's actual identity is determined, and if the matching results meet preset matching conditions, the actual identity is confirmed as the identity recognition result. The preset matching conditions can be set by those skilled in the art according to actual circumstances, and this application does not impose specific limitations.

[0070] For example, in this application embodiment, the target gesture features and target posture features of children can be extracted from the children feature database, and the actual collected gesture features are matched with the target gesture features, and the posture features are matched with the target posture features. Then, if the matching result is completely mismatched, the user's actual identity is determined to be an adult, and the matching result is determined to meet the preset matching conditions. The actual identity is determined as the identity recognition result, and the standard recognition process is entered. Subsequent instruction recognition is carried out according to the conventional recognition logic for adult operating habits and features.

[0071] In addition, in the case of a perfect match, this application embodiment can determine that the user’s actual identity is a child and, after determining that the matching result meets the preset matching conditions, control the vehicle to enter the interaction restriction mode, thereby more strictly screening and intercepting instructions that may be caused by the child’s misoperation, and preventing the child from accidentally triggering the vehicle system function due to unintentional hand movements or changes in sitting posture.

[0072] Optionally, in one embodiment of this application, determining the user's gesture recognition result based on driving status data and gesture features includes: determining whether the gesture corresponding to the gesture features is an interfering action based on the gesture features; if the gesture is not an interfering action, extracting the vehicle speed from the driving status data and dynamically determining the gesture recognition threshold based on the vehicle speed; calculating the matching degree between the gesture features and the expected gesture features; if the matching degree is greater than the gesture recognition threshold, determining the gesture as a valid gesture and outputting the gesture recognition result based on the valid gesture.

[0073] It is understandable that, in this embodiment of the application, since the display status is real-time and dynamic, and the input sources are from multiple parties while the output end is only from one party, it is necessary to set a priority algorithm for the display status. The priority is designed according to vehicle speed, and the ultimate goal of this algorithm is to provide the most effective real-time information. This can be understood as the gesture recognition threshold being determined by vehicle speed. The main content is as follows: Figure 4 As shown, it includes the following steps: Step S401: Collect vehicle driving status data.

[0074] In this embodiment, vehicle-mounted sensors, such as vehicle speed sensors, acceleration sensors, and vehicle posture sensors, can collect comprehensive driving status data of the vehicle during driving, providing comprehensive basic data for subsequent judgment of driving scenarios and adjustment of gesture recognition thresholds.

[0075] Step S402: Extract vehicle speed.

[0076] In this embodiment, vehicle speed, a core parameter, can be extracted from driving status data. Vehicle speed is one of the most critical indicators for determining the driving scenario (such as high speed, low speed, or stationary). It directly reflects the speed of the vehicle's movement and is the core basis for subsequent determination of the driving scenario.

[0077] Step S403: Detect the driving scene and adjust the gesture recognition threshold.

[0078] The driving scenarios in this application embodiment may include, but are not limited to, high-speed driving scenarios, low-speed driving scenarios, parking scenarios, etc., and this application does not impose specific limitations.

[0079] Furthermore, this embodiment of the application determines whether the vehicle is traveling at high speed by measuring vehicle speed, and automatically increases the gesture recognition threshold to reduce the risk of accidental touches. This is because when traveling at high speeds, the driver needs to focus more on driving. If the gesture recognition is too sensitive, it is easy to accidentally trigger vehicle functions due to unintentional hand movements. Increasing the threshold can effectively avoid this situation and ensure driving safety.

[0080] In this embodiment, when the vehicle is determined to be in a low-speed or parked scenario based on vehicle speed, the gesture recognition threshold is automatically lowered to improve interaction sensitivity. This is because when the vehicle is at low speed or parked, the driver or passengers have a more frequent need to perform gesture operations, and the risk of accidental touches is lower. Lowering the threshold makes gestures easier to recognize, facilitating various in-vehicle function interactions.

[0081] Step S404: Update the gesture recognition threshold to adapt to the current driving scenario.

[0082] In this embodiment, the corresponding gesture recognition threshold can be automatically updated. The updated threshold can better adapt to the current driving scenario of the vehicle, so that the gesture recognition function can ensure both the convenience of interaction and driving safety in different scenarios, and achieve a more intelligent and safer in-vehicle interaction experience.

[0083] In some embodiments, this application can determine whether the gesture corresponding to the gesture feature is an interfering action based on the gesture feature. If the gesture is not an interfering action, the vehicle speed in the driving status data is extracted, and the gesture recognition threshold is dynamically determined based on the vehicle speed. Then, the matching degree between the gesture feature and the expected gesture feature is calculated. If the matching degree is greater than the gesture recognition threshold, the gesture is determined as a valid gesture, and the gesture recognition result is output based on the valid gesture.

[0084] The interference actions may include, but are not limited to, children waving their arms randomly or making meaningless hand movements; this application does not impose specific restrictions.

[0085] For example, embodiments of this application can analyze the input gesture features to identify whether the corresponding gesture belongs to common interference actions, thereby assisting in the subsequent judgment of valid gestures. Then, if the gesture does not belong to interference actions, the gesture recognition threshold is dynamically determined based on the vehicle speed in the driving status data, and the matching degree between the gesture features and the expected gesture features (such as the shape of the gesture, movement trajectory, etc.) is calculated. If the matching degree is greater than the gesture recognition threshold, it is marked as a valid gesture so that the corresponding vehicle function can be executed subsequently; otherwise, it is directly filtered and not processed as a valid gesture, thereby outputting the corresponding gesture recognition result.

[0086] Optionally, in one embodiment of this application, generating an initial gesture command based on the identity recognition result and the gesture recognition result includes: generating an initial gesture command based on the identity recognition result and the gesture recognition result when the identity recognition result meets the preset authentication conditions and the gesture recognition result meets the preset gesture verification conditions.

[0087] In some embodiments, when generating an initial gesture command based on the identity recognition result and the gesture recognition result, the present application can verify the identity recognition result and the gesture recognition result, i.e., whether the identity recognition result meets preset authentication conditions and whether the gesture recognition result meets preset gesture verification conditions. If both the identity recognition result and the gesture recognition result meet the preset authentication conditions and the gesture verification conditions, the corresponding initial gesture command is generated based on the identity recognition result and the gesture recognition result. The preset authentication conditions and preset gesture verification conditions can be set by those skilled in the art according to actual circumstances, and the present application does not impose specific limitations.

[0088] For example, in this application embodiment, the identity recognition result that meets the preset identity verification conditions and the gesture recognition result that meets the preset gesture verification conditions can be integrated to form a comprehensive result containing identity information and valid gesture information, generate a corresponding initial gesture command, and output the initial gesture command to the next stage, so that the next stage can perform corresponding vehicle function control or interactive operation according to the initial gesture command.

[0089] In step S103, the verification result of the initial gesture command is obtained, and if the verification result meets the preset verification conditions, the gesture command of the vehicle is determined according to the initial gesture command, and the vehicle is controlled according to the gesture command.

[0090] In some embodiments, the present application can perform secondary verification on the initial gesture command to obtain the corresponding verification result. If the verification result meets preset verification conditions, the vehicle's gesture command is determined based on the initial gesture command, and the vehicle is controlled. The preset verification conditions can be set by those skilled in the art according to actual conditions, and the present application does not impose specific limitations.

[0091] For example, embodiments of this application may employ a two-stage control flow of wake-up and execution, setting up a feedback confirmation mechanism to perform secondary verification of the initial gesture command. When the initial gesture command is detected, visual and auditory cues are output. Only when a confirmation gesture is detected within a specified time (e.g., 5 seconds, this application does not impose a specific limitation) is the initial gesture command used as the actual gesture command, and the vehicle is controlled according to the gesture command. The main content is as follows: Figure 5 As shown, it includes: Step S501: Receive initial gesture command.

[0092] The initial gesture instruction can be understood as an instruction obtained through identity recognition and gesture recognition, and its validity cannot be fully determined at present, and further verification is needed.

[0093] Step S502: Initiate the two-stage operation of wake-up and execution.

[0094] In this embodiment, upon receiving an initial gesture command, a two-stage operation of wake-up and execution is initiated. The wake-up stage can be understood as informing the user, through prompts or other means, that the system has received the command and is preparing for further confirmation; the execution stage can be understood as performing the corresponding function after confirming the command's validity.

[0095] Step S503: Output visual and auditory cues, and start timing simultaneously.

[0096] Visual cues may include, but are not limited to, displaying a specific confirmation interface on the vehicle screen, flashing indicator lights, etc., and this application does not impose specific limitations on them; auditory cues may include, but are not limited to, issuing prompts or voice prompts, etc., and this application does not impose specific limitations on them.

[0097] Furthermore, embodiments of this application can inform the user that an initial gesture command can be used for confirmation through visual and auditory cues. A timer is then started to determine whether the user has performed a confirmation gesture within a specified time.

[0098] Step S504: Detect whether a confirmation gesture is detected within a specified time.

[0099] In this embodiment, after the timing begins, the system can continuously detect whether the user makes a confirmation gesture within a specified time. It should be noted that the confirmation gesture can be pre-set by the system and is a gesture action used to clearly indicate that the user confirms the initial gesture command.

[0100] Further, if so, proceed to step S506; otherwise, proceed to step S505.

[0101] Step S505: Termination instruction.

[0102] In this embodiment of the application, if no confirmation gesture is detected within a specified time, i.e., a timeout occurs, the instruction process will be terminated and the initial gesture instruction will no longer be processed.

[0103] Step S506: Use the initial gesture command as a gesture command and control the vehicle.

[0104] In this embodiment of the application, if a confirmation gesture is detected within a specified time, the initial gesture command can be used as a gesture command, and the corresponding vehicle function operation can be completed to control the vehicle.

[0105] Optionally, in one embodiment of this application, the method further includes: controlling the vehicle to enter an interaction restriction mode if the verification result does not meet the preset verification conditions.

[0106] As one possible implementation, embodiments of this application can control the vehicle to enter an interactive restriction mode if the verification result does not meet the preset verification conditions.

[0107] Furthermore, embodiments of this application can unlock the interaction restriction mode via password input or a dedicated wake-up gesture. For example... Figure 6 As shown, this application embodiment can unlock the interaction restriction mode based on password input and a dedicated wake-up gesture. The main contents are as follows: Step S601: Extract the vehicle speed.

[0108] Among them, vehicle speed is a key parameter for determining the vehicle unlocking requirement, which can be adapted to different scenarios and improve adaptability in different scenarios.

[0109] Step S602: Detect whether the unlocking requirement is triggered.

[0110] In this application, different verification methods can be generated when an adult triggers the unlocking request, such as password input verification and exclusive wake-up gesture verification. This application does not impose any specific limitations.

[0111] Among them, password input verification, that is, the embodiment of this application can verify the validity of the password entered by the user.

[0112] The exclusive wake-up gesture verification means that the embodiments of this application can recognize and verify the exclusive gestures made by the user.

[0113] Step S603: Determine the unlock verification method.

[0114] Step S604: Password input verification.

[0115] In this embodiment of the application, if the password input verification method is successful, step S606 is executed; otherwise, step S607 is executed.

[0116] Step S605: Verify exclusive wake-up gesture.

[0117] In this embodiment of the application, if the unlock verification method is a dedicated wake-up gesture verification, step S606 is executed if the verification is successful; otherwise, step S607 is executed.

[0118] Step S606: Unlock the interaction restriction mode.

[0119] Step S607: Maintain interaction restriction mode.

[0120] It can be understood that the embodiments of this application can unlock the interaction restriction mode when the password input verification or the dedicated wake-up gesture verification is successful, and the relevant functions can be used normally without being restricted by the interaction restriction; otherwise, the interaction restriction mode will be maintained to continue to prevent accidental operation.

[0121] The vehicle interaction control method proposed in this application can collect vehicle driving status data, user gesture data, and human posture information respectively. Then, based on the driving status data, gesture data, and human posture information, corresponding initial gesture commands are obtained. If the verification result of the initial gesture command meets preset verification conditions, the initial gesture command is used as the vehicle's gesture command to complete vehicle control. The initial gesture command is generated collaboratively based on multiple types of information, including driving status, gesture data, and human posture. After compliance verification and screening, the final control command is determined. This improves the accuracy of gesture command recognition, effectively identifies invalid accidental touch operations, and balances the safety of use in driving scenarios with the reliability of in-vehicle gesture interaction control. Therefore, it solves the problems in related technologies that rely solely on a single trajectory or simple gesture features for recognition, without considering children's physiological characteristics and behavioral patterns, lacking multi-dimensional differentiation criteria, graded verification, and scenario adaptation capabilities. This makes it difficult to effectively identify children's accidental touch behavior, easily leading to misoperations, affecting normal device use, threatening driving safety, and making it difficult to balance anti-accidental touch effects with the convenience of adult interaction.

[0122] Next, the vehicle interaction control device proposed according to the embodiments of this application is described with reference to the accompanying drawings.

[0123] Figure 7 This is a block diagram of a vehicle interaction control device provided according to an embodiment of this application.

[0124] like Figure 7 As shown, the vehicle interaction control device 10 includes: a data acquisition module 100, a generation module 200, and a first control module 300.

[0125] The acquisition module 100 is used to collect vehicle driving status data and collect the gesture data and human posture information of the corresponding user inside the vehicle.

[0126] The generation module 200 is used to generate initial gesture commands for the vehicle based on driving status data, gesture data, and human posture information.

[0127] The first control module 300 is used to obtain the verification result of the initial gesture command, and if the verification result meets the preset verification conditions, determine the vehicle's gesture command according to the initial gesture command and control the vehicle according to the gesture command.

[0128] Optionally, in one embodiment of this application, the generation module 200 includes: an extraction unit, a first determination unit, a second determination unit, and a generation unit.

[0129] The extraction unit is used to extract gesture features from gesture data and posture features from human posture information.

[0130] The first determining unit is used to determine the user's identity recognition result based on gesture features and posture features.

[0131] The second determining unit is used to determine the user's gesture recognition result based on driving status data and gesture features.

[0132] The generation unit is used to generate initial gesture instructions based on the identity recognition results and gesture recognition results.

[0133] Optionally, in one embodiment of this application, the first determining unit includes: an acquisition subunit, a matching subunit, and a first determining subunit.

[0134] The acquisition subunit is used to acquire the target gesture features and target posture features corresponding to the target identity in the preset identity database.

[0135] The matching subunit is used to match gesture features with target gesture features and posture features with target posture features, and determine the user's actual identity based on the matching results.

[0136] The first determining subunit is used to determine the actual identity as the identity recognition result when the matching result meets the preset matching conditions.

[0137] Optionally, in one embodiment of this application, the second determining unit includes: a second determining subunit, a third determining subunit, a calculation subunit, and a fourth determining subunit.

[0138] The second determining subunit is used to determine whether the gesture corresponding to the gesture feature is an interfering action based on the gesture feature.

[0139] The third determining subunit is used to extract the vehicle speed from the driving status data when the gesture is not a disturbing action, and dynamically determine the gesture recognition threshold based on the vehicle speed.

[0140] The computational subunit is used to calculate the matching degree between the gesture features and the desired gesture features.

[0141] The fourth determining subunit is used to determine the gesture as a valid gesture when the matching degree is greater than the gesture recognition threshold, and output the gesture recognition result based on the valid gesture.

[0142] Optionally, in one embodiment of this application, the generating unit includes: a generating subunit.

[0143] The generation subunit is used to generate an initial gesture command based on the identity recognition result and the gesture recognition result, provided that the identity recognition result meets the preset identity verification condition and the gesture recognition result meets the preset gesture verification condition.

[0144] Optionally, in one embodiment of this application, a second control module is also included.

[0145] The second control module is used to control the vehicle to enter the interaction restriction mode when the verification result does not meet the preset verification conditions.

[0146] It should be noted that the foregoing explanation of the vehicle interaction control method embodiment also applies to the vehicle interaction control device of this embodiment, and will not be repeated here.

[0147] The vehicle interaction control device proposed in this application can collect vehicle driving status data, user gesture data, and human posture information respectively. Based on these data, it obtains corresponding initial gesture commands. If the verification result of the initial gesture command meets preset verification conditions, the initial gesture command is used as the vehicle's gesture command to control the vehicle. By collaboratively generating the initial gesture command using multiple types of information—driving status, gesture data, and human posture—and then verifying and selecting the final control command, the device improves the accuracy of gesture command recognition, effectively identifies invalid accidental touches, and balances driving safety with the reliability of in-vehicle gesture interaction control. This solves the problems in related technologies that rely solely on a single trajectory or simple gesture features for recognition, failing to consider children's physiological characteristics and behavioral patterns, lacking multi-dimensional differentiation criteria, graded verification, and scenario adaptation capabilities. These technologies are unable to effectively identify children's accidental touches, easily leading to misoperations, affecting normal device use, threatening driving safety, and failing to balance anti-accidental touch effects with adult ease of use.

[0148] Figure 8 This is a schematic diagram of the structure of a vehicle according to an embodiment of this application. The vehicle may include: The memory 801, the processor 802, and the computer program stored on the memory 801 and capable of running on the processor 802.

[0149] When the processor 802 executes the program, it implements the vehicle interaction control method provided in the embodiment.

[0150] Furthermore, the vehicle also includes: Communication interface 803 is used for communication between memory 801 and processor 802.

[0151] The memory 801 is used to store computer programs that can run on the processor 802.

[0152] The memory 801 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0153] If the memory 801, processor 802, and communication interface 803 are implemented independently, then the communication interface 803, memory 801, and processor 802 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized into address buses, data buses, control buses, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0154] Optionally, in a specific implementation, if the memory 801, processor 802, and communication interface 803 are integrated on a single chip, then the memory 801, processor 802, and communication interface 803 can communicate with each other through an internal interface.

[0155] The processor 802 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0156] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described vehicle interaction control method.

[0157] This application also provides a computer program product, including a computer program that, when executed, implements the above-described vehicle interaction control method.

[0158] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0159] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0160] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0161] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0162] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0163] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0164] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0165] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A control method for vehicle interaction, characterized in that, Includes the following steps: Collect vehicle driving status data, and collect the corresponding user's gesture data and human posture information inside the vehicle; Based on the driving status data, the gesture data, and the human posture information, the initial gesture command for the vehicle is generated; The verification result of the initial gesture command is obtained, and if the verification result meets the preset verification conditions, the gesture command of the vehicle is determined according to the initial gesture command, and the vehicle is controlled according to the gesture command.

2. The method according to claim 1, characterized in that, The step of generating initial gesture commands for the vehicle based on the driving status data, the gesture data, and the human posture information includes: Extract gesture features from the gesture data and extract posture features from the human posture information; Based on the gesture features and posture features, the user's identity recognition result is determined; Based on the driving status data and the gesture features, the user's gesture recognition result is determined; Based on the identity recognition result and the gesture recognition result, the initial gesture command is generated.

3. The method according to claim 2, characterized in that, The process of determining the user's identity based on the gesture features and posture features includes: Obtain the target gesture features and target posture features corresponding to the target identity from the preset identity database; The gesture features are matched with the target gesture features, and the posture features are matched with the target posture features. The user's actual identity is determined based on the matching results. If the matching result meets the preset matching conditions, the actual identity is determined as the identity recognition result.

4. The method according to claim 2, characterized in that, The step of determining the user's gesture recognition result based on the driving status data and the gesture features includes: Based on the gesture features, determine whether the gesture corresponding to the gesture features is an interfering action; If the gesture is not a distracting action, extract the vehicle speed feature from the driving status data, and dynamically determine the gesture recognition threshold based on the vehicle speed feature; Calculate the matching degree between the gesture features and the expected gesture features; If the matching degree is greater than the gesture recognition threshold, the gesture is determined as a valid gesture, and the gesture recognition result is output based on the valid gesture.

5. The method according to claim 2, characterized in that, The step of generating the initial gesture command based on the identity recognition result and the gesture recognition result includes: If the identity recognition result meets the preset identity verification conditions and the gesture recognition result meets the preset gesture verification conditions, the initial gesture command is generated based on the identity recognition result and the gesture recognition result.

6. The method according to claim 1, characterized in that, Also includes: If the verification result does not meet the preset verification conditions, the vehicle is controlled to enter the interaction restriction mode.

7. A vehicle interaction control device, characterized in that, include: The data acquisition module is used to collect vehicle driving status data, as well as the gesture data and human posture information of the corresponding user inside the vehicle. The generation module is used to generate the initial gesture commands for the vehicle based on the driving status data, the gesture data, and the human posture information. The control module is used to acquire the verification result of the initial gesture command, and if the verification result meets the preset verification conditions, determine the gesture command of the vehicle according to the initial gesture command, and control the vehicle according to the gesture command.

8. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the vehicle interaction control method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the vehicle interaction control method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed, is used to implement the vehicle interaction control method as described in any one of claims 1-6.

Images