Vehicle tail door control method and device, equipment, storage medium and program product

Abstract

The invention provides a vehicle tail door control method and device, equipment, a storage medium and a program product, and relates to the technical field of vehicles. The target image data is acquired by a visual sensor associated with a vehicle tail door; performing human body key point detection on the target image data based on sensor parameters of the visual sensor to obtain human body key point data in the target image data; the sensor parameters are used for adaptively adjusting a feature extraction weight for performing human body key point detection on the target image data; kicking behavior recognition is conducted based on the human body key point data to obtain a recognition result, and the vehicle tail door is controlled based on the recognition result. By adopting the method, the platformization adaptation capability of the technology of kicking induction control of the tail door can be improved, so that a sensor bracket and a detection algorithm do not need to be developed independently for different vehicle types, and the overall research and development cost and period of the vehicle are saved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a vehicle tailgate control method, device, equipment, storage medium, and program product. Background Art

[0002] With the rapid development of intelligent vehicles, the automotive industry is evolving towards a more convenient and humanized direction. Among them, the electric tailgate, as an important configuration to improve the convenience of vehicle use, has gradually become a standard feature of mid-to-high-end models.

[0003] In related technologies, users can control the opening and closing of the tailgate through various methods, such as remote control keys, in-vehicle buttons, or tailgate buttons. These control methods meet daily needs to a certain extent and promote the development of automated tailgate operation. To address the inconvenience of traditional operation when both hands are occupied (such as when carrying items or holding a baby), kick-sensing tailgate control technology has emerged. However, because kick sensing generally requires the sensor to be fixed in a specific position on the bumper, the sensor bracket and detection algorithm must be developed separately for different vehicle models. Summary of the Invention

[0004] The main purpose of the embodiments of this application is to propose a vehicle tailgate control method, device, equipment, storage medium and program product, aiming to improve the platform adaptability of the technology of kick-sensing tailgate control, so that there is no need to develop sensor brackets and detection algorithms separately for different vehicle models, saving the overall vehicle R&D cost and cycle.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application provides a vehicle tailgate control method, the method comprising: Acquire target image data; the target image data is collected by a visual sensor associated with the vehicle tailgate; Performing human key point detection on the target image data based on sensor parameters of the visual sensor to obtain human key point data in the target image data; the sensor parameters are used to adaptively adjust feature extraction weights for performing human key point detection on the target image data; A kicking behavior is recognized based on the human body key point data to obtain a recognition result, and the vehicle tailgate is controlled based on the recognition result.

[0006] In some embodiments, the performing human body key point detection on the target image data based on the sensor parameters of the visual sensor includes: Adaptively adjusting a preset initial feature extraction weight based on sensor parameters of the visual sensor to obtain an adjusted target feature extraction weight; the initial feature extraction weight is an initial feature extraction weight for performing human key point detection on the target image data; Human body key point detection is performed on the target image data based on the target feature extraction weights.

[0007] In some embodiments, the performing human key point detection on the target image data based on the sensor parameters of the visual sensor includes: The target image data is subjected to human key point detection by a preset adaptive detection model, and the feature extraction weights for human key point detection on the target image data are adaptively adjusted based on the sensor parameters by the adaptive detection model.

[0008] In some embodiments, the adaptively adjusting the feature extraction weights for performing human key point detection on the target image data based on the sensor parameters by the adaptive detection model includes: Generating a scaling factor and a bias term of a convolution block based on the sensor parameters by the adaptive detection model, so as to adaptively adjust a feature extraction weight for performing human key point detection on the target image data based on the scaling factor and the bias term; The convolution block is any convolution block in the multi-layer convolution block of the adaptive detection model.

[0009] In some embodiments, the human body key point data includes planar human body key point data, and the kicking behavior recognition based on the human body key point data to obtain a recognition result includes: Performing spatial mapping processing on the planar human body key point data to obtain spatial human body key point data; The kicking behavior is recognized based on the spatial human body key point data to obtain a recognition result.

[0010] In some embodiments, performing spatial mapping processing on the planar human body key point data to obtain spatial human body key point data includes: The planar human body key point data is subjected to multi-scale temporal feature fusion processing by a preset dilated time convolution model to obtain spatial human body key point data output by the dilated time convolution model.

[0011] In some embodiments, the step of performing kicking behavior recognition based on the spatial human body key point data to obtain a recognition result includes: Extracting spatial foot key point data from the spatial human body key point data; Performing spatiotemporal feature encoding processing on the spatial foot key point data to obtain a feature vector corresponding to the spatial foot key point data; The feature vector is subjected to behavioral action decoding processing to obtain a recognition result of the kicking behavior.

[0012] In some embodiments, performing behavioral action decoding processing on the feature vector to obtain a kicking behavior recognition result includes: Performing behavioral action decoding processing on the feature vector using a preset classifier to obtain a predicted probability of the kicking behavior output by the classifier; When the predicted probability is greater than or equal to the preset probability threshold, determining the recognition result of the kicking behavior as a kicking behavior being recognized; When the predicted probability is less than the preset probability threshold, the recognition result of the kicking behavior is determined to be that the kicking behavior is not recognized.

[0013] To achieve the above-mentioned objectives, a second aspect of an embodiment of the present application provides a vehicle tailgate control device, the device comprising: A data acquisition module is used to acquire target image data; the target image data is acquired by a visual sensor associated with the vehicle tailgate; An adaptive detection module is configured to perform human key point detection on the target image data based on sensor parameters of the visual sensor to obtain human key point data in the target image data; the sensor parameters are used to adaptively adjust feature extraction weights for performing human key point detection on the target image data; A control module is used to identify the kicking behavior based on the human body key point data to obtain an identification result, and to control the vehicle tailgate based on the identification result.

[0014] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes a vehicle tailgate control device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the vehicle tailgate control method described in the first aspect when executing the computer program.

[0015] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the vehicle tailgate control method described in the first aspect above.

[0016] To achieve the above-mentioned purpose, the fifth aspect of the embodiments of the present application proposes a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the vehicle tailgate control method provided in the first aspect above.

[0017] The vehicle tailgate control method, apparatus, device, computer-readable storage medium and computer program product proposed in the embodiments of the present application obtain target image data; the target image data is collected by a visual sensor associated with the vehicle tailgate; human key point detection is performed on the target image data based on sensor parameters of the visual sensor to obtain human key point data in the target image data; the sensor parameters are used to adaptively adjust the feature extraction weights for human key point detection of the target image data; kicking behavior is recognized based on the human key point data to obtain a recognition result, and the vehicle tailgate is controlled based on the recognition result.

[0018] Compared to traditional kick-sensing tailgate control methods, the present embodiment, after acquiring target image data through a visual sensor associated with the tailgate, performs human key point detection on the target image data based on the visual sensor's sensor parameters. Specifically, the feature extraction weights for human key point detection in the target image data are adaptively adjusted based on the visual sensor's sensor parameters, thereby obtaining human key point data in the target image data. Finally, kicking behavior recognition is performed based on this human key point data, resulting in corresponding recognition results for controlling the tailgate.

[0019] In this way, the embodiment of the present application adaptively adjusts the feature extraction weights when performing human key point detection on the target image based on the sensor parameters of the visual sensor, which can optimize the human key point detection effect of the image data collected by the visual sensor under different installation environments. That is, no matter what type of vehicle or what position the visual sensor is installed in, the feature extraction weights can be adaptively adjusted to ensure the accuracy of human key point detection on the image data collected by the visual sensor. In other words, the embodiment of the present application can implement the same human key point detection algorithm for image data collected by visual sensors with different parameters to perform human key point detection, that is, enable the same algorithm to adapt to visual sensors of different vehicle models, thereby effectively solving the technical bottleneck of cross-model platform deployment of the technology of kick-sensing vehicle tailgate control, and improving the platform adaptability of the technology of kick-sensing vehicle tailgate control, thereby eliminating the need to develop detection algorithms and / or sensor brackets separately for each vehicle model. Without having to develop sensor brackets and detection algorithms separately for different vehicle models, the cross-model development cycle can be greatly shortened and R&D costs can be reduced.

[0020] In addition, the embodiment of the present application adaptively adjusts the feature extraction weights when detecting human key points in the target image based on the sensor parameters of the visual sensor, thereby optimizing the human key point detection effect of the image data collected by the visual sensor under different installation environments, and can further improve the accuracy of subsequent kicking action recognition based on human key data, thereby solving the problem of high trigger failure rate due to non-standard actions in the traditional kick sensing control method of the vehicle tailgate, so that different users can successfully trigger the tailgate to open under various kicking postures, thereby reducing the learning cost of user operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic flow chart of the steps of the vehicle tailgate control method provided in some embodiments of the present application; Figure 2 for Figure 1 A schematic diagram of a detailed step flow of step S102; Figure 3 for Figure 1 Another detailed flowchart of step S102; Figure 4 for Figure 3 Schematic diagram of the detailed steps of step S301; Figure 5 for Figure 1 Schematic diagram of the detailed process of step S103; Figure 6 for Figure 5 Schematic diagram of the detailed steps of step S502; Figure 7 for Figure 6 Schematic diagram of the detailed steps of step S603; Figure 8 A schematic flow chart of a complete embodiment of the vehicle tailgate control method provided in an embodiment of the present application; Figure 9 A schematic diagram of the structure of a vehicle tailgate control device provided in an embodiment of the present application; Figure 10 This is a schematic diagram of the hardware structure of the vehicle tailgate control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0023] It should be noted that although the device / system schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device / system or the sequence in the flowcharts. The terms "first," "second," and so on, used in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0025] First, the overall concept of the vehicle tailgate control method provided in the embodiment of the present application is described.

[0026] With the rapid development of intelligent vehicles, the automotive industry is evolving towards greater convenience and user-friendliness. Powered tailgates, a key feature enhancing vehicle convenience, have gradually become standard on mid- to high-end models. Currently, users can control the tailgate's opening and closing via a variety of methods, including remote control keys, in-vehicle buttons, or the tailgate button. These control methods meet daily needs to a certain extent and promote the development of automated tailgate operation. To address the inconvenience of traditional operating methods in scenarios where a user's hands are occupied (such as when carrying items or holding a baby), kick sensing technology has emerged. This technology allows users to trigger the tailgate to automatically open through leg movements (such as kicking or sweeping the leg), eliminating the need for manual operation, thereby improving ease of use to a certain extent. However, existing kick sensing technology has the obvious drawback of poor platform adaptability. Because the sensor needs to be fixedly installed in a specific position on the bumper, different vehicle models (such as multi-purpose vehicles (MPVs), sport utility vehicles (SUVs), and sedans) must separately develop sensor brackets and detection algorithms, increasing R&D costs and cycles.

[0027] In summary, although the kick sensing technology of existing electric tailgates in cars has solved the operation problem in scenarios where both hands are occupied to a certain extent, it still has the problem of poor platform adaptability, making it difficult to meet the usage needs of different user groups and the adaptation requirements of different models, and urgently needs further improvement and optimization.

[0028] To this end, the embodiments of the present application provide a vehicle tailgate control method, device, equipment, computer-readable storage medium and computer program product, aiming to overcome the shortcomings of the above-mentioned related technologies and enhance the platform adaptability of the technology of foot-sensing tailgate control, thereby eliminating the need to develop sensor brackets and detection algorithms separately for different vehicle models, and saving the overall vehicle R&D cost and cycle.

[0029] The vehicle tailgate control method, apparatus, device, computer-readable storage medium and computer program product proposed in the embodiments of the present application obtain target image data; the target image data is collected by a visual sensor associated with the vehicle tailgate; human key point detection is performed on the target image data based on sensor parameters of the visual sensor to obtain human key point data in the target image data; the sensor parameters are used to adaptively adjust the feature extraction weights for human key point detection of the target image data; kicking behavior is recognized based on the human key point data to obtain a recognition result, and the vehicle tailgate is controlled based on the recognition result.

[0030] Compared to traditional kick-sensing tailgate control methods, the present embodiment, after acquiring target image data through a visual sensor associated with the tailgate, performs human key point detection on the target image data based on the visual sensor's sensor parameters. Specifically, the feature extraction weights for human key point detection in the target image data are adaptively adjusted based on the visual sensor's sensor parameters, thereby obtaining human key point data in the target image data. Finally, kicking behavior recognition is performed based on this human key point data, resulting in corresponding recognition results for controlling the tailgate.

[0031] In this way, the embodiment of the present application adaptively adjusts the feature extraction weights when performing human key point detection on the target image based on the sensor parameters of the visual sensor, which can optimize the human key point detection effect of the image data collected by the visual sensor under different installation environments. That is, no matter what type of vehicle or what position the visual sensor is installed in, the feature extraction weights can be adaptively adjusted to ensure the accuracy of human key point detection on the image data collected by the visual sensor. In other words, the embodiment of the present application can implement the same human key point detection algorithm for image data collected by visual sensors with different parameters to perform human key point detection, that is, enable the same algorithm to adapt to visual sensors of different vehicle models, thereby effectively solving the technical bottleneck of cross-model platform deployment of the technology of kick-sensing vehicle tailgate control, and improving the platform adaptability of the technology of kick-sensing vehicle tailgate control, thereby eliminating the need to develop detection algorithms and / or sensor brackets separately for each vehicle model. Without having to develop sensor brackets and detection algorithms separately for different vehicle models, the cross-model development cycle can be greatly shortened and R&D costs can be reduced.

[0032] In addition, the embodiment of the present application adaptively adjusts the feature extraction weights when detecting human key points in the target image based on the sensor parameters of the visual sensor, thereby optimizing the human key point detection effect of the image data collected by the visual sensor under different installation environments, and can further improve the accuracy of subsequent kicking action recognition based on human key data, thereby solving the problem of high trigger failure rate due to non-standard actions in the traditional kick sensing control method of the vehicle tailgate, so that different users can successfully trigger the tailgate to open under various kicking postures, thereby reducing the learning cost of user operation.

[0033] Furthermore, when identifying kicking behaviors based on key human data, the present embodiment utilizes spatial perception technology to map key human points detected in a 2D plane into 3D space, constructing a 3D human key point sequence with temporal information. Subsequently, a combined action decoding module, based on spatiotemporal feature encoding, analyzes and identifies human actions in 3D space, accurately determining kicking behaviors. This solves the problem of traditional kick-sensing tailgate control methods, which lack the ability to analyze 3D spatial features and result in a high failure rate for triggering non-standard actions (such as small movements and special group movements). This improves the adaptability and robustness of action recognition for different user groups and different action forms, further reducing the learning curve for user operations.

[0034] Next, the vehicle tailgate control method, device, equipment, computer-readable storage medium and computer program product provided in the embodiments of the present application are specifically described through the following embodiments, and the vehicle tailgate control method provided in the embodiments of the present application is first described in detail.

[0035] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0036] It should be noted that the vehicle tailgate control method provided in the embodiments of the present application can be applied to a terminal or a server, and can also be software running on a terminal or server. In some embodiments, the terminal can be a computer device such as a smartphone, tablet computer, laptop computer, desktop computer, or an onboard terminal (such as an onboard computing platform) in a vehicle, or a terminal associated with the vehicle. "Associated with the vehicle" means that the terminal can communicate and exchange data with the vehicle over a network. The server can be a backend server terminal device, which can be configured as an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The software can include an application that implements the vehicle tailgate control method, a computer program, and a storage medium that carries the computer program. It should be understood that based on different design requirements of actual applications, in different feasible embodiments, the terminal, server side, and software that apply the vehicle tailgate control method provided in the embodiments of the present application may of course also be in other forms not listed here, and the vehicle tailgate control method provided in the embodiments of the present application does not specifically limit this.

[0037] Furthermore, the present application may be used in a wide variety of general-purpose or specialized computer system environments or configurations. For example, personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer computer devices, personal computers (PCs), minicomputers, mainframe computers, vehicles, distributed computing environments including any of the above, and the like. The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0038] For ease of understanding and explanation, the following text will use the terminal device applying the vehicle tailgate control method provided in the embodiment of the present application as an example to describe the various specific embodiments of the present application in detail. The implementation of the vehicle tailgate control method provided in the embodiment of the present application by any of the above-mentioned other forms of subject can refer to the process of the terminal device applying the vehicle tailgate control method described below.

[0039] Please refer to Figure 1 , Figure 1 The following is a flow chart of the steps in some embodiments of the vehicle tailgate control method provided in the present application. Figure 1 The following flowcharts of other steps illustrate the execution order of some method steps, but based on different design requirements of actual applications, the vehicle tailgate control method provided in the embodiment of the present application can certainly adopt an execution order different from the method steps shown in the figure. That is, Figure 1 The order of the steps in the method shown does not constitute a limitation on the execution logic order of the vehicle tailgate control method provided in the embodiment of the present application. Figure 1 Reasonable changes in the order of the steps of the method shown should be included in the scope of protection of the vehicle tailgate control method provided in the embodiments of the present application.

[0040] like Figure 1 As shown, in some embodiments, the vehicle tailgate control method provided by the embodiments of the present application may include steps S101 to S103 as shown below.

[0041] Step S101: Acquire target image data; the target image data is acquired by a visual sensor associated with a vehicle tailgate.

[0042] It should be noted that the visual sensor associated with the tailgate of the vehicle may be a camera installed on the tailgate of the vehicle. A communication connection is established between the terminal device and the camera. For example, the terminal device may establish a communication connection with the camera using the Ethernet network inside the vehicle, thereby transmitting data via an Ethernet cable. In addition, the visual sensor associated with the tailgate of the vehicle may also be a camera installed at other locations of the vehicle. The camera establishes a communication connection with the terminal device via a wired or wireless method, and is capable of capturing images of the environment at the tailgate of the vehicle, thereby inputting the captured image data into the terminal device. It should be understood that, based on different design requirements of actual applications, the visual sensor can be installed at different locations of the vehicle or even outside the vehicle. As long as the visual sensor is capable of capturing images of the environment at the tailgate of the vehicle and is capable of inputting the captured image data into the terminal device, it should be included in the scope of protection of this application.

[0043] When a user controls the tailgate by kicking, the visual sensor associated with the vehicle's tailgate captures real-time image data of the user's kicking action and uploads the captured target image data to the terminal device. The terminal device then obtains the target image data input by the visual sensor.

[0044] In some embodiments, the terminal device can continuously receive environmental image data collected in real time by the visual sensor, and then filter out target image data containing image elements of the user performing a kicking action to control the tailgate switch from the environmental image data.

[0045] In some embodiments, the terminal device can continuously receive video data collected by the visual sensor in real time and pre-process the video data, including image noise reduction to eliminate the impact of environmental noise on image quality, and then extract target image data containing image elements of the user performing a kicking action to control the tailgate switch from the pre-processed video data.

[0046] Step S102: performing human key point detection on the target image data based on sensor parameters of the visual sensor to obtain human key point data in the target image data; the sensor parameters are used to adaptively adjust feature extraction weights for performing human key point detection on the target image data.

[0047] After acquiring target image data input from the visual sensor, the terminal device further performs human key point detection on the target image data. During this process, the terminal device adaptively adjusts feature extraction weights for human key point detection based on sensor parameters of the visual sensor. In this way, after completing human key point detection on the target image data by adaptively adjusting the feature extraction weights for human key point detection, the terminal device can obtain human key point data from the target image data when the user performs a kicking action to control the tailgate.

[0048] In some embodiments, the terminal device adaptively adjusts the feature extraction weights for human key point detection of the target image data. The human key point data obtained by the human key point detection of the target image data may include but is not limited to the coordinates of 17 key points such as the user's ankles, knees, and hip joints in the image.

[0049] In some embodiments, when the terminal device adaptively adjusts the feature extraction weights for human key point detection in target image data based on the sensor parameters of the visual sensor, the sensor parameters used may include but are not limited to the installation height, viewing angle, intrinsic parameter matrix, distortion coefficient and other parameters of the visual sensor.

[0050] In some embodiments, the terminal device can perform parameter recognition on the visual sensor while acquiring the target image data input by the visual sensor, thereby obtaining the sensor parameters of the visual sensor. For example, in the case where the visual sensor is a camera installed on the tailgate of a vehicle, the terminal device can obtain the installation parameters through camera calibration, including: the installation height H of the rearview camera in the trunk, the viewing angle, , camera intrinsic parameter matrix K, distortion coefficient k.

[0051] In other embodiments, the terminal device may also perform parameter identification on the visual sensor before obtaining the target image data input by the visual sensor to identify the kicking behavior to control the tailgate of the vehicle (for example, the first time the visual sensor establishes a communication connection, and / or the user triggers the calibration of the visual sensor installation parameters, etc.), thereby obtaining the sensor parameters of the visual sensor, and storing the sensor parameters locally so that when the target image data is subsequently obtained, the sensor parameters are called from the local area for use in performing human body key point data on the target image data, and the feature extraction weights are adaptively adjusted.

[0052] Step S103: performing kicking behavior recognition based on the human body key point data to obtain a recognition result, and controlling the vehicle tailgate based on the recognition result.

[0053] After detecting key points of the target image data and obtaining the data, the terminal device then performs kicking behavior recognition based on the data, generating a corresponding recognition result. Based on the kicking behavior recognition result, the terminal device then controls the vehicle's tailgate accordingly, for example, opening or closing the tailgate.

[0054] In some embodiments, when the terminal device processes kicking behavior recognition based on human body key point data, it can extract the coordinates of key points such as the user's ankles, knees, hip joints, etc. in the human body key point data, and analyze the spatial position changes, motion trajectories and time series characteristics of these coordinates to determine whether kicking behavior is recognized.

[0055] In an embodiment of the present application, a terminal device acquires target image data input from a visual sensor associated with a vehicle's tailgate and performs human key point detection on the target image data. Furthermore, during this human key point detection process, the feature extraction weights for human key point detection on the target image data are adaptively adjusted based on the sensor parameters of the visual sensor, thereby obtaining human key point data of the user in the target image data when the user performs a kicking action to control the tailgate. Finally, the terminal device performs kicking action recognition based on the human key point data to obtain a corresponding recognition result. The vehicle's tailgate is then controlled accordingly based on the kicking action recognition result.

[0056] Compared to traditional kick-sensing tailgate control methods, the present embodiment, after acquiring target image data through a visual sensor associated with the tailgate, performs human key point detection on the target image data based on the visual sensor's sensor parameters. Specifically, the feature extraction weights for human key point detection in the target image data are adaptively adjusted based on the visual sensor's sensor parameters, thereby obtaining human key point data in the target image data. Finally, kicking behavior recognition is performed based on this human key point data, resulting in corresponding recognition results for controlling the tailgate.

[0057] In this way, the embodiment of the present application adaptively adjusts the feature extraction weights when performing human key point detection on the target image based on the sensor parameters of the visual sensor, which can optimize the human key point detection effect of the image data collected by the visual sensor under different installation environments. That is, no matter what type of vehicle or what position the visual sensor is installed in, the feature extraction weights can be adaptively adjusted to ensure the accuracy of human key point detection on the image data collected by the visual sensor. In other words, the embodiment of the present application can implement the same human key point detection algorithm for image data collected by visual sensors with different parameters to perform human key point detection, that is, enable the same algorithm to adapt to visual sensors of different vehicle models, thereby effectively solving the technical bottleneck of cross-model platform deployment of the technology of kick-sensing vehicle tailgate control, and improving the platform adaptability of the technology of kick-sensing vehicle tailgate control, thereby eliminating the need to develop detection algorithms and / or sensor brackets separately for each vehicle model. Without having to develop sensor brackets and detection algorithms separately for different vehicle models, the cross-model development cycle can be greatly shortened and R&D costs can be reduced.

[0058] In addition, the embodiment of the present application adaptively adjusts the feature extraction weights when detecting human key points in the target image based on the sensor parameters of the visual sensor, thereby optimizing the human key point detection effect of the image data collected by the visual sensor under different installation environments, and can further improve the accuracy of subsequent kicking action recognition based on human key data, thereby solving the problem of high trigger failure rate due to non-standard actions in the traditional kick sensing control method of the vehicle tailgate, so that different users can successfully trigger the tailgate to open under various kicking postures, thereby reducing the learning cost of user operation.

[0059] In some embodiments, when the terminal device performs human key point detection on a target image, it can first adaptively adjust the feature extraction weights for human key point detection on the target image data based on sensor parameters, and then use the adjusted feature extraction weights to perform human key point detection on the target image data, thereby obtaining human key point data.

[0060] Please refer to Figure 2 , Figure 2 for Figure 1 A schematic diagram of a detailed step flow of step S102.

[0061] like Figure 2 As shown, in some embodiments, the step of "performing human key point detection on the target image data based on the sensor parameters of the visual sensor" in the above-mentioned step S102 may include steps S201 and S202 as shown below.

[0062] Step S201: adaptively adjusting a preset initial feature extraction weight based on the sensor parameters of the visual sensor to obtain an adjusted target feature extraction weight; the initial feature extraction weight is an initial feature extraction weight for performing human key point detection on the target image data.

[0063] Before formally performing human key point detection on the target image data input by the visual sensor, the terminal device can adaptively adjust the initial feature extraction weights for human key point detection on the target image data based on the sensor parameters of the visual sensor to obtain the adjusted target feature extraction weights.

[0064] In some embodiments, the terminal device may first read the initial feature extraction weights for human key point detection in the target image data, which are pre-set (e.g., determined during training) by the detection algorithm for human key point detection in the target image data. The terminal device then adaptively adjusts some or all of these initial feature extraction weights based on the sensor parameters of the visual sensor, for example, by increasing the weight of a particular key point feature and / or decreasing the weight of a particular key point feature, thereby obtaining adjusted target feature extraction weights.

[0065] Step S202: performing human body key point detection on the target image data based on the target feature extraction weights.

[0066] After the terminal device obtains the adjusted target feature extraction weight, it can formally perform human key point detection on the target image data based on the target feature extraction weight, thereby obtaining human key point data in the target image data.

[0067] In some embodiments, the terminal device can also adaptively adjust the initial feature extraction weights based on the sensor parameters of the visual sensor while performing human key point detection on the target image data to obtain the adjusted target feature extraction weights, and then perform human key point detection on the target image data based on the target feature extraction weights to obtain human key point data in the target image data.

[0068] In some embodiments, the terminal device can perform human key point detection on the target image data through an adaptive detection model. In this case, the terminal device can integrate the sensor parameters of the visual sensor into the model input, so that the model can dynamically and adaptively adjust the feature extraction weights for human key point detection on the target image data according to the sensor parameters.

[0069] It should be noted that the adaptive detection model can be a 2D key point detection model based on multi-layer convolution blocks. For example, the single-stage model Hourglass Network and SimpleBaseline based on heat maps, the regression-based models DeepPose and Integral Human Pose Regression, the two-stage models Mask R-CNN and AlphaPose, as well as the lightweight models MobileNet and PP-TinyPose, etc. It should be understood that based on the different design requirements of actual applications, terminal devices in different implementations can use different models to perform human key point detection on the target image data. The vehicle tailgate control method provided in the embodiment of the present application does not limit the specific type of adaptive detection model.

[0070] Please refer to Figure 3 , Figure 3 for Figure 1 Another detailed flowchart of step S102 is shown in FIG.

[0071] like Figure 3 As shown, in some embodiments, the step of "performing human key point detection on the target image data based on the sensor parameters of the visual sensor" in the above-mentioned step S102 may also include step S301 as shown below.

[0072] Step S301: performing human key point detection on the target image data through a preset adaptive detection model, and adaptively adjusting the feature extraction weights for performing human key point detection on the target image data based on the sensor parameters through the adaptive detection model.

[0073] When the terminal device performs human key point detection on the target image data through the adaptive detection model, the target image data and the sensor parameters of the visual sensor are used as the model input of the adaptive detection model. Therefore, when the terminal device performs human key point detection on the target image data through the adaptive detection model, the feature extraction weights for human key point detection on the target image data are adaptively adjusted based on the sensor parameters.

[0074] In some embodiments, the terminal device may also pre-input the sensor parameters of the visual sensor into the adaptive detection model, so that the adaptive detection model can adaptively adjust the feature extraction weights for detecting human key points on the target image data based on the sensor parameters, thereby obtaining a new detection model with the adjusted weights. Subsequently, when the terminal device performs human key point detection on the target image data input by the visual sensor, it can use the target image data as the model input of the new detection model, thereby performing human key point detection on the target image data using the new detection model.

[0075] Please refer to Figure 4 , Figure 4 for Figure 3 Schematic diagram of the detailed process flow of step S301.

[0076] like Figure 4 As shown, in some embodiments, the step of "adaptively adjusting the feature extraction weights for human key point detection on the target image data based on the sensor parameters through the adaptive detection model" in the above-mentioned step S301 may include but is not limited to the step S401 shown below.

[0077] Step S401: generating a scaling factor and a bias term of a convolution block based on the sensor parameters through the adaptive detection model, so as to adaptively adjust a feature extraction weight for performing human key point detection on the target image data based on the scaling factor and the bias term.

[0078] It should be noted that the convolution block can be any one or more layers of convolution blocks in the multi-layer convolution blocks that constitute the adaptive detection model.

[0079] When the terminal device adaptively adjusts the feature extraction weights for human key point detection in the target image data based on the sensor parameters through the adaptive detection model, the sensor parameters can be incorporated into the model input, thereby dynamically adjusting the scaling factor γ and bias β of the convolution block based on the sensor parameters through the adaptive detection model, thereby achieving adaptive adjustment of the feature extraction weights.

[0080] For example, when a visual sensor (such as a camera) is mounted at a high height on a vehicle's tailgate (e.g., an SUV), its viewing angle is wide and the proportion of human figures in the image is relatively small. Therefore, the terminal device can incorporate the camera's mounting height parameter into the model input of the adaptive detection model. The adaptive detection model then adjusts the scaling factor γ of the convolutional block based on the input mounting height parameter, thereby increasing the weight of the extracted key point features of small-sized human figures, thereby enhancing the feature extraction capability of these key points. Furthermore, for cameras with different distortion coefficients, the terminal device can incorporate the camera's distortion coefficient into the model input of the adaptive detection model. The adaptive detection model then adjusts the bias β based on the input distortion coefficient, thereby increasing the weight of the extracted key point features to counteract lens distortion and correct for key point position deviations caused by lens distortion.

[0081] In this embodiment, the sensor parameters of the visual sensor (such as installation height, viewing angle, internal parameter matrix, distortion coefficient and other parameters) are used as input conditions of the model for detecting human key points in the target image through the terminal device, and the scaling factor γ and bias β of the convolution block are dynamically generated within the model, so that the feature extraction weights during human key point detection can be adaptively adjusted according to the sensor parameters, thereby achieving the adaptation of the same detection algorithm to visual sensors with different installation heights and viewing angle ranges, thereby eliminating the need to develop a separate algorithm for each vehicle model, and solving the technical bottleneck of cross-vehicle platform deployment.

[0082] Furthermore, this embodiment optimizes keypoint detection in image data under various installation environments through dynamically adjusted feature extraction weights. That is, regardless of the vehicle type or location of the visual sensor, the adaptive detection model adaptively adjusts its internal parameters to ensure accurate keypoint detection of the human body. This provides reliable basic data for subsequent kick motion recognition, further enhancing the overall stability and adaptability of kick-sensing tailgate control.

[0083] Furthermore, this embodiment, based on a model design based on adaptive sensor parameters, enhances algorithm versatility. This allows for more convenient integration of the kick-sensing tailgate control technology into production lines for different vehicle models during automotive manufacturing, reducing the workload associated with technical adaptation due to vehicle model differences. This, in turn, facilitates the rapid rollout of this intelligent configuration by automotive companies, enhancing the intelligent competitiveness of their products. Furthermore, because the adaptive detection model can adaptively adjust internal computational parameters (i.e., feature extraction weights) based on the hardware characteristics of different visual sensors (one of the sensor parameters incorporated into the model input, such as resolution, frame rate, and distortion), it enables efficient feature extraction and action recognition across vehicle platforms with varying hardware configurations. This avoids performance fluctuations caused by hardware differences and ensures stable functional operation. This embodiment also improves the adaptability of hardware resources.

[0084] In some embodiments, the human key point data obtained by the terminal device through human key point detection on the target image data may be 2D planar human key point data. The terminal device may employ a spatial intelligent mapping mechanism, using spatial perception technology to map the human key points detected in the 2D plane into 3D space, constructing a 3D human key point sequence with time-series information, and then analyzing and identifying human movements in the 3D space to accurately determine kicking behavior.

[0085] Please refer to Figure 5 , Figure 5 for Figure 1 Schematic diagram of the detailed process flow of step S103.

[0086] like Figure 5 As shown, in some embodiments, the step of "performing kicking behavior recognition based on the human body key point data to obtain a recognition result" in the above-mentioned step S103 may include steps S501 and S502 as shown below.

[0087] Step S501: performing spatial mapping processing on the planar human body key point data to obtain spatial human body key point data.

[0088] When the terminal device performs kicking behavior recognition on the human body key point data, it can first perform 3D spatial mapping processing on the 2D planar human body key point data to obtain 3D spatial human body key point data.

[0089] In some embodiments, the terminal device can use RNN to model the time-series human body key point information to map 2D planar human body key point data to 3D space, thereby obtaining 3D spatial human body key point data.

[0090] In other embodiments, in order to further improve efficiency and accuracy, the terminal device may also use an extended time convolution model to process a 2D key point sequence for spatial mapping. The core design of each layer is as follows: .

[0091] in, Indicates the current layer At time step Output; is an activation function used to increase the nonlinear representation ability of the feature vector; is batch normalization; is the convolution kernel size, Indicates that the convolution kernel is at position The weight of Represents the input of the previous layer, indexed by , is the expansion factor; is the bias term.

[0092] Based on this, the above-mentioned step S501: performing spatial mapping processing on the planar human body key point data to obtain spatial human body key point data may include the following steps: The planar human body key point data is subjected to multi-scale temporal feature fusion processing by a preset dilated time convolution model to obtain spatial human body key point data output by the dilated time convolution model.

[0093] When a terminal device uses a dilated temporal convolution model to process a 2D keypoint sequence for spatial mapping, it inputs 2D planar human keypoint data into the dilated temporal convolution model. The model then performs multi-scale temporal feature fusion processing on the planar human keypoint data, mapping the 2D keypoint sequence in the planar human keypoint data into a 3D coordinate sequence, thereby obtaining and outputting 3D spatial human keypoint data. In this way, the terminal device can obtain the spatial human keypoint data.

[0094] For example, it is assumed that the 2D key point coordinate sequence in the planar human body key point data is ,in, express time Key points of the human body The terminal device performs multi-scale temporal feature fusion processing on the 2D key point coordinate sequence through the dilated time convolution model, and the obtained 3D coordinate sequence is .

[0095] Step S502: performing kicking behavior recognition based on the spatial human body key point data to obtain a recognition result.

[0096] After the terminal device maps the 2D human body key point data into 3D spatial data, it can then process the kicking behavior based on this data to obtain the corresponding recognition result. In this way, the terminal device can control the vehicle tailgate accordingly based on the recognition result.

[0097] In some embodiments, the terminal device can extract the coordinates of key points such as the user's ankles, knees, and hip joints from the 3D spatial human body key point data, and analyze the spatial position changes, motion trajectories, and timing characteristics of these coordinates to determine whether kicking behavior is recognized.

[0098] In this embodiment, a terminal device uses 3D spatial mapping to map 2D human key point data into 3D space, generating 3D spatial human key point data. Specifically, by converting a sequence of 2D human key points (e.g., the coordinates of 17 key points in the image, such as the ankle, knee, and hip joints) with time-series information into 3D spatial coordinates, the true spatial form of the kicking action can be restored and its changing trends analyzed. Compared to traditional vision-based kick recognition methods (which typically use 2D images as input, but 2D images only represent the projection of the action, losing key information such as depth and spatial trajectory), this embodiment effectively captures the spatial motion of the kicking action by mapping 2D key points into 3D space, thereby improving the robustness of action recognition. For example, due to physical limitations, the amplitude of a pregnant woman's kicking motion may not be apparent in a 2D image. However, through 3D spatial mapping, the terminal device can accurately capture the three-dimensional characteristics of the kicking action, such as its trajectory and displacement distance. In this way, it can not only solve the problem of difficulty in identifying kicking actions caused by special groups or objects partially blocking the human body, but also mine the three-dimensional characteristics of human movements through 3D spatial mapping, so as to have more accurate recognition capabilities for small movements and unconventional movements such as side kicks and multi-angle kicks, thereby greatly reducing the trigger failure rate caused by non-standard movements and covering a wider range of user groups and action scenarios.

[0099] Furthermore, while this embodiment incorporates 3D spatial mapping in the foot motion recognition process, it still efficiently implements the overall algorithmic process for kick-sensing tailgate control through a rational model architecture design that dynamically adjusts convolutional block parameters to adaptively adjust feature extraction weights. Compared to computationally complex processes such as the DTW algorithm in related technologies, this embodiment optimizes the computational process, reduces computational complexity, and improves the real-time performance of kick motion recognition while maintaining recognition accuracy. This enables rapid detection of kicks and timely tailgate control, enhancing the user's experience of smooth and immediate foot-sensing tailgate control.

[0100] In some embodiments, the terminal device can also adopt a kicking recognition mechanism based on spatiotemporal feature decoding, and analyze and identify human body movements in 3D space based on spatiotemporal feature coding and behavioral action decoding modules, so as to achieve the purpose of accurately judging kicking behavior.

[0101] Please refer to Figure 6 , Figure 6 for Figure 5 Schematic diagram of the detailed steps of step S502.

[0102] like Figure 6 As shown, in some embodiments, the above-mentioned step S502: performing kicking behavior recognition based on the spatial human body key point data to obtain a recognition result may include steps S601 to S603 as shown below.

[0103] Step S601: extracting spatial foot key point data from the spatial human body key point data.

[0104] It should be noted that the spatial foot key point data may be a 3D key point sequence only related to foot movements in the spatial human body key point data.

[0105] When the terminal device identifies kicking behavior based on spatial human key point data based on spatiotemporal feature decoding, in order to further improve efficiency or ensure that the overall efficiency is not affected, the spatial human key point data can be first processed with spatial constraints, thereby extracting spatial foot key point data from the spatial human key point data.

[0106] For example, the terminal device can filter out irrelevant objects by setting a detection area (such as a 1.5-meter-by-1-meter cubic space under the rear of a vehicle) for spatial human key point data, thereby retaining only the 3D key point sequence related to foot movements in the spatial human key point data (spatial foot key point data).

[0107] Step S602: performing spatiotemporal feature encoding processing on the spatial foot key point data to obtain a feature vector corresponding to the spatial foot key point data.

[0108] After extracting the spatial foot key point data from the spatial human body key point data, the terminal device can perform spatiotemporal feature encoding processing on the spatial foot key point data to obtain a feature vector corresponding to the spatial foot key point data.

[0109] In some embodiments, the terminal device may use a long short-term memory (LSTM) network to perform spatiotemporal feature encoding on a 3D key point sequence in the spatial foot key point data, thereby generating a feature vector corresponding to the spatial foot key point data.

[0110] For example, it is assumed that the key point data of the space foot is The terminal device uses the long short-term memory network LSTM to encode the 3D key point sequence, and can generate a feature vector corresponding to the key point data of the foot in the space. .

[0111] Step S603: performing action decoding processing on the feature vector to obtain a recognition result of the kicking behavior.

[0112] After the terminal device performs spatiotemporal feature encoding processing on the spatial foot key point data, the terminal device further performs behavioral action decoding processing on the feature vector obtained by encoding, so as to determine whether the foot action of the user corresponding to the spatial foot key point data is a kicking action, and uses the judgment result as the recognition result of the current kicking behavior recognition.

[0113] Please refer to Figure 7 , Figure 7 for Figure 6 Schematic diagram of the detailed steps of step S603.

[0114] like Figure 7 As shown, in some embodiments, the above step S603: performing behavioral action decoding processing on the feature vector to obtain the recognition result of the kicking behavior may include steps S701 to S703 as shown below.

[0115] Step S701: performing behavioral action decoding processing on the feature vector through a preset classifier to obtain the predicted probability of the kicking behavior output by the classifier.

[0116] It should be noted that the classifier can be a machine learning classifier, a deep learning classifier, or an integrated or hybrid classifier. It should be understood that based on the different design requirements of actual applications, in different implementations, the terminal device can use different classifiers to decode the behavior action of the feature vector. The vehicle tailgate control method provided in the embodiment of the present application does not limit the type of classifier.

[0117] The terminal device can decode the feature vector obtained by encoding the spatiotemporal features through a classifier to perform behavioral action decoding on the feature vector, so as to judge the kicking behavior through the classifier and obtain the predicted probability of the kicking behavior output by the classifier.

[0118] For example, the terminal device can use a fully connected neural network to analyze the feature vector Decode the action to determine whether it is a kicking action. The decision function of the fully connected neural network can be: .

[0119] Where p is the predicted probability, is the Sigmoid function.

[0120] Step S702: When the predicted probability is greater than or equal to a preset probability threshold, determining the recognition result of the kicking behavior as a kicking behavior being recognized.

[0121] It should be noted that the preset probability threshold can be a classifier prediction probability threshold pre-configured in the terminal device for determining whether the classifier has decoded a kicking behavior. It should be understood that based on the different design requirements of actual applications, terminal devices in different implementations may use different threshold values. The vehicle tailgate control method provided in the embodiments of this application does not limit the specific value of the preset probability threshold.

[0122] After obtaining the predicted probability of the kicking behavior output by the classifier, the terminal device compares the preset probability with the preset probability threshold. If the comparison finds that the predicted probability is greater than or equal to the preset probability threshold, the terminal device determines that the classifier has decoded the kicking behavior, and thus determines that the recognition result of the kicking behavior is that the kicking behavior is recognized.

[0123] Step S703: When the predicted probability is less than a preset probability threshold, determining that the recognition result of the kicking behavior is that the kicking behavior is not recognized.

[0124] After the terminal device compares the predicted probability of the kicking behavior output by the classifier with the preset probability threshold, if the comparison finds that the predicted probability is less than the preset probability threshold, the terminal device determines that the classifier has not decoded the kicking behavior, and thus determines that the recognition result of the kicking behavior is that the kicking behavior was not recognized.

[0125] For example, the terminal device adopts the decision function through the classifier (fully connected neural network) After decoding the above feature vector V into action and obtaining the predicted probability p, if the predicted probability Preset probability threshold , the terminal device determines in this case that the user foot action corresponding to the above-mentioned spatial foot key point data is a kicking action, and if the predicted probability <Preset probability threshold , the terminal device determines that the user foot movement corresponding to the spatial foot key point data is not a kicking movement.

[0126] In this embodiment, a terminal device uses spatiotemporal feature coding and a behavioral action decoding module to analyze and identify 3D spatial human key point data for human action analysis and recognition. This allows the spatiotemporal feature coding and behavioral action decoding modules to simultaneously consider the spatial features (the position and posture of the 3D key points) and temporal features (how the key point sequence changes over time) of the action. Thus, compared to related techniques that rely solely on the optical flow and shape features of two-dimensional images to identify kicking behaviors, this embodiment can more comprehensively and deeply analyze the essence of kicking actions. For example, for non-standard actions (such as kicks with a small angle or slow speed), this embodiment can extract features such as the temporal dimension of the action and the spatial dimension of the movement trend through spatiotemporal feature coding. After behavioral action decoding, it can accurately identify the action as a kicking action, thereby reducing the user's learning cost and improving the robustness of kicking behavior recognition.

[0127] Furthermore, this embodiment uses spatiotemporal feature encoding and action decoding to identify kicking behaviors, fully utilizing both the temporal sequence and spatial morphological information of the action. This allows for stable and accurate identification of kicking actions even in complex scenarios, such as occlusion and complex backgrounds. For example, in scenarios where a handheld object obstructs the leg, this embodiment uses the motion features and spatiotemporal correlation information of the unobstructed area in 3D space to identify the kicking action. Furthermore, in scenarios involving multiple people, this embodiment can distinguish target human actions based on both 3D spatial position and spatiotemporal sequence.

[0128] Next, a complete embodiment of the vehicle tailgate control method provided in the embodiment of the present application is proposed.

[0129] Please refer to Figure 8 , Figure 8 A schematic flow chart of a complete embodiment of the vehicle tailgate control method provided in an embodiment of the present application.

[0130] like Figure 8As shown, when the terminal device applies the vehicle tailgate control method provided by the embodiment of the present application, it can first pre-process the video data (target image data) input by the camera (visual sensor), such as performing image noise reduction on the video data to eliminate the impact of environmental noise on image quality. At the same time, the terminal device also performs camera parameter recognition processing, that is, obtains installation parameters through camera calibration, including the installation height H of the trunk rearview camera, the viewing angle, and the camera angle. , camera intrinsic parameter matrix K, distortion coefficient k.

[0131] Afterwards, the terminal device performs kicking behavior detection on the pre-processed video data.

[0132] In this process, the terminal device first performs 2D key point detection and tracking processing based on camera parameter adaptation, that is, the terminal device first converts the camera parameter vector And the preprocessed image Input to the 2D keypoint detection model (adaptive detection model), which generates the scaling factor of the convolution block through the adaptive module and bias , and based on this model, 17 key points of the human body (such as ankles, knees, hip joints, etc.) are detected, and the ID tracking algorithm is used to establish the temporal association of key points across frames to obtain 2D planar human key point data: .

[0133] After the terminal device detects the kicking behavior and obtains the 2D human body key point data, it further performs 3D spatial mapping of the key points and constrains the region of interest. That is, the terminal device performs 3D spatial mapping of the human body key point data based on the dilated time convolution model to obtain a 3D key point coordinate sequence. , thereby obtaining 3D spatial human key point data. Furthermore, the terminal device filters out irrelevant objects by setting a detection area (such as a 1.5m x 1m cube under the rear of the vehicle), retaining only the 3D key point sequence related to the foot movements, thereby extracting spatial foot key point data.

[0134] Finally, the terminal device determines whether a kicking action is currently recognized by encoding spatiotemporal features and decoding behavioral actions. That is, the terminal device uses the long short-term memory network LSTM to encode the 3D key point sequence in the spatial foot key point data to generate a feature vector Afterwards, the terminal device uses a classifier (such as a fully connected neural network) to classify the feature vector Decode to determine whether a kicking action is recognized.

[0135] See also Figure 9, an embodiment of the present application also provides a vehicle tailgate control device, which can implement the above-mentioned vehicle tailgate control method.

[0136] like Figure 9 As shown, the vehicle tailgate control device provided by the embodiment of the present application includes a data acquisition module 901, an adaptive detection module 902 and a control module 903. The data acquisition module 901 is used to acquire target image data; the target image data is acquired by a visual sensor associated with the vehicle tailgate; An adaptive detection module 902 is configured to perform human key point detection on the target image data based on sensor parameters of the visual sensor to obtain human key point data in the target image data; the sensor parameters are used to adaptively adjust feature extraction weights for performing human key point detection on the target image data; The control module 903 is configured to perform kicking behavior recognition based on the human body key point data to obtain a recognition result, and control the vehicle tailgate based on the recognition result.

[0137] In some embodiments, the adaptive detection module 902 is also used to adaptively adjust the preset initial feature extraction weights based on the sensor parameters of the visual sensor to obtain the adjusted target feature extraction weights; the initial feature extraction weights are the initial feature extraction weights for performing human key point detection on the target image data; and, performing human key point detection on the target image data based on the target feature extraction weights.

[0138] In some embodiments, the adaptive detection module 902 is also used to perform human key point detection on the target image data through a preset adaptive detection model, and adaptively adjust the feature extraction weights for human key point detection on the target image data based on the sensor parameters through the adaptive detection model.

[0139] In some embodiments, the adaptive detection module 902 is further used to generate a scaling factor and a bias term of a convolution block based on the sensor parameters through the adaptive detection model, so as to adaptively adjust the feature extraction weights for human key point detection of the target image data based on the scaling factor and the bias term; wherein the convolution block is any convolution block in the multi-layer convolution block of the adaptive detection model.

[0140] In some embodiments, the human body key point data includes planar human body key point data, and the control module 903 is also used to perform spatial mapping processing on the planar human body key point data to obtain spatial human body key point data; and to perform kicking behavior recognition based on the spatial human body key point data to obtain recognition results.

[0141] In some embodiments, the control module 903 is further used to perform multi-scale temporal feature fusion processing on the planar human key point data through a preset dilated time convolution model to obtain spatial human key point data output by the dilated time convolution model.

[0142] In some embodiments, the control module 903 is also used to extract spatial foot key point data from the spatial human body key point data; perform spatiotemporal feature encoding processing on the spatial foot key point data to obtain a feature vector corresponding to the spatial foot key point data; and perform behavioral action decoding processing on the feature vector to obtain a recognition result of the kicking behavior.

[0143] In some embodiments, the control module 903 is further used to perform behavioral action decoding processing on the feature vector through a preset classifier to obtain the predicted probability of the kicking behavior output by the classifier; when the predicted probability is greater than or equal to the preset probability threshold, the recognition result of the kicking behavior is determined to be the kicking behavior being recognized; and when the predicted probability is less than the preset probability threshold, the recognition result of the kicking behavior is determined to be the kicking behavior not being recognized.

[0144] It should be noted that the specific implementation of the vehicle tailgate control device provided in the embodiment of the present application is basically the same as the specific implementation of the above-mentioned vehicle tailgate control method, and will not be repeated here.

[0145] See also Figure 10 An embodiment of the present application also provides a vehicle tailgate control device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned visual language model evaluation method when executing the computer program.

[0146] In some embodiments, the vehicle tailgate control device can be any smart terminal such as a vehicle-mounted hardware platform (such as a vehicle-mounted computer, etc.), a tablet computer, a smart phone, a wearable device, etc.

[0147] like Figure 10 As shown, the vehicle tailgate control device provided in the embodiment of the present application may include: The processor 1001 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application; The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called by the processor 1001 to execute the vehicle tailgate control method of the embodiments of this application. Input / output interface 1003, used to implement information input and output; Communication interface 1004, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.); Bus 1005 , which transmits information between various components of the device (e.g., processor 1001 , memory 1002 , input / output interface 1003 , and communication interface 1004 ); The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0148] An embodiment of the present application also provides a vehicle, which includes the above-mentioned vehicle tailgate control device, and the vehicle tailgate control device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned vehicle tailgate control method when executing the computer program.

[0149] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned vehicle tailgate control method is implemented.

[0150] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0151] An embodiment of the present application also provides a computer program product, including a computer program. The steps implemented when the computer program is executed by a processor are basically the same as the specific embodiments of the above-mentioned vehicle tailgate control method, and will not be repeated here.

[0152] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0153] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0154] The system embodiment described above is merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0155] Those skilled in the art will appreciate that all or some of the steps, functional modules / units in the apparatus / system, and device disclosed above may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0156] The terms "first," "second," "third," "fourth," and the like (if any) in the specification of the present application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, apparatus / system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to such processes, methods, products, or apparatus.

[0157] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed devices / systems and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0159] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0160] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0161] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0162] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A vehicle tailgate control method, characterized in that: The method comprises: Acquire target image data; the target image data is collected by a visual sensor associated with the vehicle tailgate; Performing human key point detection on the target image data based on sensor parameters of the visual sensor to obtain human key point data in the target image data; the sensor parameters are used to adaptively adjust feature extraction weights for performing human key point detection on the target image data; A kicking behavior is recognized based on the human body key point data to obtain a recognition result, and the vehicle tailgate is controlled based on the recognition result.

2. The method according to claim 1, characterized in that The performing human body key point detection on the target image data based on the sensor parameters of the visual sensor includes: Adaptively adjusting a preset initial feature extraction weight based on sensor parameters of the visual sensor to obtain an adjusted target feature extraction weight; the initial feature extraction weight is an initial feature extraction weight for performing human key point detection on the target image data; Human body key point detection is performed on the target image data based on the target feature extraction weights.

3. The method according to claim 1, characterized in that The performing human body key point detection on the target image data based on the sensor parameters of the visual sensor includes: The target image data is subjected to human key point detection by a preset adaptive detection model, and the feature extraction weights for human key point detection on the target image data are adaptively adjusted based on the sensor parameters by the adaptive detection model.

4. The method according to claim 3, characterized in that Adaptively adjusting the feature extraction weights for performing human key point detection on the target image data based on the sensor parameters by the adaptive detection model includes: Generating a scaling factor and a bias term of a convolution block based on the sensor parameters by the adaptive detection model, so as to adaptively adjust a feature extraction weight for performing human key point detection on the target image data based on the scaling factor and the bias term; The convolution block is any convolution block in the multi-layer convolution block of the adaptive detection model.

5. The method according to claim 1, characterized in that The human body key point data includes planar human body key point data, and the kicking behavior recognition based on the human body key point data obtains a recognition result, including: Performing spatial mapping processing on the planar human body key point data to obtain spatial human body key point data; The kicking behavior is recognized based on the spatial human body key point data to obtain a recognition result.

6. The method according to claim 5, characterized in that The performing spatial mapping processing on the planar human body key point data to obtain spatial human body key point data includes: The planar human body key point data is subjected to multi-scale temporal feature fusion processing by a preset dilated time convolution model to obtain spatial human body key point data output by the dilated time convolution model.

7. The method according to claim 5, characterized in that The kicking behavior recognition based on the spatial human body key point data obtains a recognition result, including: Extracting spatial foot key point data from the spatial human body key point data; Performing spatiotemporal feature encoding processing on the spatial foot key point data to obtain a feature vector corresponding to the spatial foot key point data; The feature vector is subjected to behavioral action decoding processing to obtain a recognition result of the kicking behavior.

8. The method according to claim 7, characterized in that The step of performing behavioral action decoding processing on the feature vector to obtain a kicking behavior recognition result includes: Performing behavioral action decoding processing on the feature vector using a preset classifier to obtain a predicted probability of the kicking behavior output by the classifier; When the predicted probability is greater than or equal to the preset probability threshold, determining the recognition result of the kicking behavior as a kicking behavior being recognized; When the predicted probability is less than the preset probability threshold, the recognition result of the kicking behavior is determined to be that the kicking behavior is not recognized.

9. A vehicle tailgate control device, characterized in that: The device comprises: A data acquisition module is used to acquire target image data; the target image data is acquired by a visual sensor associated with the vehicle tailgate; An adaptive detection module is configured to perform human key point detection on the target image data based on sensor parameters of the visual sensor to obtain human key point data in the target image data; the sensor parameters are used to adaptively adjust feature extraction weights for performing human key point detection on the target image data; A control module is used to identify the kicking behavior based on the human body key point data to obtain an identification result, and to control the vehicle tailgate based on the identification result.

10. A vehicle tailgate control device, characterized in that: The vehicle tailgate control device includes a memory and a processor, the memory stores a computer program, and the processor implements the vehicle tailgate control method according to any one of claims 1 to 8 when executing the computer program.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the vehicle tailgate control method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, the vehicle tailgate control method according to any one of claims 1 to 8 is implemented.

Images