Door control method, device, equipment and storage medium

Abstract

The present disclosure provides a vehicle door control method, apparatus, device and storage medium, which collect sensor data of a vehicle door control sensor, wherein the sensor data includes at least one of a distance measurement value, a signal strength value and an ambient noise value; perform vehicle door control action recognition based on the sensor data to obtain a first recognition result of whether the vehicle door control action is detected; determine whether the sensor data meets a corresponding preset threshold standard and whether the duration of meeting the corresponding preset threshold standard is within a preset duration range, and obtain a second recognition result of whether the vehicle door control action exists; obtain a fused recognition result of whether the vehicle door control action exists based on the first recognition result and the second recognition result; and send a vehicle door control signal to a vehicle door controller in response to the fused recognition result of the existence of the vehicle door control action, thereby solving, to a certain extent, the problems of low action recognition rate and false triggering of the door control sensor in actual applications.

Description

Technical Field

[0001] The present disclosure relates to control technology, and in particular to a vehicle door control method, device, equipment and storage medium. Background Art

[0002] In recent years, with the rapid development of intelligent automotive technology, in-vehicle intelligent sensing products have become increasingly popular. Traditional tailgate switches, which include a key button and a button on the outside of the tailgate, are effective, but inconvenient in some scenarios. For example, when a user's hands are full, they cannot touch the button or switch. To address this issue, a new light-sensitive foot sensor has emerged.

[0003] The light-sensing foot sensor uses a laser time-of-flight (TOF) sensor to realize the sensing function. Its main working principle is to project a light spot on the ground to indicate the position of the user's foot. When an object is detected stepping into the range of the light spot, the distance value collected by the TOF sensor is used to identify whether there is a foot movement. If so, the car door is controlled to open.

[0004] However, while current optical foot sensors offer convenience, they still suffer from low recognition rates and false triggering in practical applications. For example, when the material of the user's shoe and the floor differ significantly, the TOF sensor's measurement results may be inaccurate due to the impact on the laser's reflectivity, making it unable to accurately identify the foot movement. Alternatively, the TOF sensor may mistakenly identify non-foot movement as a foot movement. For example, if a small animal passes through the light spot area or leaves fall into the light spot area, this may cause the car door to be falsely triggered to open. These issues limit the application of optical foot sensors. Summary of the Invention

[0005] The embodiments of the present disclosure provide a vehicle door control method, apparatus, device, and storage medium, which can, to a certain extent, solve the problems of low motion recognition rate and false triggering of vehicle door control sensors in actual applications.

[0006] In one aspect of an embodiment of the present disclosure, a vehicle door control method is provided, which is applied to a vehicle door control sensor. The method includes:

[0007] collecting sensor data from the door control sensor, the sensor data including at least one of a distance measurement value, a signal strength value, and an ambient noise value; wherein the distance measurement value indicates a distance between the door control sensor and a signal reflection surface, the signal reflection surface being a surface of an object that reflects a detection signal from the door control sensor;

[0008] performing door control action recognition based on the sensor data to obtain a first recognition result of whether a door control action exists;

[0009] determining whether the sensor data satisfies a corresponding preset threshold standard and whether a duration of satisfying the corresponding preset threshold standard is within a preset duration range, and obtaining a second recognition result of whether a door control action exists;

[0010] Obtaining a fusion recognition result of whether a door control action exists based on the first recognition result and the second recognition result;

[0011] In response to the fusion recognition result that there is a door control action, a door control signal is sent to a door controller, so that the door controller controls the door to be opened or closed based on the door control signal and the current state of the door.

[0012] Optionally, collecting sensor data includes:

[0013] transmitting the detection signal in a preset direction at a preset frequency and receiving a reflected signal of the detection signal;

[0014] determining the distance measurement value based on a flight time between the detection signal and the reflected signal, wherein the flight time refers to the time from when the detection signal is transmitted to when the sensor receives the reflected signal;

[0015] detecting a signal strength value of the received reflected signal;

[0016] The ambient noise value of the surrounding environment during a period in which the reflected signal is not received is detected.

[0017] Optionally, the first recognition result includes a first confidence level and a first classification result for indicating whether the door control action occurs within the preset time period, and the second recognition result includes a second confidence level and a second classification result for indicating whether the door control action occurs within the preset time period;

[0018] The obtaining, based on the first recognition result and the second recognition result, of a fusion recognition result of whether a door control action exists includes:

[0019] The first confidence level is used as the weight of the first classification result, and the second confidence level is used as the weight of the second classification result, and a weighted sum calculation is performed to obtain the fusion recognition result.

[0020] Optionally, performing door control action recognition based on the sensor data to obtain a first recognition result of whether a door control action exists includes:

[0021] Using the input layer in the deep learning model to input the sensor data sequence collected within the preset time period;

[0022] Performing feature extraction on the sensor data sequence using a convolutional layer in a deep learning model to obtain a feature matrix of the sensor data sequence;

[0023] Performing a nonlinear transformation on the feature matrix of the sensor data sequence using an activation function in a deep learning model;

[0024] Using a fully connected layer in a deep learning model to classify the feature matrix of the sensor data sequence after nonlinear transformation;

[0025] The output layer in the deep learning model is used to output the first confidence level and the first classification result.

[0026] Optionally, determining whether the sensor data satisfies a corresponding preset threshold standard and whether a duration of satisfying the corresponding preset threshold standard is within a preset duration range, to obtain a second recognition result of whether a door control action exists, includes:

[0027] Collecting the sensor data using a sliding window, and determining whether the current sliding window is in a stable state based on a range within the current sliding window, wherein the range refers to a difference between a maximum value of the sensor data and a minimum value of the sensor data, and a sliding window in the stable state is a sliding window in which the range is less than a range threshold;

[0028] In response to a previous sliding window being in an unstable state and the current sliding window being in the stable state, determining whether sensor data within the current sliding window meets the preset threshold standard;

[0029] Determining a duration of time during which the sensor data is in the stable state and meets the preset threshold criterion based on the number of sliding windows that are continuously in the stable state and meet the preset threshold criterion starting from the current sliding window;

[0030] In response to the duration being within the preset duration range, obtaining a second classification result of detecting the door control action;

[0031] The second confidence level is calculated based on the distance threshold interval, the signal strength threshold interval, the noise value threshold interval, and the value of the sensor data.

[0032] Optionally, determining whether the current sliding window is in a stable state based on the range in the sliding window includes:

[0033] Determining a signal-to-noise ratio corresponding to the sliding window based on a mean signal strength value and a mean ambient noise value within the current sliding window; determining a range threshold corresponding to the current sliding window based on the signal-to-noise ratio, wherein the range threshold is negatively correlated with the signal-to-noise ratio;

[0034] In response to the range being less than the range threshold, determining that the current sliding window is in the stable state;

[0035] In response to the range being greater than or equal to the range threshold, it is determined that the current sliding window is in the unstable state.

[0036] Optionally, the second confidence level is calculated as follows:

[0037] Second confidence level = (threshold interval endpoint difference - 2*|sensor data value - threshold interval median|) / threshold interval endpoint difference

[0038] The threshold interval includes at least one of the distance threshold interval, the signal strength threshold interval and the noise value threshold interval; the threshold interval endpoint difference is the absolute value of the difference between the values ​​corresponding to the two endpoints of the threshold interval; and the threshold interval median is the value corresponding to the middle point of the threshold interval.

[0039] Another aspect of the present disclosure provides a vehicle door control device, which is applied to a vehicle door control sensor. The device includes:

[0040] an acquisition module configured to acquire sensor data from the door control sensor, the sensor data including at least one of a distance measurement value, a signal strength value, and an ambient noise value; wherein the distance measurement value indicates the distance between the door control sensor and a signal reflection surface, the signal reflection surface being a surface of an object that reflects a detection signal from the sensor;

[0041] a first recognition module, configured to perform door control action recognition based on the sensor data, and obtain a first recognition result of whether a door control action exists;

[0042] a second recognition module, configured to determine whether the sensor data satisfies a corresponding preset threshold standard and whether a duration of satisfying the corresponding preset threshold standard is within a preset duration range, and obtain a second recognition result of whether a door control action exists;

[0043] a fusion module, configured to obtain a fusion recognition result of whether a door control action exists based on the first recognition result and the second recognition result;

[0044] The control module is configured to send a door control signal to a door controller in response to a fusion recognition result indicating the presence of a door control action, so that the door controller controls the door to be opened or closed based on the door control signal and the current state of the door.

[0045] Another aspect of the present disclosure provides an electronic device, including:

[0046] memory for storing computer programs;

[0047] The processor is configured to execute the computer program stored in the memory, and when the computer program is executed, the method described in the above aspects is implemented.

[0048] Another aspect of the embodiments of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described in the above aspects is implemented.

[0049] Another aspect of the embodiments of the present disclosure provides a computer program, comprising computer program instructions, which implement the method described in the above aspects when executed by a processor.

[0050] Based on the embodiment of the present disclosure, by real-time collection of at least one sensor data including distance measurement values, signal strength values ​​and environmental noise values, door control actions are identified based on the sensor data, taking into account that the user's door control actions will not only affect the distance measurement values ​​but also affect the signal strength of the reflected signal and the intensity of the environmental noise; secondly, on the one hand, door control actions are identified based on the sensor data to obtain a first recognition result, and on the other hand, based on a preset threshold standard and time continuity, it is determined whether there is sensor data corresponding to the door control action to obtain a second recognition result. The results of at least two recognition methods are combined to obtain a fused recognition result of the door control action, which can improve the reliability of the door control action recognition results and, to a certain extent, solve the problems of low action recognition rate and false triggering of door control sensors in actual applications, thereby improving the accuracy of door control.

[0051] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0053] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0054] Figure 1 This is a flow chart of an embodiment of the vehicle door control method disclosed herein;

[0055] Figure 2 A schematic diagram of how the detection data values ​​of a preset door opening action change over time in an embodiment of the present disclosure;

[0056] Figure 3 A schematic diagram of a model structure of a deep learning model in an embodiment of the present disclosure;

[0057] Figure 4 A schematic structural diagram of a door control sensor according to an embodiment of the present disclosure;

[0058] Figure 5 A histogram generated by a photodetector in an embodiment of the present disclosure;

[0059] Figure 6 is a flow chart of a process for generating a second classification result in an embodiment of the present disclosure;

[0060] Figure 7 A schematic diagram of the state change of sensor data in an embodiment of the present disclosure;

[0061] Figure 8 This is a schematic structural diagram of an embodiment of a vehicle door control device disclosed herein;

[0062] Figure 9 The figure is a schematic structural diagram of an application embodiment of the electronic device disclosed herein. DETAILED DESCRIPTION

[0063] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0064] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, and do not represent any specific technical meanings, nor do they indicate a necessary logical order between them.

[0065] It should also be understood that in the embodiments of the present disclosure, “a plurality of” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0066] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0067] In addition, the term "and / or" in this disclosure is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this disclosure generally indicates that the related objects are in an "or" relationship.

[0068] It should also be understood that the description of the various embodiments in this disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced with each other. For the sake of brevity, they will not be described one by one.

[0069] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0070] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0071] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0072] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0073] The embodiments of the present disclosure can be applied to a vehicle door control sensor, which includes a main control module and a sensor module, wherein the main control module is connected to the sensor module. The sensor module is used to collect sensor data and send the sensor data to the main control module; the main control module is used to perform action recognition based on the sensor data. The main control module is also connected to the vehicle's door controller and is used to send a door opening signal to the door controller when a preset door opening action is detected. The door controller controls the door to open in response to the door opening signal. Schematically, the door control sensor can adopt a capacitive sensor, an ultrasonic sensor, an infrared sensor, a lidar sensor, etc. The present disclosure uses a TOF sensor as an example for explanation.

[0074] Please refer to Figure 1 , which shows a flow chart of a vehicle door control method provided by an exemplary embodiment of the present disclosure. The vehicle door control method of the embodiment of the present disclosure can be implemented by a vehicle door control sensor. The embodiment includes the following steps:

[0075] Step 101 : collecting sensor data of a door control sensor, where the sensor data includes at least one of a distance measurement value, a signal strength value, and an ambient noise value.

[0076] The distance measurement value indicates the distance between the door control sensor and the signal reflection surface; the signal strength value is the signal strength of the reflected signal corresponding to the detection signal emitted by the sensor; and the ambient noise value indicates the noise signal strength of the surrounding environment. The signal reflection surface is the surface of the object that reflects the detection signal of the door control sensor. For example, for a tailgate step sensor installed in the vehicle bumper, the signal reflection surface is the ground when no object enters the sensing area. When a user performs a door control action within the sensing area, the signal reflection surface is the user's foot or shoe surface. When an animal passes through the sensing area, the signal reflection surface is the animal's skin surface.

[0077] Optionally, the detection signal emitted by the door control sensor may be a light signal, and accordingly, the signal strength value is the signal strength of the reflected light, and the ambient noise value is the signal strength of the ambient light received by the door control sensor excluding the reflected light. Alternatively, the detection signal emitted by the door control sensor may be a sound signal, and accordingly, the signal strength value is the signal strength of the reflected sound wave, and the ambient noise value is the signal strength of the ambient sound received by the door control sensor excluding the reflected sound wave. This embodiment of the application is described using a light signal as an example.

[0078] Taking a vehicle's tailgate foot-step sensor as an example, the door control action is a foot-step. When the user steps on the sensing area, the distance between the door control sensor and the foot surface is smaller than the distance between the door control sensor and the ground, resulting in a smaller distance measurement. However, action recognition based solely on distance measurements is less reliable. For example, if a falling leaf passes through the sensing area and obscures the ground, the door control sensor may mistakenly identify it as a door opening action, triggering the door to open. Alternatively, if a small animal passes through the sensing area, the sensor may mistakenly identify it as a door opening action. For door control sensors using Time of Flight (TOF) technology, the reflectivity of laser light varies significantly across different surface materials, resulting in significant variations in the reflected light received by the door control sensor. This can affect the distance measurement and cause inaccurate distance measurement. For example, if the distance between the door control sensor and the ground is 50 cm, the ground is white, and the user's shoe is black. Because the signal strength of the laser reflected from the black shoe is less than that from the white ground, the distance measurement result may still be close to 50 cm, making it impossible to recognize the user's intended door opening action. The embodiment of the present disclosure performs action recognition by combining distance measurement values, signal strength values, and environmental noise values, which can make the action recognition results more accurate.

[0079] In one possible embodiment, the door control sensor collects at least one of a distance measurement value, a signal strength value, and an ambient noise value in real time. For example, for a door control sensor on a vehicle's tailgate, the corresponding preset door opening / closing action is typically a foot-operated action. If the door control sensor is located at the rear of the vehicle, the distance between the surface of an object on the ground and the sensor can be measured as one of the criteria for identifying the preset door opening action, namely, the distance measurement value. Furthermore, the door control sensor can emit a light beam in a preset direction (toward the ground) at a certain frequency and sense the reflected light beam to obtain a signal strength value. Furthermore, during the period when no reflected light beam is received (the period when the reflected light beams corresponding to two adjacent detection light beams reach the sensor), the sensor can measure the ambient light intensity to obtain an ambient noise value.

[0080] Taking a TOF sensor as an example, a door control sensor continuously emits laser pulses toward the ground. The distance measurement value is calculated based on the time it takes for the laser pulses to return to the door control sensor. The signal strength of the reflected laser is detected, and the intensity of ambient light other than the reflected laser is measured as ambient noise. Different types of door control sensors may have different methods for collecting sensor data. The present disclosure does not limit the specific methods for collecting distance measurements and signal parameters.

[0081] Step 102 : Perform door control action recognition based on the sensor data to obtain a first recognition result of whether a door control action exists.

[0082] In one possible implementation, a model with classification function, such as a deep learning model, a random forest model, or a decision tree model, may be used to recognize door control actions. The deep learning model may be pre-trained using sample sensor data carrying sample labels, where the sample labels are used to indicate "door control action detected" or "door control action not detected."

[0083] Optionally, the deep learning model can be constructed using convolutional neural networks (CNN), recurrent neural networks (RNN), deep neural networks (DNN), etc.

[0084] Step 103 , determining whether the sensor data meets the corresponding preset threshold standard and whether the duration of meeting the corresponding preset threshold standard is within a preset duration range, and obtaining a second recognition result of whether there is a door control action.

[0085] In one possible implementation, the preset threshold standards include a distance threshold, a signal strength threshold, and an ambient noise threshold. When the value of the sensor data at a certain moment meets the distance threshold, the signal strength threshold, and the ambient noise threshold, it is determined that the sensor data at that moment meets the preset threshold standards, and the duration for which the subsequently collected sensor data meets the corresponding preset threshold standards is continued to be detected.

[0086] Optionally, the door control sensor can simultaneously detect whether the distance measurement value, signal strength value, and ambient noise value each meet corresponding preset threshold standards; alternatively, the door control sensor can first detect whether one of the sensor data meets the corresponding preset threshold standard, and if not, directly obtain a second recognition result indicating that no door control action is currently in effect; if so, continue to determine whether the next sensor data meets the corresponding preset threshold standard. This embodiment of the present application does not limit the method for generating the second recognition result.

[0087] The disclosed embodiments take into account that the user's door control action usually lasts for a certain period of time (for example, for the foot-stepping door opening action, the duration of the user's foot stepping is longer than the duration of a small animal passing by, and shorter than the duration of obstruction by obstacles such as leaves). Therefore, based on numerical detection, a time continuity check can be performed on the sensor data to determine whether the duration of the current action is within the reasonable duration range of the door control action.

[0088] Furthermore, since the user's posture usually remains unchanged while maintaining the door control action, that is, the corresponding sensor data should remain unchanged or fluctuate very little, therefore, in order to further improve the accuracy of the recognition result, a second recognition result can be obtained by determining whether the sensor data meets the corresponding preset threshold standard, and whether the duration of meeting the corresponding preset threshold standard and the value change being less than the change threshold is within the preset time range.

[0089] Optionally, the door control sensor can also perform a time continuity check based on the sensor data changes in five stages: before the action occurs, during the action start, during the action maintenance, during the action end, and after the action ends. Taking the foot-operated action as the preset door opening action as an example, Figure 3 The diagram shows a schematic diagram of the change of detection data values ​​over time during a user's preset door opening action. The detection data includes at least one of a distance measurement value and a signal parameter. Figure 2As shown, before the user performs the foot-stepping action, the detection data collected by the door control sensor is consistent with the data when the sensor directly detects the ground (i.e., the ground is stable). When the user starts to perform the preset door opening action, data fluctuations will occur (i.e., the stepping process). The preset door opening action will be maintained for a certain period of time (i.e., the shoe upper is stable). When the user ends the preset door opening action, data fluctuations will occur again (i.e., the lifting process). Then, the detection data received by the door control sensor will return to the value when the ground is directly detected (i.e., the ground is restored to stability). If the door control sensor data meets the preset threshold standard and the change characteristics meet the requirements, Figure 2 The features shown in FIG. 1 are used to obtain a second recognition result indicating that a door control action exists.

[0090] Step 104 : Based on the first recognition result and the second recognition result, a fusion recognition result of whether there is a door control action is obtained.

[0091] Optionally, one action recognition algorithm may be used to perform action recognition to obtain an action recognition result, or at least two action recognition algorithms may be used to perform action recognition, and the recognition results obtained by each algorithm may be combined to comprehensively generate an action recognition result.

[0092] Step 105 : In response to the fusion recognition result indicating that there is a door control action, a door control signal is sent to a door controller, so that the door controller controls the door to be opened or closed based on the door control signal and the current state of the door.

[0093] When the fusion recognition result indicates a door control action, the door control sensor sends a door control signal to the door controller. In response, the door controller controls the door opening or closing based on the door control signal and the door's current state. For example, if the door is currently closed when the door control signal is received, the controller opens the door; if the door is currently open when the door control signal is received, the controller closes the door.

[0094] Optionally, the door control sensor can be used for the tailgate of the vehicle, as well as other doors other than the tailgate of the vehicle. Other doors include the side doors on the left and right sides of the vehicle. Schematically, when the vehicle is a small car, the other doors include the four doors on the left and right sides of the small car, that is, the passenger compartment door and the driver's cab door. When the door control sensor is used for the tailgate, it can be installed around the tailgate, such as being set on the rear bumper of the vehicle to control the tailgate; when used for other doors, it can be set around the other doors to control other doors. If used for a side door, it can be installed around the side door. In one possible embodiment, a plurality of door control sensors can be set on the vehicle, each used to control a different door.

[0095] Based on the embodiment of the present disclosure, by real-time collection of at least one sensor data including distance measurement values, signal strength values ​​and environmental noise values, door control actions are identified based on the sensor data, taking into account that the user's door control actions will not only affect the distance measurement values ​​but also affect the signal strength of the reflected signal and the intensity of the environmental noise; secondly, on the one hand, door control actions are identified based on the sensor data to obtain a first recognition result, and on the other hand, based on a preset threshold standard and time continuity, it is determined whether there is sensor data corresponding to the door control action to obtain a second recognition result. The results of at least two recognition methods are combined to obtain a fused recognition result of the door control action, which can improve the reliability of the door control action recognition results and, to a certain extent, solve the problems of low action recognition rate and false triggering of door control sensors in actual applications, thereby improving the accuracy of door control.

[0096] In a possible implementation, the door control sensor of the embodiment of the present disclosure adopts a TOF sensor. Figure 1 Based on the embodiment shown, step 101 may specifically include the following steps:

[0097] Step 101a: transmit a detection signal in a preset direction at a preset frequency and receive a reflection signal of the detection signal.

[0098] Step 101b: determining a distance measurement value based on a flight time between the detection signal and the reflected signal, where the flight time refers to the time from when the detection signal is transmitted to when the sensor receives the reflected signal.

[0099] Step 101c: Detect the signal strength value of the received reflected signal.

[0100] Step 101d: detecting the ambient noise value of the surrounding environment during the period when no reflected signal is received.

[0101] In one possible embodiment, the sensor continuously emits a detection signal in a preset direction at a preset frequency, such as a laser pulse of a TOF sensor, and can determine the distance measurement value based on the flight duration of the detection signal, while detecting the signal strength value of the reflected signal corresponding to the detection light, and detecting the ambient noise value (for example, the ambient noise value is measured during the period of receiving the reflected light of two adjacent beams of detection light).

[0102] Indicative, such as Figure 4As shown, the door control sensor is provided with a control module 401, a laser 402, a transmitting optical system 403, a receiving optical system 404 and a photodetector 405. The control module 401 is used to generate a trigger signal for the laser 402 in the working mode to trigger the laser 402 to generate laser light. The laser light emitted by the laser 402 is shaped into a laser beam by the transmitting optical system 403. When the laser beam reaches the surface of an object, it is reflected to generate a reflection signal. The reflection signal enters the receiving optical system 404 and is sensed by the photodetector 405. The photodetector 405 can output a histogram based on the input reflection signal, such as Figure 5 As shown, the peak value of the reflected signal in the histogram is the reflected signal intensity, the noise value in the signal-free area is the ambient noise value, and the acquisition time of the reflected signal is the laser flight time. The distance measurement value can be calculated based on the speed of light and the laser flight time. The control module 401 can obtain the distance measurement value and information such as the signal intensity of the reflected signal and the ambient noise value by parsing the histogram. Schematically, the photodetector can be implemented using a single-photon avalanche diode (SPAD).

[0103] In one possible implementation, the first recognition result includes a first confidence level and a first classification result indicating whether a door control action has occurred within a preset time period, and the second recognition result includes a second confidence level and a second classification result indicating whether a door control action has occurred within a preset time period. Accordingly, step 104 may include the following steps:

[0104] The first confidence level is used as the weight of the first classification result, and the second confidence level is used as the weight of the second classification result, and a weighted sum calculation is performed to obtain the fusion recognition result.

[0105] Schematically, the first classification result includes a result score, and different result types correspond to different result scores. For example, the result types of the first classification result include door control action and invalid action, where the result score corresponding to the door control action is 1, and the result score corresponding to the invalid action is -1. The first confidence level is used to characterize the reliability of the first classification result, and can be calculated based on the numerical value of the sensor data and the various thresholds in the preset threshold standard. Correspondingly, different result types of the second classification result correspond to different result scores, and the second confidence level is used to characterize the reliability of the second classification result, which can be obtained through a deep learning model. The calculation formula of the weighted fusion score is as follows:

[0106] Fusion recognition result = first confidence * first classification result + second confidence * second classification result

[0107] For example, it can be set that when the value of the fusion recognition result is greater than or equal to 0.5, it is determined that the door control action is detected, and when it is less than 0.5, it is determined that the door control action is not detected.

[0108] The disclosed embodiments illustrate motion recognition using two recognition methods. In other possible implementations, more recognition methods may be used for motion recognition, and the disclosed embodiments are not limited thereto. By performing motion recognition based on sensor data using at least two different types of motion recognition methods, a classification result and a confidence level representing the reliability of the result are obtained for each motion recognition method. The recognition result is then fused based on the classification results and the confidence level, further improving the reliability of the motion recognition result and the accuracy of door control.

[0109] In one possible implementation, a deep learning model can be used to identify a door control action based on sensor data to obtain a first recognition result. The deep learning model can perform action recognition by learning information such as the values ​​of sensor data in various scenarios, trends in value changes, and correlations between different types of sensor data. The sensor can input a sequence of sensor data collected within a preset time period into the model at one time to determine whether a door control action occurs within the preset time period. Step 102 may include the following steps:

[0110] Step 102a: Use the input layer of the deep learning model to input the sensor data sequence collected within a preset time period.

[0111] Step 102b: Use the convolution layer of the deep learning model to extract features from the sensor data sequence to obtain a feature matrix of the sensor data sequence.

[0112] Step 102c: Perform nonlinear transformation on the feature matrix of the sensor data sequence using the activation function of the deep learning model.

[0113] Step 102d: Use the fully connected layer of the deep learning model to classify the feature matrix of the sensor data sequence after nonlinear transformation.

[0114] Step 102e: Use the output layer of the deep learning model to output a first confidence level and a first classification result.

[0115] Indicative, Figure 3The architecture of a deep learning model is shown. The model includes an input layer, a convolutional layer, an activation layer, a fully connected layer, and an output layer. First, the input layer is used to input a sequence of sensor data collected within a preset time length (for example, 1 second), and then the convolution kernel in the convolutional layer is used to extract features of the input sensor data sequence. After obtaining the feature matrix, it is input into the activation layer, and the activation function in the activation layer, such as the S-shaped growth curve function (Sigmoid), the linear rectifier function (Rectified Linear Unit, ReLU), the normalized exponential function (Softmax), etc., is used to perform nonlinear transformation on the feature matrix. Then, the fully connected layer is used to perform global feature integration and classification on the feature matrix after nonlinear transformation, and the first classification result "door control action detected" or "door control action not detected" and the first confidence level are output through the output layer, wherein the first confidence level can be generated by the activation function.

[0116] In another possible implementation, the sensor may sequentially input each set of sensor data sequences collected within a preset time period into the model to obtain a classification result and confidence level corresponding to each set of sensor data. A weighted average calculation is then performed to obtain a first classification result and a first confidence level of the sensor data within the preset time period to determine whether a door control action occurs within the preset time period. Step 102 may include the following steps:

[0117] Step 102f: Use the input layer of the deep learning model to input a set of sensor data collected within a preset time period.

[0118] Step 102g: Use the convolution layer of the deep learning model to extract features from a set of sensor data to obtain a feature matrix of the set of sensor data.

[0119] Step 102h: Use the activation function of the deep learning model to perform nonlinear transformation on the feature matrix of a set of sensor data.

[0120] Step 102i uses the fully connected layer of the deep learning model to classify the feature matrix of a set of sensor data after nonlinear transformation.

[0121] Step 102j: Use the output layer of the deep learning model to output the confidence and classification results corresponding to a set of sensor data.

[0122] Step 102k: Perform weighted average calculation based on the confidence and classification results of each group of sensor data collected within a preset time period to obtain a first confidence and a first classification result.

[0123] Among them, a set of sensor data includes distance measurement values, signal strength values ​​and environmental noise values ​​collected at a certain moment.

[0124] like Figure 3 As shown, first, a group of sensor data is input into the input layer, and then the convolution kernel in the convolution layer is used to extract features of the input group of sensor data. After obtaining the feature matrix, it is input into the activation layer, and the activation function in the activation layer is used to perform nonlinear transformation on the feature matrix. Then, the fully connected layer is used to perform global feature integration and classification on the feature matrix after nonlinear transformation. The classification result and confidence of the group of sensor data are output through the output layer. Finally, the confidence and classification results of each group of sensor data collected within a preset time are weighted averaged to obtain the first confidence and the first classification result.

[0125] In a possible implementation, the second classification result may be determined based on a preset threshold criterion, and the second confidence level may be calculated according to the sensor data and the preset threshold criterion.

[0126] Optional, Figure 6 A schematic diagram shows a process for generating a second classification result: 1. Numerical detection of distance measurement value, judging whether the distance measurement value passes the detection based on the distance threshold interval, if so, executing step 2, if not, determining that the second classification result is that the door control action is not detected; 2. Numerical detection of signal strength value, judging whether the reflected signal strength value passes the detection based on the signal strength threshold interval, if so, executing step 3, if not, determining that the second classification result is that the door control action is not detected; 3. Numerical detection of ambient noise, judging whether the ambient noise value passes the detection based on the noise value threshold interval in the preset conditions, if so, executing step 4, if not, determining that the second classification result is that the door control action is not detected; 4. Time continuity check, judging whether the time continuity check passes based on the change of sensor data, if so, determining that the second classification result is that the door control action is detected, if not, determining that the second classification result is that the door control action is not detected.

[0127] Optionally, since there is usually a motion change period and a motion maintenance period when the user performs the door control action, the sensor data is in an unstable state and a stable state accordingly, such as Figure 7 As shown, it is also possible to first determine whether the sensor data is in a stable state. If it is in a stable state, it is then determined whether the sensor data meets the preset threshold condition. The above step 103 may include the following steps:

[0128] Step 103a: Use a sliding window to collect sensor data, and determine whether the current sliding window is in a stable state based on the range within the sliding window, where the range refers to the difference between the maximum value of the sensor data and the minimum value of the sensor data. A sliding window in a stable state is a sliding window with a range less than a range threshold.

[0129] For example, the door control sensor's data acquisition frequency is 100Hz, and the sliding window contains L frames of data. Therefore, the sliding window can be used to determine the stability of sensor data within a period of L*10ms. A stable state refers to a state in which sensor data fluctuations are minimal, meaning that the distance between the signal reflecting surface and the door control sensor is relatively stable and the signal reflecting surface remains unchanged. Therefore, a range threshold can be set to determine whether the sensor data within the sliding window is stable. If the range of the sensor data is less than the range threshold, it indicates that the sensor data fluctuations are minimal and the sliding window is currently stable.

[0130] Furthermore, the range threshold can be dynamically adjusted according to the signal-to-noise ratio of the current sliding window. Step 103a can specifically include the following steps: Step 1, determining the signal-to-noise ratio corresponding to the sliding window based on the mean signal strength and the mean ambient noise value in the current sliding window; Step 2, determining the range threshold corresponding to the current sliding window based on the signal-to-noise ratio, wherein the range threshold is negatively correlated with the signal-to-noise ratio; Step 3, in response to the range being less than the range threshold, determining that the current sliding window is in a stable state; Step 4, in response to the range being greater than or equal to the range threshold, determining that the current sliding window is in an unstable state.

[0131] Schematically, the signal-to-noise ratio is expressed as follows: log(S / N), where S is the mean signal strength within the sliding window (i.e., the average signal strength value of each frame within the sliding window), and N is the mean ambient noise within the sliding window (i.e., the average ambient noise value of each frame within the sliding window). When the signal-to-noise ratio is large, the sensor is less affected by ambient noise and the ranging fluctuation is small. Setting a smaller range threshold can improve detection accuracy. When the signal-to-noise ratio is small, the sensor is greatly affected by ambient noise and the ranging fluctuation is large. Appropriately relaxing the limit can avoid false detection caused by data fluctuation. Schematically, the range threshold is calculated as follows: th1 = K / log(S / N), where th1 is the range threshold of the current sliding window, K is the proportional coefficient, and log(S / N) is the signal-to-noise ratio of the current sliding window.

[0132] Step 103b: In response to the previous sliding window being in an unstable state and the current sliding window being in a stable state, determining whether the sensor data in the current sliding window meets a preset threshold standard.

[0133] Step 103c: Determine the duration of the sensor data being in a stable state and meeting the preset threshold standard based on the number of sliding windows that are continuously in a stable state and meet the preset threshold standard starting from the current sliding window.

[0134] If the previous sliding window is in an unstable state and the current sliding window is in a stable state, it is possible that the user is performing a door control action. In this case, the sensor data of the current sliding window is further determined to determine whether it meets the preset threshold standard. If so, the status of each subsequent sliding window is continuously determined and whether it meets the preset threshold standard until it enters an unstable state or does not meet the preset threshold standard. The number of sliding windows that are continuously in a stable state and meet the preset threshold standard is thus obtained. Based on the data collection duration corresponding to the sliding window, the duration of time that the sensor data is in a stable state and meets the preset threshold standard is determined. Illustratively, the preset threshold standard may include, but is not limited to: at least one of a distance measurement value falling within a preset distance range, a signal strength value falling within a preset signal strength range, and an ambient noise value falling within a preset noise value range.

[0135] Step 103d: In response to the duration being within a preset duration range, a second classification result of detecting the door control action is obtained.

[0136] If the sensor data is in a stable state and the duration of meeting the preset threshold standard is within the preset time range, it is determined that the door control action is detected; if the sensor data is in an unstable state, or the sensor data does not meet the preset threshold standard, or the sensor data is in a stable state and the duration of meeting the preset threshold standard is not within the preset time range, a second classification result is obtained that the door control action is not detected.

[0137] Step 103e: Calculate a second confidence level based on the distance threshold interval, the signal strength threshold interval, the noise value threshold interval, and the value of the sensor data.

[0138] Optionally, the second confidence level is calculated as follows:

[0139] Second confidence level = (threshold interval endpoint difference - 2*|sensor data value - threshold interval median|) / threshold interval endpoint difference

[0140] Among them, the threshold interval includes at least one of the distance threshold interval, the signal strength threshold interval and the noise value threshold interval, the threshold interval endpoint difference is the absolute value of the difference between the values ​​corresponding to the two endpoints of the threshold interval, and the threshold interval median is the value corresponding to the middle point of the threshold interval.

[0141] For example, the preset threshold standard stipulates that the distance measurement value for the door control action is between 1 cm and 9 cm (inclusive), that is, the numerical interval of the distance measurement value is [1, 9], the difference between the numerical interval endpoints is 8, and the numerical interval median is 5. When the distance measurement value is 5 cm, the confidence level is 1; when the distance measurement value is 9 cm, the confidence level is 0; when the distance measurement value is 11 cm, the confidence level is 0.25, and the action score is -1.

[0142] Optionally, the confidence level of each set of sensor data within a preset time period can be calculated, and then the average (or lowest value) of the confidence levels corresponding to all sensor data within the preset time period can be taken as the second confidence level in the second recognition result. The confidence level of each set of sensors can be calculated by averaging the confidence level of the distance measurement value, the confidence level of the signal strength value, and the confidence level of the ambient noise value.

[0143] In the embodiment of the present disclosure, the credibility of the first classification result and the second classification result are respectively characterized by calculating the first confidence level and the second confidence level, and a fusion calculation is performed based on the first confidence level, the second confidence level, the first classification result and the second classification result to obtain a fusion recognition result, thereby improving the reliability of the fusion recognition result, reducing the possibility of misidentification and improving the sensor's recognition rate of the door control action.

[0144] Please refer to Figure 8 , which shows a structural block diagram of a vehicle door control device provided by an exemplary embodiment of the present disclosure. The vehicle door control device provided in this embodiment is applied to a vehicle door control sensor, and the device includes:

[0145] An acquisition module 801 is configured to acquire sensor data from a door control sensor, the sensor data including but not limited to at least one of a distance measurement value, a signal strength value, and an ambient noise value; wherein the distance measurement value indicates the distance between the sensor and a signal reflection surface, the signal reflection surface being a surface that reflects the detection signal from the door control sensor;

[0146] A first recognition module 802 is configured to recognize a door control action based on the sensor data collected by the collection module 801, and obtain a first recognition result of whether a door control action exists;

[0147] The second recognition module 803 is configured to determine whether the sensor data collected by the collection module 801 meets a corresponding preset threshold standard and whether the duration of meeting the corresponding preset threshold standard is within a preset time range, thereby obtaining a second recognition result of whether a door control action is present;

[0148] A fusion module 804 is configured to obtain a fusion recognition result of whether a door control action exists based on the first recognition result obtained by the first recognition module 802 and the second recognition result obtained by the second recognition module 803;

[0149] The control module 805 is used to send a door control signal to the door controller in response to the fusion recognition result of the door control action obtained by the fusion module 804, so that the door controller controls the door to open or close based on the door control signal and the current state of the door.

[0150] Optionally, in a possible implementation, the acquisition module 801 may also be used to:

[0151] Transmitting a detection signal in a preset direction at a preset frequency and receiving a reflected signal of the detection signal;

[0152] Determining a distance measurement based on a time-of-flight between the detection signal and the reflected signal, wherein the time-of-flight is the time from when the detection signal is transmitted to when the reflected signal is received by the door control sensor;

[0153] Detecting the signal strength value of the received reflected signal;

[0154] The ambient noise value of the surrounding environment during the period when no reflected signal is received is detected.

[0155] Optionally, in one possible implementation, the first recognition result includes a first confidence level and a first classification result indicating whether a door control action occurs within a preset duration, and the second recognition result includes a second confidence level and a second classification result indicating whether a door control action occurs within a preset duration. The fusion module 804 of the embodiment of the present disclosure is further configured to:

[0156] The first confidence level is used as the weight of the first classification result, and the second confidence level is used as the weight of the second classification result, and a weighted sum calculation is performed to obtain the fusion recognition result.

[0157] Optionally, in a possible implementation manner, the first identification module 802 of the embodiment of the present disclosure is further configured to:

[0158] Using the input layer in the deep learning model to input the sensor data sequence collected by the acquisition module 801 within a preset time length;

[0159] The convolutional layer in the deep learning model is used to extract features from the sensor data sequence to obtain the feature matrix of the sensor data sequence;

[0160] The activation function in the deep learning model is used to perform nonlinear transformation on the feature matrix of the sensor data sequence;

[0161] The fully connected layer in the deep learning model is used to classify the feature matrix of the sensor data sequence after nonlinear transformation;

[0162] The output layer in the deep learning model is used to output the first confidence and the first classification result.

[0163] Optionally, in another possible implementation, the first identification module 802 of the embodiment of the present disclosure is further configured to:

[0164] Using the input layer in the deep learning model to input a set of sensor data collected by the collection module 801 within a preset time period;

[0165] Use the convolutional layer in the deep learning model to extract features from a set of sensor data to obtain a feature matrix of a set of sensor data;

[0166] Using the activation function in the deep learning model to perform nonlinear transformation on the feature matrix of a set of sensor data;

[0167] Use the fully connected layer in the deep learning model to classify the feature matrix of a set of sensor data after nonlinear transformation;

[0168] Use the output layer in the deep learning model to output the confidence and classification results corresponding to a set of sensor data;

[0169] A first confidence level and a first classification result are obtained by performing a weighted average calculation based on the confidence levels and classification results of each group of sensor data collected by the collection module 801 within a preset time period.

[0170] Optionally, in a possible implementation manner, the second identification module 803 of the embodiment of the present disclosure is further configured to:

[0171] Use a sliding window to collect sensor data, and determine whether the current sliding window is in a stable state based on the range within the current sliding window, where the range refers to the difference between the maximum value of the sensor data and the minimum value of the sensor data. A sliding window in a stable state is a sliding window with a range less than a range threshold;

[0172] In response to the previous sliding window being in an unstable state and the current sliding window being in a stable state, determining whether the sensor data within the current sliding window meets a preset threshold standard;

[0173] Determining a duration during which the sensor data is in a stable state and meets a preset threshold criterion based on the number of sliding windows that are continuously in a stable state and meet a preset threshold criterion starting from the current sliding window;

[0174] In response to the duration being within a preset duration range, obtaining a second classification result of detecting the door control action;

[0175] A second confidence level is calculated based on the distance threshold interval, the signal strength threshold interval, the noise value threshold interval, and the value of the sensor data.

[0176] Optionally, in a possible implementation, the second confidence level is calculated as follows:

[0177] Second confidence level = (threshold interval endpoint difference - 2*|sensor data value - threshold interval median|) / threshold interval endpoint difference

[0178] Among them, the threshold interval includes at least one of the distance threshold interval, the signal strength threshold interval and the noise value threshold interval, the threshold interval endpoint difference is the absolute value of the difference between the values ​​corresponding to the two endpoints of the threshold interval, and the threshold interval median is the value corresponding to the middle point of the threshold interval.

[0179] Optionally, in a possible implementation manner, the second identification module 803 of the embodiment of the present disclosure is further configured to:

[0180] Determine the signal-to-noise ratio corresponding to the sliding window based on the mean signal strength and the mean ambient noise within the current sliding window;

[0181] Determine the range threshold corresponding to the current sliding window based on the signal-to-noise ratio, where the range threshold is negatively correlated with the signal-to-noise ratio;

[0182] In response to the range being less than the range threshold, determining that the current sliding window is in a stable state;

[0183] In response to the range being greater than or equal to the range threshold, it is determined that the current sliding window is in an unstable state.

[0184] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The parts of the embodiments that are the same, similar, or corresponding can be referred to each other. Since the system, method, device, and equipment embodiments basically correspond to each other, the relevant parts can be referred to the description of the corresponding parts. The systems, methods, devices, and equipment of the embodiments of the present disclosure also correspond to each other in terms of specific implementation and beneficial technical effects. The relevant contents can be referenced to each other and will not be repeated here.

[0185] In addition, an embodiment of the present disclosure further provides an electronic device, including:

[0186] memory for storing computer programs;

[0187] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the vehicle door control method described in any of the above embodiments of the present disclosure is implemented.

[0188] Figure 9 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. Figure 9 The electronic device according to the embodiment of the present disclosure is described. The electronic device may be either or both of the first device and the second device, or a standalone device independent of them, and the standalone device may communicate with the first device and the second device to receive collected input signals from them.

[0189] like Figure 9 As shown, the electronic device includes one or more processors and memory.

[0190] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0191] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement the vehicle door control method of the various embodiments of the present disclosure described above and / or other desired functions.

[0192] In one example, the electronic device may further include an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0193] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0194] The output device can output various information to the outside, including determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0195] Of course, to simplify, Figure 9 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.

[0196] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the vehicle door control method according to various embodiments of the present disclosure described in the above part of this specification.

[0197] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0198] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the vehicle door control method according to various embodiments of the present disclosure described in the above part of this specification.

[0199] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0200] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc. Various media that can store program codes.

[0201] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0202] Each embodiment in this specification is described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. References to the same or similar parts between the various embodiments are sufficient. For system embodiments, since they largely correspond to method embodiments, their description is relatively simple. For relevant parts, references to the description of the method embodiments are sufficient.

[0203] The block diagrams of the devices, devices, equipment, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0204] The methods and apparatus of the present disclosure may be implemented in many ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers recording media that store programs for executing the methods according to the present disclosure.

[0205] It should also be noted that in the apparatus, device, and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0206] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0207] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A vehicle door control method, characterized in that: Applied to a door control sensor, the method includes: collecting sensor data from the door control sensor, the sensor data including at least one of a distance measurement value, a signal strength value, and an ambient noise value; wherein the distance measurement value indicates a distance between the door control sensor and a signal reflection surface, the signal reflection surface being a surface of an object that reflects a detection signal from the door control sensor; Performing door control action recognition based on the sensor data using a deep learning model to obtain a first recognition result of whether a door control action exists; determining whether the sensor data satisfies a corresponding preset threshold standard and whether a duration of satisfying the corresponding preset threshold standard is within a preset duration range, and obtaining a second recognition result of whether a door control action exists; Obtaining a fusion recognition result of whether a door control action exists based on the first recognition result and the second recognition result; In response to a fusion recognition result indicating that a door control action is present, sending a door control signal to a door controller so that the door controller controls the door to be opened or closed based on the door control signal and a current state of the door; The first recognition result includes a first confidence level and a first classification result for characterizing whether the door control action occurs within a preset time period. The second recognition result includes a second confidence level and a second classification result for characterizing whether the door control action occurs within the preset time period. The first confidence level is obtained by identifying the sensor data using the deep learning model. The second confidence level is calculated based on the value of the sensor data and various thresholds in the preset threshold standard. The calculation formula for the second confidence level is as follows: Second confidence level = / Threshold interval endpoint difference; The threshold interval includes at least one of a distance threshold interval, a signal strength threshold interval, and a noise value threshold interval; the threshold interval endpoint difference is the absolute value of the difference between the values ​​corresponding to the two endpoints of the threshold interval; and the threshold interval median is the value corresponding to the middle point of the threshold interval; The obtaining, based on the first recognition result and the second recognition result, of a fusion recognition result of whether a door control action exists includes: The first confidence level is used as the weight of the first classification result, and the second confidence level is used as the weight of the second classification result, and a weighted sum calculation is performed to obtain the fusion recognition result.

2. The method according to claim 1, characterized in that The collecting of sensor data of the door control sensor includes: transmitting the detection signal in a preset direction at a preset frequency and receiving a reflected signal of the detection signal; determining the distance measurement value based on a flight time between the detection signal and the reflected signal, wherein the flight time is the time from when the detection signal is transmitted to when the reflected signal is received by the door control sensor; detecting a signal strength value of the received reflected signal; The ambient noise value of the surrounding environment during a period in which the reflected signal is not received is detected.

3. The method according to claim 1, characterized in that The method of performing door control action recognition based on the sensor data using a deep learning model to obtain a first recognition result of whether a door control action exists includes: Using the input layer in the deep learning model to input the sensor data sequence collected within the preset time period; Performing feature extraction on the sensor data sequence using a convolutional layer in the deep learning model to obtain a feature matrix of the sensor data sequence; Performing a nonlinear transformation on the feature matrix of the sensor data sequence using an activation function in the deep learning model; Classifying the feature matrix of the sensor data sequence after nonlinear transformation using a fully connected layer in the deep learning model; Utilize the output layer in the deep learning model to output the first confidence and the first classification result.

4. The method according to claim 1, wherein The determining whether the sensor data satisfies a corresponding preset threshold standard and whether a duration of satisfying the corresponding preset threshold standard is within a preset duration range, and obtaining a second recognition result of whether a door control action exists, includes: Collecting the sensor data using a sliding window, and determining whether the current sliding window is in a stable state based on a range within the current sliding window, wherein the range refers to a difference between a maximum value of the sensor data and a minimum value of the sensor data, and a sliding window in the stable state is a sliding window in which the range is less than a range threshold; In response to a previous sliding window being in an unstable state and the current sliding window being in the stable state, determining whether sensor data within the current sliding window meets the preset threshold standard; Determining a duration of time during which the sensor data is in the stable state and meets the preset threshold criterion based on the number of sliding windows that are continuously in the stable state and meet the preset threshold criterion starting from the current sliding window; In response to the duration being within the preset duration range, obtaining a second classification result of detecting the door control action; The second confidence level is calculated based on the distance threshold interval, the signal strength threshold interval, the noise value threshold interval, and the value of the sensor data.

5. The method according to claim 4, characterized in that The determining whether the current sliding window is in a stable state based on the range in the sliding window includes: Determining a signal-to-noise ratio corresponding to the sliding window based on a mean signal strength value and a mean ambient noise value within the current sliding window; Determining the range threshold corresponding to the current sliding window based on the signal-to-noise ratio, wherein the range threshold is negatively correlated with the signal-to-noise ratio; In response to the range being less than the range threshold, determining that the current sliding window is in the stable state; In response to the range being greater than or equal to the range threshold, it is determined that the current sliding window is in the unstable state.

6. A door control device, characterized in that: Applied to a door control sensor, the device comprises: an acquisition module configured to acquire sensor data from the door control sensor, the sensor data including at least one of a distance measurement value, a signal strength value, and an ambient noise value; wherein the distance measurement value indicates the distance between the door control sensor and a signal reflection surface, the signal reflection surface being a surface of an object that reflects a detection signal from the door control sensor; a first recognition module, configured to perform door control action recognition based on the sensor data using a deep learning model to obtain a first recognition result of whether a door control action exists; a second recognition module, configured to determine whether the sensor data satisfies a corresponding preset threshold standard and whether a duration of satisfying the corresponding preset threshold standard is within a preset duration range, and obtain a second recognition result of whether a door control action exists; a fusion module, configured to obtain a fusion recognition result of whether a door control action exists based on the first recognition result and the second recognition result; a control module, configured to send a door control signal to a door controller in response to a fusion recognition result indicating that a door control action exists, so that the door controller controls the door to be opened or closed based on the door control signal and a current state of the door; The first recognition result includes a first confidence level and a first classification result for characterizing whether the door control action occurs within a preset time period. The second recognition result includes a second confidence level and a second classification result for characterizing whether the door control action occurs within the preset time period. The first confidence level is obtained by identifying the sensor data using the deep learning model. The second confidence level is calculated based on the value of the sensor data and various thresholds in the preset threshold standard. The calculation formula for the second confidence level is as follows: Second confidence level = / Threshold interval endpoint difference; The threshold interval includes at least one of a distance threshold interval, a signal strength threshold interval, and a noise value threshold interval; the threshold interval endpoint difference is the absolute value of the difference between the values ​​corresponding to the two endpoints of the threshold interval; and the threshold interval median is the value corresponding to the middle point of the threshold interval; The fusion module is further configured to: The first confidence level is used as the weight of the first classification result, and the second confidence level is used as the weight of the second classification result, and a weighted sum calculation is performed to obtain the fusion recognition result.

7. An electronic device, characterized in that: include: Memory for storing computer programs; A processor is configured to execute a computer program stored in the memory, and when the computer program is executed, implements the method described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

9. A computer program comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images