Vehicle door control method, device and equipment and storage medium

Abstract

The invention provides a vehicle door control method, device and equipment and a storage medium, sensor data of a vehicle door control sensor is collected, and the sensor data comprises at least one of a distance measurement value, a signal strength value and an environment noise value; vehicle door control action recognition is conducted based on the sensor data, and a first recognition result about whether the vehicle door control action is detected or not is obtained; determining whether the sensor data meets a corresponding preset threshold standard or not and whether the duration of the sensor data meeting the corresponding preset threshold standard is within a preset duration range or not, and obtaining a second identification result of whether a vehicle door control action exists or not; based on the first recognition result and the second recognition result, obtaining a fusion recognition result about whether the vehicle door control action exists or not; in response to the fusion recognition result of the vehicle door control action, the vehicle door control signal is sent to the vehicle door controller, and the problems of low action recognition rate and false triggering of the vehicle door control sensor in practical application are solved to a certain extent.

Description

Technical Field

[0001] The present disclosure relates to control technology, and in particular to a vehicle door control method, device, equipment and storage medium. Background Art

[0002] In recent years, with the rapid development of automobile intelligent technology, in-vehicle intelligent sensing products have gradually become popular. The traditional car tailgate switch includes a key button and a button on the outside of the tailgate. Although it can effectively control the tailgate switch, it is not convenient enough in some scenarios. For example, when the user's hands are full of items, they cannot touch the button or switch. In order to solve the above problems, a new type of light-sensing foot sensor came into being.

[0003] The optical foot sensor uses a laser time-of-flight (TOF) sensor to realize the sensing function. Its main working principle is to project a light spot on the ground to indicate the position of the user's foot. When an object is detected stepping into the range of the light spot, it identifies whether there is a foot action based on the distance value collected by the TOF sensor. If so, the door is controlled to open.

[0004] However, although the current optical foot sensor provides convenience for users, there are still problems with low foot movement recognition rate and false triggering in actual applications. For example, when the material difference between the user's shoe upper and the ground is large, the TOF sensor's measurement results may be inaccurate due to the impact on the reflectivity of the laser, and the TOF sensor may not be able to accurately identify the foot movement; or, the TOF sensor may mistakenly identify non-foot movement as foot movement, such as when a small animal passes through the light spot area or leaves fall into the light spot area, which may cause false triggering to open the car door. The above problems limit the application of optical foot sensors. Summary of the invention

[0005] The embodiments of the present disclosure provide a door control method, device, equipment and storage medium, which can solve the problems of low motion recognition rate and false triggering of door control sensors in actual applications to a certain extent.

[0006] In one aspect of an embodiment of the present disclosure, a vehicle door control method is provided, which is applied to a vehicle door control sensor. The method includes:

[0007] Collecting sensor data of the door control sensor, the sensor data including at least one of a distance measurement value, a signal strength value, and an environmental noise value; wherein the distance measurement value is used to indicate the distance between the door control sensor and a signal reflection surface, the signal reflection surface being a surface of an object that reflects a detection signal of the door control sensor;

[0008] Performing door control action recognition based on the sensor data to obtain a first recognition result of whether there is a door control action;

[0009] Determine whether the sensor data meets a corresponding preset threshold standard, and whether the duration of meeting the corresponding preset threshold standard is within a preset duration range, and obtain a second recognition result of whether there is a door control action;

[0010] Based on the first recognition result and the second recognition result, obtaining a fusion recognition result of whether there is a door control action;

[0011] In response to the fusion recognition result that there is a door control action, a door control signal is sent to a door controller so that the door controller controls the door to be opened or closed based on the door control signal and the current state of the door.

[0012] Optionally, collecting sensor data includes:

[0013] Transmitting the detection signal in a preset direction at a preset frequency and receiving a reflected signal of the detection signal;

[0014] Determine the distance measurement value based on the flight time between the detection signal and the reflection signal, the flight time being the time from the emission of the detection signal to the receipt of the reflection signal by the sensor;

[0015] Detecting a signal strength value of the received reflected signal;

[0016] The ambient noise value of the surrounding environment during a period when the reflected signal is not received is detected.

[0017] Optionally, the first recognition result includes a first confidence level and a first classification result for characterizing whether the door control action exists within the preset time period, and the second recognition result includes a second confidence level and a second classification result for characterizing whether the door control action exists within the preset time period;

[0018] The obtaining, based on the first recognition result and the second recognition result, a fusion recognition result of whether a door control action exists includes:

[0019] The first confidence level is used as the weight of the first classification result, and the second confidence level is used as the weight of the second classification result, and a weighted sum calculation is performed to obtain the fusion recognition result.

[0020] Optionally, the performing door control action recognition based on the sensor data to obtain a first recognition result of whether a door control action exists includes:

[0021] Using the input layer in the deep learning model to input the sensor data sequence collected within the preset time length;

[0022] Using a convolutional layer in a deep learning model to extract features from the sensor data sequence, to obtain a feature matrix of the sensor data sequence;

[0023] Using an activation function in a deep learning model to perform a nonlinear transformation on a feature matrix of the sensor data sequence;

[0024] Using a fully connected layer in a deep learning model to classify a feature matrix of the sensor data sequence after nonlinear transformation;

[0025] The first confidence level and the first classification result are outputted using an output layer in a deep learning model.

[0026] Optionally, determining whether the sensor data satisfies a corresponding preset threshold standard and whether a duration of satisfying the corresponding preset threshold standard is within a preset duration range, and obtaining a second recognition result of whether there is a door control action, includes:

[0027] The sensor data is collected using a sliding window, and whether the current sliding window is in a stable state is determined based on the range in the current sliding window, wherein the range refers to the difference between the maximum value of the sensor data and the minimum value of the sensor data, and the sliding window in the stable state is a sliding window whose range is less than a range threshold;

[0028] In response to the last sliding window being in an unstable state and the current sliding window being in the stable state, determining whether the sensor data in the current sliding window meets the preset threshold standard;

[0029] Determine a duration for which the sensor data is in the stable state and meets the preset threshold standard based on the number of sliding windows that are continuously in the stable state and meet the preset threshold standard starting from the current sliding window;

[0030] In response to the duration being within the preset duration range, obtaining a second classification result of detecting the door control action;

[0031] The second confidence level is calculated based on the distance threshold interval, the signal strength threshold interval, the noise value threshold interval, and the value of the sensor data.

[0032] Optionally, the determining whether the current sliding window is in a stable state based on the range in the sliding window includes:

[0033] Determine the signal-to-noise ratio corresponding to the sliding window based on the mean value of the signal strength and the mean value of the ambient noise in the current sliding window; determine the extreme difference threshold corresponding to the current sliding window based on the signal-to-noise ratio, wherein the extreme difference threshold is negatively correlated with the signal-to-noise ratio;

[0034] In response to the range being less than the range threshold, determining that the current sliding window is in the stable state;

[0035] In response to the range being greater than or equal to the range threshold, it is determined that the current sliding window is in the unstable state.

[0036] Optionally, the second confidence level is calculated as follows:

[0037] Second confidence level = (threshold interval endpoint difference - 2*|sensor data value - threshold interval median|) / threshold interval endpoint difference

[0038] Among them, the threshold interval includes at least one of the distance threshold interval, the signal strength threshold interval and the noise value threshold interval, the threshold interval endpoint difference is the absolute value of the difference between the numerical values ​​corresponding to the two endpoints of the threshold interval, and the threshold interval median is the numerical value corresponding to the middle point of the threshold interval.

[0039] Another aspect of the present disclosure provides a door control device, which is applied to a door control sensor, and includes:

[0040] an acquisition module, used for acquiring sensor data of the door control sensor, the sensor data including at least one of a distance measurement value, a signal strength value and an environmental noise value; wherein the distance measurement value is used to indicate the distance between the door control sensor and a signal reflection surface, the signal reflection surface being a surface of an object that reflects a detection signal of the sensor;

[0041] A first recognition module, configured to perform door control action recognition based on the sensor data to obtain a first recognition result of whether a door control action exists;

[0042] A second recognition module is used to determine whether the sensor data meets the corresponding preset threshold standard and whether the duration of meeting the corresponding preset threshold standard is within a preset duration range, and obtain a second recognition result of whether there is a door control action;

[0043] a fusion module, configured to obtain a fusion recognition result of whether there is a door control action based on the first recognition result and the second recognition result;

[0044] The control module is used to send a door control signal to a door controller in response to a fusion recognition result indicating the existence of a door control action, so that the door controller controls the door to be opened or closed based on the door control signal and the current state of the door.

[0045] Another aspect of the present disclosure provides an electronic device, including:

[0046] Memory for storing computer programs;

[0047] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method described in the above aspects is implemented.

[0048] Another aspect of the embodiments of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in the above aspects is implemented.

[0049] Another aspect of the embodiments of the present disclosure provides a computer program, including computer program instructions, which implement the methods described in the above aspects when executed by a processor.

[0050] Based on the embodiments of the present disclosure, by real-time acquisition of at least one sensor data including a distance measurement value, a signal strength value and an environmental noise value, a door control action is identified based on the sensor data, taking into account that the user's door control action will not only affect the distance measurement value but also the signal strength of the reflected signal and the intensity of the environmental noise; secondly, on the one hand, the door control action is identified based on the sensor data to obtain a first recognition result, and on the other hand, based on a preset threshold standard and time continuity, it is determined whether there is sensor data corresponding to the door control action to obtain a second recognition result, and the results of at least two recognition methods are combined to obtain a fused recognition result of the door control action, which can improve the reliability of the door control action recognition result, and to a certain extent solve the problems of low action recognition rate and false triggering of the door control sensor in actual applications, thereby improving the accuracy of door control.

[0051] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0053] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0054] Figure 1 A flow chart of an embodiment of the vehicle door control method disclosed herein;

[0055] Figure 2 A schematic diagram of the change of the detection data value of the preset door opening action over time in the embodiment of the present disclosure;

[0056] Figure 3 A schematic diagram of a model structure of a deep learning model in an embodiment of the present disclosure;

[0057] Figure 4 A schematic diagram of the structure of a door control sensor in an embodiment of the present disclosure;

[0058] Figure 5 A histogram generated by a photodetector in an embodiment of the present disclosure;

[0059] Figure 6 is a flow chart of a second classification result generation process in an embodiment of the present disclosure;

[0060] Figure 7 A schematic diagram of the state change of sensor data in an embodiment of the present disclosure;

[0061] Figure 8 It is a structural schematic diagram of an embodiment of the vehicle door control device disclosed in the present invention;

[0062] Fig. 9 The figure is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. DETAILED DESCRIPTION

[0063] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0064] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0065] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0066] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0067] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0068] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0069] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0070] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0071] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0072] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0073] The embodiment of the present disclosure can be applied to a door control sensor, which includes a main control module and a sensor module, and the main control module is connected to the sensor module. Among them, the sensor module is used to collect sensor data and send the sensor data to the main control module; the main control module is used to perform action recognition based on the sensor data. The main control module is also connected to the door controller of the vehicle, and is used to send a door opening signal to the door controller when a preset door opening action is detected, and the door controller controls the door to open in response to the door opening signal. Schematically, the door control sensor can adopt a capacitive sensor, an ultrasonic sensor, an infrared sensor, a lidar sensor, etc., and the present disclosure takes a TOF sensor as an example for explanation.

[0074] Please refer to Figure 1 , which shows a flow chart of a vehicle door control method provided by an exemplary embodiment of the present disclosure. The vehicle door control method of the embodiment of the present disclosure may be implemented by a vehicle door control sensor in a vehicle. The embodiment includes the following steps:

[0075] Step 101 : collecting sensor data of a door control sensor, where the sensor data includes at least one of a distance measurement value, a signal strength value, and an ambient noise value.

[0076] The distance measurement value is used to indicate the distance between the door control sensor and the signal reflection surface; the signal strength value is the signal strength of the reflection signal corresponding to the detection signal emitted by the sensor, and the environmental noise value is used to indicate the noise signal strength of the surrounding environment. The signal reflection surface is the surface of the object that reflects the detection signal of the door control sensor. For example, for a tailgate pedal sensor installed in the vehicle bumper, the signal reflection surface is the ground when no object enters the sensing area, the signal reflection surface is the user's foot or shoe surface when the user performs a door control action in the sensing area, and the signal reflection surface is the skin surface of the animal when an animal passes through the sensing area.

[0077] Optionally, the detection signal emitted by the door control sensor may be a light signal, and accordingly, the signal strength value is the signal strength of the reflected light, and the environmental noise value is the signal strength of the environmental light received by the door control sensor except the reflected light; or, the detection signal emitted by the door control sensor may be a sound signal, and accordingly, the signal strength value is the signal strength of the reflected sound wave, and the environmental noise value is the signal strength of the environmental sound received by the door control sensor except the reflected sound wave. The embodiment of the present application is described by taking a light signal as an example.

[0078] Taking the vehicle tailgate pedal sensor as an example, the door control action is a pedal action. When the user steps on the sensing area, the distance between the door control sensor and the foot surface is less than the distance between the door control sensor and the ground, that is, the distance measurement value will become smaller. However, the reliability of action recognition based only on the distance measurement value is low. For example, when the falling leaves pass through the sensing area and block the ground, they may be mistakenly recognized by the door control sensor as a door opening action and trigger the door to open by mistake, or when a small animal passes through the sensing area, it may also be mistakenly recognized by the door control sensor as a door opening action; for the door control sensor using TOF technology, the reflectivity of the laser on different surface materials varies greatly, resulting in a large difference in the reflected light received by the door control sensor, which affects the distance measurement result and makes the distance measurement inaccurate (for example, the distance between the door control sensor and the ground is 50cm, the ground is white, and the user's shoe upper is black. Since the signal strength of the laser reflected by the black shoe upper is less than the signal strength of the laser reflected by the white ground, the distance measurement result may still be close to 50cm), and the user's preset door opening action cannot be recognized. The embodiment of the present disclosure performs action recognition by combining distance measurement values, signal strength values, and environmental noise values, so that the action recognition result can be more accurate.

[0079] In a possible implementation, the door control sensor collects at least one of a distance measurement value, a signal strength value, and an ambient noise value in real time. For example, for a door control sensor of a car tailgate, the corresponding preset door opening / closing action is usually a foot-operated action, and the door control sensor is arranged at the rear of the car. Then, the distance between the surface of an object on the ground and the sensor can be measured as one of the identification bases for the preset door opening action, i.e., the distance measurement value. In addition, the door control sensor can emit a light beam in a preset direction (ground direction) at a certain frequency and sense the reflected light beam, measure the signal strength value, and measure the ambient light intensity to obtain the ambient noise value during the period when the reflected light beam is not received (during the period when the reflected light beams corresponding to two adjacent detection light beams reach the sensor).

[0080] Taking the TOF sensor as an example, the door control sensor continuously emits laser pulses toward the ground, and can calculate the distance measurement value based on the time from the pulse laser emission to the return to the door control sensor, detect the signal intensity value of the reflected laser, and detect the intensity of the ambient light other than the reflected laser as the ambient noise value. For different types of door control sensors, the sensor data collection method may be different, and the embodiments of the present disclosure do not limit the specific collection method of the distance measurement value and signal parameters.

[0081] Step 102 , performing door control action recognition based on the sensor data to obtain a first recognition result of whether there is a door control action.

[0082] In one possible implementation, a model with classification function, such as a deep learning model, a random forest model, or a decision tree model, may be used to recognize door control actions. The deep learning model may be pre-trained using sample sensor data carrying sample labels, where the sample labels are used to indicate "door control action detected" or "door control action not detected."

[0083] Optionally, the deep learning model can be constructed using convolutional neural networks (CNN), recurrent neural networks (RNN), deep neural networks (DNN), etc.

[0084] Step 103, determining whether the sensor data meets the corresponding preset threshold standard and whether the duration of meeting the corresponding preset threshold standard is within a preset duration range, and obtaining a second recognition result of whether there is a door control action.

[0085] In one possible implementation, the preset threshold standards include a distance threshold, a signal strength threshold, and an environmental noise threshold. When the value of the sensor data at a certain moment meets the distance threshold, the signal strength threshold, and the environmental noise threshold, it is determined that the sensor data at that moment meets the preset threshold standards, and the duration for which the subsequently collected sensor data meets the corresponding preset threshold standards is continued to be detected.

[0086] Optionally, the door control sensor can simultaneously detect whether the distance measurement value, signal strength value, and ambient noise value meet the corresponding preset threshold standards respectively; or the door control sensor can first detect whether one of the sensor data meets the corresponding preset threshold standard, and if not, directly obtain the second recognition result that there is no door control action at present, and if it meets, continue to determine whether the next sensor data meets the corresponding preset threshold standard. The embodiment of the present application does not limit the method for generating the second recognition result.

[0087] The disclosed embodiments take into account that the user's door control action usually lasts for a certain length of time (for example, for a foot-operated door opening action, the duration of the user's foot operation is longer than the duration of a small animal passing by, and shorter than the duration of an obstacle such as leaves blocking the door). Therefore, based on numerical detection, a time continuity check can be performed on the sensor data to determine whether the duration of the current action is within a reasonable duration range for the door control action.

[0088] Furthermore, since the user's posture usually remains unchanged while maintaining the door control action, that is, the corresponding sensor data should remain unchanged or fluctuate very little, in order to further improve the accuracy of the recognition result, the second recognition result can be obtained by determining whether the sensor data meets the corresponding preset threshold standard, and whether the duration of meeting the corresponding preset threshold standard and the value change being less than the change threshold is within the preset time range.

[0089] Optionally, the door control sensor can also perform a time continuity check based on the sensor data changes in five stages: before the action occurs, during the action start, during the action maintenance, during the action end, and after the action ends. Taking the foot pedal action as the preset door opening action as an example, Figure 3 The schematic diagram shows the change of detection data value over time during a user's preset door opening action. The detection data includes at least one of a distance measurement value and a signal parameter. Figure 2As shown, before the user performs the foot-stepping action, the detection data collected by the door control sensor conforms to the data when the sensor directly detects the ground (i.e., the ground is stable). When the user starts to perform the preset door opening action, data fluctuations will occur (i.e., the stepping process). The preset door opening action will be maintained for a certain period of time (i.e., the upper is stable). When the user ends the preset door opening action, data fluctuations will occur again (i.e., the lifting process). Then, the detection data received by the door control sensor will be restored to the value when the ground is directly detected (i.e., the ground is restored to stability). If the door control sensor data meets the preset threshold standard and the change characteristics meet the Figure 2 The features shown in FIG. 10A and FIG. 11B are used to identify the presence of a door control action.

[0090] Step 104 , based on the first recognition result and the second recognition result, a fusion recognition result of whether there is a door control action is obtained.

[0091] Optionally, one action recognition algorithm may be used to perform action recognition to obtain an action recognition result, or at least two action recognition algorithms may be used to perform action recognition, and the recognition results obtained by each algorithm may be combined to comprehensively generate an action recognition result.

[0092] Step 105 , in response to the fusion recognition result that there is a door control action, a door control signal is sent to a door controller, so that the door controller controls the door to be opened or closed based on the door control signal and the current state of the door.

[0093] When the fusion recognition result indicates that there is a door control action, the door control sensor sends a door control signal to the door controller. Accordingly, the door controller controls the door to open or close based on the door control signal and the current state of the door, for example, when the door control signal is received and the door is currently in a closed state, the door is opened, and when the door control signal is received and the door is currently in an open state, the door is closed.

[0094] Optionally, the door control sensor can be used for the tailgate of the vehicle, and other doors other than the tailgate of the vehicle. Other doors include the side doors on the left and right sides of the vehicle. Schematically, when the vehicle is a small car, the other doors include the four doors on the left and right sides of the small car, that is, the door of the passenger compartment and the driver's cab door. When the door control sensor is used for the tailgate, it can be installed on the side of the tailgate, such as being set on the rear bumper of the vehicle to control the tailgate; when used for other doors, it is set on the side of the other doors to control other doors. If used for the side door, it can be installed on the side of the side door. In a possible embodiment, multiple door control sensors can be set on the vehicle, respectively used to control different doors.

[0095] Based on the embodiments of the present disclosure, by real-time acquisition of at least one sensor data including a distance measurement value, a signal strength value and an environmental noise value, a door control action is identified based on the sensor data, taking into account that the user's door control action will not only affect the distance measurement value but also the signal strength of the reflected signal and the intensity of the environmental noise; secondly, on the one hand, the door control action is identified based on the sensor data to obtain a first recognition result, and on the other hand, based on a preset threshold standard and time continuity, it is determined whether there is sensor data corresponding to the door control action to obtain a second recognition result, and the results of at least two recognition methods are combined to obtain a fused recognition result of the door control action, which can improve the reliability of the door control action recognition result, and to a certain extent solve the problems of low action recognition rate and false triggering of the door control sensor in actual applications, thereby improving the accuracy of door control.

[0096] In a possible implementation manner, the door control sensor of the embodiment of the present disclosure adopts a TOF sensor. Figure 1 Based on the illustrated embodiment, step 101 may specifically include the following steps:

[0097] Step 101a: transmit a detection signal in a preset direction at a preset frequency and receive a reflection signal of the detection signal.

[0098] Step 101b, determining the distance measurement value based on the flight time between the detection signal and the reflection signal, where the flight time refers to the time from the emission of the detection signal to the sensor receiving the reflection signal.

[0099] Step 101c: Detect the signal strength value of the received reflected signal.

[0100] Step 101d: Detect the ambient noise value of the surrounding environment during the period when no reflected signal is received.

[0101] In one possible implementation, the sensor continuously emits a detection signal in a preset direction at a preset frequency, such as a laser pulse of a TOF sensor, and can determine a distance measurement value based on the flight duration of the detection signal, while detecting a signal strength value of a reflected signal corresponding to the detection light, and detecting an ambient noise value (for example, measuring the ambient noise value during the period of receiving reflected light from two adjacent detection lights).

[0102] Indicatively, Figure 4As shown, the door control sensor is provided with a control module 401, a laser 402, a transmitting optical system 403, a receiving optical system 404 and a photodetector 405, wherein the control module 401 is used to generate a trigger signal of the laser 402 in the working mode to trigger the laser 402 to generate laser light, and the laser light emitted by the laser 402 is shaped into a laser beam by the transmitting optical system 403. When the laser beam reaches the surface of the object, it will be reflected to generate a reflection signal, and the reflection signal will enter the receiving optical system 404 and be sensed by the photodetector 405. The photodetector 405 can output a histogram based on the input reflection signal, such as Figure 5 As shown, the peak value of the reflected signal in the histogram is the reflected signal intensity, the noise value in the signal-free area is the environmental noise value, the acquisition time of the reflected signal is the laser flight time, and the distance measurement value can be calculated based on the speed of light and the laser flight time. The control module 401 can obtain the distance measurement value and information such as the signal intensity and environmental noise value of the reflected signal by parsing the histogram. Schematically, the photodetector can be implemented using a single-photon avalanche diode (SPAD).

[0103] In a possible implementation, the first recognition result includes a first confidence level and a first classification result for characterizing whether a door control action exists within a preset time period, and the second recognition result includes a second confidence level and a second classification result for characterizing whether a door control action exists within a preset time period. Accordingly, the above step 104 may include the following steps:

[0104] The first confidence level is used as the weight of the first classification result, and the second confidence level is used as the weight of the second classification result, and a weighted sum calculation is performed to obtain a fusion recognition result.

[0105] Schematically, the first classification result includes a result score, and different result types correspond to different result scores. For example, the result types of the first classification result include door control action and invalid action, where the result score corresponding to the door control action is 1, and the result score corresponding to the invalid action is -1. The first confidence level is used to characterize the reliability of the first classification result, and can be calculated based on the numerical value of the sensor data and the various thresholds in the preset threshold standard. Correspondingly, different result types of the second classification result correspond to different result scores, and the second confidence level is used to characterize the reliability of the second classification result, which can be obtained through a deep learning model. The calculation formula of the weighted fusion score is as follows:

[0106] Fusion recognition result = first confidence level * first classification result + second confidence level * second classification result

[0107] For example, it can be set that when the value of the fusion recognition result is greater than or equal to 0.5, it is determined that the door control action is detected, and when it is less than 0.5, it is determined that the door control action is not detected.

[0108] The disclosed embodiment uses two recognition methods to perform action recognition as an example. In other possible implementations, more recognition methods can be used for action recognition, and the disclosed embodiment does not limit this. At least two different types of action recognition methods are used to perform action recognition based on sensor data, respectively, to obtain the classification results output by each action recognition method and the confidence level that can characterize the reliability of the results, and the recognition results are fused based on the classification results and the confidence level calculation, thereby further improving the reliability of the action recognition results and the accuracy of the door control.

[0109] In a possible implementation, a deep learning model can be used to perform door control action recognition based on sensor data to obtain a first recognition result. The deep learning model can perform action recognition by learning information such as the value of sensor data, the trend of value changes, and the correlation between different types of sensor data in various scenarios. The sensor can input the sensor data sequence collected within a preset time length into the model at one time to determine whether there is a door control action within the preset time length. The above step 102 may include the following steps:

[0110] Step 102a, using the input layer of the deep learning model to input the sensor data sequence collected within a preset time length.

[0111] Step 102b: Use the convolution layer of the deep learning model to extract features from the sensor data sequence to obtain a feature matrix of the sensor data sequence.

[0112] Step 102c: Use the activation function of the deep learning model to perform nonlinear transformation on the feature matrix of the sensor data sequence.

[0113] Step 102d, using the fully connected layer of the deep learning model to classify the feature matrix of the sensor data sequence after nonlinear transformation.

[0114] Step 102e, using the output layer of the deep learning model to output a first confidence level and a first classification result.

[0115] Indicative, Figure 3The architecture of a deep learning model is shown. The model includes an input layer, a convolution layer, an activation layer, a fully connected layer, and an output layer. First, the input layer is used to input a sensor data sequence collected within a preset time length (for example, 1 second), and then the convolution kernel in the convolution layer is used to extract features of the input sensor data sequence, and the feature matrix is ​​input into the activation layer after obtaining the feature matrix. The activation function in the activation layer, such as the S-shaped growth curve function (Sigmoid), the linear rectifier function (Rectified Linear Unit, ReLU), the normalized exponential function (Softmax), etc., is used to perform nonlinear transformation on the feature matrix, and then the fully connected layer is used to perform global feature integration and classification on the feature matrix after nonlinear transformation, and the first classification result "door control action detected" or "door control action not detected" and the first confidence are output through the output layer, wherein the first confidence can be generated by the activation function.

[0116] In another possible implementation, the sensor may sequentially input each group of sensor data sequences collected within a preset time length into the model to obtain a classification result and a confidence level corresponding to each group of sensor data, and then perform a weighted average calculation to obtain a first classification result and a first confidence level of the sensor data within the preset time length to determine whether there is a door control action within the preset time length. The above step 102 may include the following steps:

[0117] Step 102f, using the input layer of the deep learning model to input a set of sensor data collected within a preset time period.

[0118] Step 102g, using the convolutional layer of the deep learning model to extract features from a set of sensor data to obtain a feature matrix of the set of sensor data.

[0119] Step 102h, using the activation function of the deep learning model to perform nonlinear transformation on the feature matrix of a set of sensor data.

[0120] Step 102i uses the fully connected layer of the deep learning model to classify the feature matrix of a set of sensor data after nonlinear transformation.

[0121] Step 102j, using the output layer of the deep learning model to output a set of confidence and classification results corresponding to the sensor data.

[0122] Step 102k: Perform weighted average calculation based on the confidence and classification results of each group of sensor data collected within a preset time period to obtain a first confidence and a first classification result.

[0123] Among them, a set of sensor data includes distance measurement values, signal strength values ​​and environmental noise values ​​collected at a certain moment.

[0124] like Figure 3 As shown, first, a group of sensor data is input into the input layer, and then the convolution kernel in the convolution layer is used to extract features of the input group of sensor data, and the feature matrix is ​​input into the activation layer, and the activation function in the activation layer is used to perform nonlinear transformation on the feature matrix, and then the fully connected layer is used to perform global feature integration and classification on the feature matrix after nonlinear transformation, and the classification results and confidence of the group of sensor data are output through the output layer. Finally, the confidence and classification results of each group of sensor data collected within a preset time are weighted averaged to obtain the first confidence and the first classification result.

[0125] In a possible implementation, the second classification result may be determined based on a preset threshold criterion and the second confidence level may be calculated according to the sensor data and the preset threshold criterion.

[0126] Optional, Figure 6 A schematic diagram of a second classification result generation process is shown: 1. Numerical detection of distance measurement value, judging whether the distance measurement value passes the detection based on the distance threshold interval, if passed, executing step 2, if not passed, determining the second classification result as no door control action is detected; 2. Numerical detection of signal strength value, judging whether the reflected signal strength value passes the detection based on the signal strength threshold interval, if passed, executing step 3, if not passed, determining the second classification result as no door control action is detected; 3. Numerical detection of ambient noise, judging whether the ambient noise value passes the detection based on the noise value threshold interval in the preset conditions, if passed, executing step 4, if not passed, determining the second classification result as no door control action is detected; 4. Time continuity check, judging whether the time continuity check passes based on the change of sensor data, if passed, determining the second classification result as door control action is detected, if not passed, determining the second classification result as no door control action is detected.

[0127] Optionally, since there is usually an action change period and an action maintenance period when the user performs the door control action, the sensor data is in an unstable state and a stable state accordingly, such as Figure 7 As shown, it is also possible to first determine whether the sensor data is in a stable state, and if it is in a stable state, then determine whether the sensor data meets the preset threshold condition. The above step 103 may include the following steps:

[0128] Step 103a, using a sliding window to collect sensor data, and determining whether the current sliding window is in a stable state based on the range in the sliding window, wherein the range refers to the difference between the maximum value of the sensor data and the minimum value of the sensor data, and a sliding window in a stable state is a sliding window whose range is less than the range threshold.

[0129] Schematically, the data acquisition frequency of the door control sensor is 100Hz, and the sliding window contains L frames of data. Therefore, the stable state of the sensor data within the L*10ms duration can be determined through the sliding window. The stable state refers to the state in which the sensor data fluctuates less, that is, the distance between the signal reflection surface and the door control sensor is relatively stable and the signal reflection surface has not changed. Therefore, it is possible to determine whether the sensor data of the sliding window is in a stable state by setting the range threshold. If the range of the sensor data is less than the range threshold, it means that the sensor data fluctuates less and the current sliding window is in a stable state.

[0130] Furthermore, the range threshold can be dynamically adjusted according to the signal-to-noise ratio of the current sliding window. Step 103a can specifically include the following steps: Step 1, determining the signal-to-noise ratio corresponding to the sliding window based on the mean signal strength value and the mean ambient noise value in the current sliding window; Step 2, determining the range threshold corresponding to the current sliding window based on the signal-to-noise ratio, wherein the range threshold is negatively correlated with the signal-to-noise ratio; Step 3, in response to the range being less than the range threshold, determining that the current sliding window is in a stable state; Step 4, in response to the range being greater than or equal to the range threshold, determining that the current sliding window is in an unstable state.

[0131] In principle, the expression of the signal-to-noise ratio is as follows: log(S / N), where S is the mean signal strength in the sliding window (i.e., the average value of the signal strength values ​​of each frame in the sliding window), and N is the mean ambient noise in the sliding window (i.e., the average value of the ambient noise values ​​of each frame in the sliding window). When the signal-to-noise ratio is large, the sensor is less affected by the ambient noise and the ranging fluctuation is small. Setting a smaller extreme value threshold can improve the detection accuracy. When the signal-to-noise ratio is small, the sensor is greatly affected by the ambient noise and the ranging fluctuation is large. Appropriate relaxation of the restrictions can avoid false detection caused by data fluctuations. In principle, the calculation formula of the extreme value threshold is as follows: th1=K / log(S / N), where th1 is the extreme value threshold of the current sliding window, K is the proportional coefficient, and log(S / N) is the signal-to-noise ratio of the current sliding window.

[0132] Step 103b, in response to the previous sliding window being in an unstable state and the current sliding window being in a stable state, determining whether the sensor data in the current sliding window meets a preset threshold standard.

[0133] Step 103c, based on the number of sliding windows that are continuously in a stable state and meet a preset threshold standard starting from the current sliding window, determine the duration for which the sensor data is in a stable state and meets the preset threshold standard.

[0134] If the previous sliding window is in an unstable state and the current sliding window is in a stable state, it is possible that the user is performing a door control action. At this time, it is further determined whether the sensor data of the current sliding window meets the preset threshold standard. If it meets the preset threshold standard, the state of each subsequent sliding window is continuously determined and whether it meets the preset threshold standard until it enters an unstable state or does not meet the preset threshold standard, thereby obtaining the number of sliding windows that are continuously in a stable state and meet the preset threshold standard, and then based on the data collection time corresponding to the sliding window, determining the duration of the sensor data being in a stable state and meeting the preset threshold standard. Indicatively, the preset threshold standard may include but is not limited to: at least one of the distance measurement value belonging to a preset distance interval, the signal strength value belonging to a preset signal strength interval, and the environmental noise value belonging to a preset noise value interval.

[0135] Step 103d: in response to the duration being within a preset duration range, a second classification result of detecting the door control action is obtained.

[0136] If the sensor data is in a stable state and the duration for which it meets the preset threshold standard is within the preset time range, it is determined that the door control action is detected; if the sensor data is in an unstable state, or the sensor data does not meet the preset threshold standard, or the sensor data is in a stable state and the duration for which it meets the preset threshold standard is not within the preset time range, a second classification result is obtained that no door control action is detected.

[0137] Step 103e: Calculate a second confidence level based on the distance threshold interval, the signal strength threshold interval, the noise value threshold interval, and the value of the sensor data.

[0138] Optionally, the second confidence level is calculated as follows:

[0139] Second confidence level = (threshold interval endpoint difference - 2*|sensor data value - threshold interval median|) / threshold interval endpoint difference

[0140] Among them, the threshold interval includes at least one of the distance threshold interval, the signal strength threshold interval and the noise value threshold interval, the threshold interval endpoint difference is the absolute value of the difference between the values ​​corresponding to the two endpoints of the threshold interval, and the threshold interval median is the value corresponding to the middle point of the threshold interval.

[0141] For example, the preset threshold standard stipulates that the distance measurement value of the door control action is between 1 cm and 9 cm (including the endpoints), that is, the numerical interval of the distance measurement value is [1, 9], the difference between the numerical interval endpoints is 8, and the median of the numerical interval is 5. When the distance measurement value is 5 cm, the confidence level is 1; when the distance measurement value is 9 cm, the confidence level is 0; when the distance measurement value is 11 cm, the confidence level is 0.25, and the action score is -1.

[0142] Optionally, the confidence of each group of sensor data within a preset time period can be calculated, and then the average (or minimum value) of the confidence corresponding to all sensor data within the preset time period can be taken as the second confidence in the second recognition result. The confidence of each group of sensors can be calculated by averaging the confidence of the distance measurement value, the confidence of the signal strength value, and the confidence of the ambient noise value.

[0143] In the disclosed embodiment, the first confidence level and the second confidence level are calculated to respectively characterize the credibility of the first classification result and the second classification result, and a fusion calculation is performed based on the first confidence level, the second confidence level, the first classification result and the second classification result to obtain a fusion recognition result, thereby improving the reliability of the fusion recognition result, reducing the possibility of misidentification and improving the recognition rate of the sensor for the door control action.

[0144] Please refer to Figure 8 , which shows a structural block diagram of a door control device provided by an exemplary embodiment of the present disclosure. The door control device provided by this embodiment is applied to a door control sensor, and the device includes:

[0145] The acquisition module 801 is used to acquire sensor data of the door control sensor, wherein the sensor data includes but is not limited to at least one of a distance measurement value, a signal strength value, and an environmental noise value; wherein the distance measurement value is used to indicate the distance between the sensor and a signal reflection surface, and the signal reflection surface is an object surface that reflects the detection signal of the door control sensor;

[0146] A first recognition module 802 is used to recognize the door control action based on the sensor data collected by the collection module 801, and obtain a first recognition result of whether there is a door control action;

[0147] The second recognition module 803 is used to determine whether the sensor data collected by the collection module 801 meets the corresponding preset threshold standard and whether the duration of meeting the corresponding preset threshold standard is within a preset duration range, and obtain a second recognition result of whether there is a door control action;

[0148] A fusion module 804 is used to obtain a fusion recognition result of whether there is a door control action based on the first recognition result obtained by the first recognition module 802 and the second recognition result obtained by the second recognition module 803;

[0149] The control module 805 is used to send a door control signal to the door controller in response to the fusion recognition result of the existence of the door control action obtained by the fusion module 804, so that the door controller controls the door to be opened or closed based on the door control signal and the current state of the door.

[0150] Optionally, in a possible implementation manner, the acquisition module 801 may also be used to:

[0151] Transmitting a detection signal in a preset direction at a preset frequency and receiving a reflected signal of the detection signal;

[0152] Determining a distance measurement value based on a flight time between the detection signal and the reflected signal, the flight time being the time from when the detection signal is transmitted to when the reflected signal is received by the door control sensor;

[0153] Detecting the signal strength value of the received reflected signal;

[0154] The ambient noise value of the surrounding environment during the period when no reflected signal is received is detected.

[0155] Optionally, in a possible implementation manner, the first recognition result includes a first confidence level and a first classification result for characterizing whether a door control action exists within a preset time period, and the second recognition result includes a second confidence level and a second classification result for characterizing whether a door control action exists within a preset time period; the fusion module 804 of the embodiment of the present disclosure is further used to:

[0156] The first confidence level is used as the weight of the first classification result, and the second confidence level is used as the weight of the second classification result, and a weighted sum calculation is performed to obtain a fusion recognition result.

[0157] Optionally, in a possible implementation manner, the first identification module 802 of the embodiment of the present disclosure is further configured to:

[0158] Using the input layer in the deep learning model to input the sensor data sequence collected by the collection module 801 within a preset time length;

[0159] The convolutional layer in the deep learning model is used to extract features from the sensor data sequence to obtain the feature matrix of the sensor data sequence;

[0160] Use the activation function in the deep learning model to perform nonlinear transformation on the feature matrix of the sensor data sequence;

[0161] The fully connected layer in the deep learning model is used to classify the feature matrix of the sensor data sequence after nonlinear transformation;

[0162] The output layer in the deep learning model is used to output the first confidence and the first classification result.

[0163] Optionally, in another possible implementation manner, the first identification module 802 of the embodiment of the present disclosure is further configured to:

[0164] Using the input layer in the deep learning model to input a set of sensor data collected by the collection module 801 within a preset time period;

[0165] The convolutional layer in the deep learning model is used to extract features from a set of sensor data to obtain a feature matrix of a set of sensor data;

[0166] Use the activation function in the deep learning model to perform nonlinear transformation on the feature matrix of a set of sensor data;

[0167] The fully connected layer in the deep learning model is used to classify the feature matrix of a set of sensor data after nonlinear transformation;

[0168] Use the output layer in the deep learning model to output the confidence and classification results corresponding to a set of sensor data;

[0169] A weighted average calculation is performed based on the confidence and classification results of each group of sensor data collected by the collection module 801 within a preset time period to obtain a first confidence and a first classification result.

[0170] Optionally, in a possible implementation manner, the second identification module 803 of the embodiment of the present disclosure is further configured to:

[0171] The sensor data is collected by using a sliding window, and whether the current sliding window is in a stable state is determined based on the range in the current sliding window, wherein the range refers to the difference between the maximum value of the sensor data and the minimum value of the sensor data, and the sliding window in a stable state is a sliding window whose range is less than the range threshold;

[0172] In response to the last sliding window being in an unstable state and the current sliding window being in a stable state, determining whether the sensor data in the current sliding window meets a preset threshold standard;

[0173] Determine the duration of the sensor data being in a stable state and meeting the preset threshold standard based on the number of sliding windows that are continuously in a stable state and meet the preset threshold standard starting from the current sliding window;

[0174] In response to the duration being within a preset duration range, obtaining a second classification result of detecting the door control action;

[0175] A second confidence level is calculated based on the distance threshold interval, the signal strength threshold interval, the noise value threshold interval, and the value of the sensor data.

[0176] Optionally, in a possible implementation manner, the calculation formula for the second confidence level is as follows:

[0177] Second confidence level = (threshold interval endpoint difference - 2*|sensor data value - threshold interval median|) / threshold interval endpoint difference

[0178] Among them, the threshold interval includes at least one of the distance threshold interval, the signal strength threshold interval and the noise value threshold interval, the threshold interval endpoint difference is the absolute value of the difference between the values ​​corresponding to the two endpoints of the threshold interval, and the threshold interval median is the value corresponding to the middle point of the threshold interval.

[0179] Optionally, in a possible implementation manner, the second identification module 803 of the embodiment of the present disclosure is further configured to:

[0180] Determine a signal-to-noise ratio corresponding to the sliding window based on a mean value of signal strength in the current sliding window and a mean value of ambient noise;

[0181] Determine the range threshold corresponding to the current sliding window based on the signal-to-noise ratio, where the range threshold is negatively correlated with the signal-to-noise ratio;

[0182] In response to the range being less than the range threshold, determining that the current sliding window is in a stable state;

[0183] In response to the range being greater than or equal to the range threshold, it is determined that the current sliding window is in an unstable state.

[0184] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The parts of the same, similar or corresponding contents between the various embodiments can be referred to each other. Since the system, method, device, and equipment embodiments basically correspond to each other, the relevant parts can be referred to the description of the corresponding parts. The systems, methods, devices, and equipment of the embodiments of the present disclosure also correspond to each other in terms of specific implementation and beneficial technical effects. The relevant contents can be referred to each other and will not be repeated.

[0185] In addition, an embodiment of the present disclosure further provides an electronic device, including:

[0186] Memory for storing computer programs;

[0187] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the door control method described in any of the above embodiments of the present disclosure is implemented.

[0188] Fig. 9 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. Fig. 9 The electronic device according to the embodiment of the present disclosure is described. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signals from them.

[0189] like Fig. 9 As shown, the electronic device includes one or more processors and memory.

[0190] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0191] The memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement the door control method of each embodiment of the present disclosure described above and / or other desired functions.

[0192] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0193] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0194] The output device can output various information to the outside, including the determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0195] Of course, to simplify, Fig. 9 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0196] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the vehicle door control method according to various embodiments of the present disclosure described in the above part of this specification.

[0197] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0198] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the vehicle door control method according to various embodiments of the present disclosure described in the above part of this specification.

[0199] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0200] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0201] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0202] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0203] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0204] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0205] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0206] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0207] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A door control method, characterized in that: Applied to a door control sensor, the method comprises: Collecting sensor data of the door control sensor, the sensor data including at least one of a distance measurement value, a signal strength value, and an environmental noise value; wherein the distance measurement value is used to indicate the distance between the door control sensor and a signal reflection surface, the signal reflection surface being a surface of an object that reflects a detection signal of the door control sensor; Performing door control action recognition based on the sensor data to obtain a first recognition result of whether there is a door control action; Determine whether the sensor data meets a corresponding preset threshold standard, and whether the duration of meeting the corresponding preset threshold standard is within a preset duration range, and obtain a second recognition result of whether there is a door control action; Based on the first recognition result and the second recognition result, obtaining a fusion recognition result of whether there is a door control action; In response to the fusion recognition result that there is a door control action, a door control signal is sent to a door controller so that the door controller controls the door to be opened or closed based on the door control signal and the current state of the door.

2. The method according to claim 1, characterized in that The collecting of sensor data includes: Transmitting the detection signal in a preset direction at a preset frequency and receiving a reflected signal of the detection signal; determining the distance measurement value based on a flight time between the detection signal and the reflection signal, wherein the flight time refers to a time from when the detection signal is transmitted to when the reflection signal is received by the door control sensor; Detecting a signal strength value of the received reflected signal; The ambient noise value of the surrounding environment during a period when the reflected signal is not received is detected.

3. The method according to claim 1, characterized in that The first recognition result includes a first confidence level and a first classification result for characterizing whether the door control action exists within the preset time period, and the second recognition result includes a second confidence level and a second classification result for characterizing whether the door control action exists within the preset time period; The obtaining, based on the first recognition result and the second recognition result, a fusion recognition result of whether a door control action exists includes: The first confidence level is used as the weight of the first classification result, and the second confidence level is used as the weight of the second classification result, and a weighted sum calculation is performed to obtain the fusion recognition result.

4. The method according to claim 3, characterized in that: The performing of door control action recognition based on the sensor data to obtain a first recognition result of whether a door control action exists includes: Using the input layer in the deep learning model to input the sensor data sequence collected within the preset time length; Using the convolutional layer in the deep learning model to extract features from the sensor data sequence, to obtain a feature matrix of the sensor data sequence; Performing a nonlinear transformation on the feature matrix of the sensor data sequence using an activation function in the deep learning model; Using a fully connected layer in the deep learning model to classify the feature matrix of the sensor data sequence after nonlinear transformation; The first confidence level and the first classification result are outputted using an output layer in the deep learning model.

5. The method according to claim 3, characterized in that: The determining whether the sensor data satisfies a corresponding preset threshold standard and whether the duration of satisfying the corresponding preset threshold standard is within a preset duration range, and obtaining a second recognition result of whether there is a door control action, includes: The sensor data is collected using a sliding window, and whether the current sliding window is in a stable state is determined based on the range in the current sliding window, wherein the range refers to the difference between the maximum value of the sensor data and the minimum value of the sensor data, and the sliding window in the stable state is a sliding window whose range is less than a range threshold; In response to the last sliding window being in an unstable state and the current sliding window being in the stable state, determining whether the sensor data in the current sliding window meets the preset threshold standard; Determine a duration for which the sensor data is in the stable state and meets the preset threshold standard based on the number of sliding windows that are continuously in the stable state and meet the preset threshold standard starting from the current sliding window; In response to the duration being within the preset duration range, obtaining a second classification result of detecting the door control action; The second confidence level is calculated based on the distance threshold interval, the signal strength threshold interval, the noise value threshold interval, and the value of the sensor data.

6. The method according to claim 5, characterized in that The determining whether the current sliding window is in a stable state based on the range in the sliding window includes: Determine a signal-to-noise ratio corresponding to the sliding window based on a mean value of signal strength and a mean value of ambient noise in the current sliding window; Determine the range threshold corresponding to the current sliding window based on the signal-to-noise ratio, wherein the range threshold is negatively correlated with the signal-to-noise ratio; In response to the range being less than the range threshold, determining that the current sliding window is in the stable state; In response to the range being greater than or equal to the range threshold, it is determined that the current sliding window is in the unstable state.

7. The method according to claim 5, characterized in that The calculation formula of the second confidence level is as follows: Second confidence level = (threshold interval endpoint difference - 2*|sensor data value - threshold interval median|) / threshold interval endpoint difference Among them, the threshold interval includes at least one of the distance threshold interval, the signal strength threshold interval and the noise value threshold interval, the threshold interval endpoint difference is the absolute value of the difference between the numerical values ​​corresponding to the two endpoints of the threshold interval, and the threshold interval median is the numerical value corresponding to the middle point of the threshold interval.

8. A door control device, characterized in that: Applied to a door control sensor, the device comprises: an acquisition module, used for acquiring sensor data of the door control sensor, the sensor data including at least one of a distance measurement value, a signal strength value and an environmental noise value; wherein the distance measurement value is used to indicate the distance between the door control sensor and a signal reflection surface, the signal reflection surface being a surface of an object that reflects the detection signal of the door control sensor; A first recognition module, configured to perform door control action recognition based on the sensor data to obtain a first recognition result of whether a door control action exists; A second recognition module is used to determine whether the sensor data meets the corresponding preset threshold standard and whether the duration of meeting the corresponding preset threshold standard is within a preset duration range, and obtain a second recognition result of whether there is a door control action; a fusion module, configured to obtain a fusion recognition result of whether there is a door control action based on the first recognition result and the second recognition result; The control module is used to send a door control signal to a door controller in response to a fusion recognition result indicating the existence of a door control action, so that the door controller controls the door to be opened or closed based on the door control signal and the current state of the door.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method described in any one of claims 1 to 7 is implemented.

11. A computer program comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the method described in any one of claims 1 to 7 is implemented.

Images