Target detection method, device and equipment based on multi-sensor fusion and storage medium

Abstract

The invention relates to the technical field of target detection, in particular to a target detection method, device and equipment based on multi-sensor fusion and a storage medium, and the method comprises the steps: tracking a target in real time through a millimeter wave radar, a photosensitive sensor and an infrared sensor, and judging the state of the target. According to the invention, the method can achieve the recognition of environment interference through the fusion of the three types of sensor data, has more intelligent environment adaptability, has excellent adaptability in different weathers and different scenes, can obtain more comprehensive target information through the mutual complementation of the three types of sensor data, and improves the detection accuracy and reliability.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a target detection method, device, equipment and storage medium based on multi-sensor fusion. Background Art

[0002] Currently, personnel detection and positioning play an increasingly important role in modern management. In the field of security, firefighting or other rescue scenarios, personnel locations can be found in time to ensure personnel safety; in medical health and home environments, personnel dynamics can be monitored at any time to avoid potential risks and prevent accidents. Although there are currently a variety of personnel detection and positioning methods, there are still great limitations in actual use.

[0003] In the related technology, the existing systems basically rely on cameras for personnel detection and positioning, which is insufficient for places where privacy needs to be considered. At the same time, the camera has high requirements for the weather environment. For weather such as haze, rain, fog, and dust, the camera cannot penetrate, and the detection efficiency is reduced. Patent CN202410808139 "A portable security system" uses infrared sensors, which are greatly affected by heat sources. For outdoor scenes with high temperatures or indoor scenes with heating, it is impossible to distinguish between ambient temperature and human body temperature, which will lead to misjudgment or missed judgment.

[0004] Therefore, the present invention proposes a target detection method, device, equipment and storage medium based on multi-sensor fusion, which can protect the privacy of personnel while having strong applicability, no restrictions on usage scenarios, and high detection accuracy. Summary of the invention

[0005] The present application provides a target detection method, device and storage medium based on multi-sensor fusion to solve the problem that in the target detection and tracking technology in the related art, a single type of sensor is relied on to obtain the position information and motion state of the target, and the single sensor may be affected by environmental factors, resulting in reduced detection accuracy and reliability. For example, millimeter wave radar may have a blind spot when detecting a stationary target, and a photosensitive sensor cannot work effectively in a dark environment.

[0006] In a first aspect, the present application provides a target detection method based on multi-sensor fusion, the method comprising:

[0007] Receive the millimeter wave signal that is emitted into space by the millimeter wave radar and then reflected from the object;

[0008] Based on the millimeter wave signal, determining motion information of the target detected in the space; the motion information includes position information and speed information of the target in the space;

[0009] Based on the motion information, clustering is performed on the detected target, and dynamic estimation is performed based on the result of the clustering process to obtain tracking information of the target;

[0010] Based on the tracking information, determining the target that can be stably detected as a candidate tracking target;

[0011] Determining whether the candidate tracking targets include a stationary target that meets a first preset condition;

[0012] If not, output the position and motion state of the moving target in the candidate tracking target in the space;

[0013] If so, at least based on the photosensitive sensor, the environmental information in the space is collected, and based on the environmental information, the position and motion state of the stationary target that meets the first preset condition in the space are determined and output.

[0014] In some embodiments, collecting environmental information in the space at least based on a photosensitive sensor, and determining and outputting the position and motion state of the stationary target that meets the first preset condition in the space based on the environmental information, includes:

[0015] Collecting environmental information in the space based on the photosensitive sensor;

[0016] Based on the environmental information, determining whether to use an infrared sensor to perform temperature detection;

[0017] If not, outputting the position and motion state of the stationary target that meets the first preset condition in the space;

[0018] If so, the position and motion state of the stationary target that meets the first preset condition are updated based on the result of the temperature detection, and the updated position and motion state of the stationary target that meets the first preset condition in the space are output.

[0019] In some embodiments, determining whether to use an infrared sensor to perform temperature detection based on the environmental information includes:

[0020] Based on the environmental information, determining whether the current environment of the space is a dark environment;

[0021] If yes, it is determined to use the infrared sensor for temperature detection.

[0022] In some embodiments, the temperature detection using an infrared sensor includes:

[0023] collecting temperature information in the space based on the infrared sensor;

[0024] Based on the environmental information, determining whether there is a temperature change in the current environment of the space;

[0025] If so, based on the temperature information, an infrared detection result is determined, and the position and motion state of the stationary target that meets the first preset condition in the space are updated according to the infrared detection result.

[0026] In some embodiments, determining the infrared detection result based on the temperature information includes:

[0027] Determining the temperature value of the stationary target satisfying the first preset condition in different time periods;

[0028] determining whether a difference in the temperature values ​​at different time periods exceeds a temperature threshold;

[0029] Determine the stationary target that meets the first preset condition and does not exceed the temperature threshold as an abnormal target;

[0030] The position and the motion state of the stationary target that exceeds the temperature threshold and satisfies the first preset condition are determined.

[0031] In some embodiments, the method further comprises:

[0032] Determine the target without the tracking information and the abnormal target among the targets as false targets;

[0033] The position of the decoy target is added to the decoy target set.

[0034] In some embodiments, determining whether the candidate tracking targets include a stationary target that meets a first preset condition includes:

[0035] Determining a stationary target among the targets based on the tracking information of the targets;

[0036] Determine whether the distance between the position of the stationary target and the position of the pre-stored false target exceeds a distance threshold;

[0037] If so, it is considered that the stationary target meets the first preset condition.

[0038] In a second aspect, the present application provides a target detection device based on multi-sensor fusion, the device comprising:

[0039] A millimeter wave signal receiving module, used to receive the millimeter wave signal reflected from the object after being emitted into the space by the millimeter wave radar;

[0040] A motion information determination module, configured to determine motion information of a target detected in the space based on the millimeter wave signal; the motion information includes position information and speed information of the target in the space;

[0041] A tracking information determination module, configured to perform clustering processing on the detected target based on the motion information, and perform dynamic estimation based on the result of the clustering processing to obtain tracking information of the target;

[0042] A tracking target determination module, configured to determine the target that can be stably detected as a candidate tracking target based on the tracking information;

[0043] A stationary target determination module, used to determine whether the candidate tracking targets include a stationary target that meets a first preset condition;

[0044] The output module is used to output the position and motion state of the moving target among the candidate tracking targets in the space when the candidate tracking targets do not include a stationary target that meets the first preset condition; and, when the candidate tracking targets include a stationary target that meets the first preset condition, collect environmental information in the space based on at least a photosensitive sensor, and determine the position and motion state of the stationary target that meets the first preset condition in the space based on the environmental information and output it.

[0045] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0046] Memory, used to store computer programs;

[0047] The processor is used to implement the steps of the target detection method based on multi-sensor fusion described in any embodiment of the first aspect when executing the program stored in the memory.

[0048] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the target detection method based on multi-sensor fusion as described in any embodiment of the first aspect are implemented.

[0049] The above-mentioned technical solution provided by the embodiment of the present application has the following advantages compared with the prior art: the present invention performs target detection and positioning based on multi-sensor fusion, and comprehensively uses the data collected by photosensitive sensors, millimeter-wave radars and infrared sensors to perform target identification and tracking. It has strong applicability while protecting the privacy of personnel, and solves the limitations of traditional target detection solutions on usage scenarios. It not only has high detection accuracy, but also improves the reliability of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0052] Figure 1 A schematic diagram of a flow chart of a target detection method based on multi-sensor fusion provided in an embodiment of the present application;

[0053] Figure 2 A flowchart of another target detection method based on multi-sensor fusion provided in an embodiment of the present application;

[0054] Figure 3 A flowchart of another target detection method based on multi-sensor fusion provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0057] Figure 1 A flowchart of a target detection method based on multi-sensor fusion provided in an embodiment of the present application. Figure 1 The process shown can be executed by an electronic device (eg, a processor) or a target detection device based on multi-sensor fusion. Figure 1 As shown, process 100 may include the following operations.

[0058] Step 101, receiving a millimeter wave signal that is emitted into space by a millimeter wave radar and then reflected from an object.

[0059] Millimeter wave radar is a radar system that can detect objects using millimeter waves (frequency between 30GHz and 300GHz).

[0060] Millimeter wave signals refer to electromagnetic waves emitted by millimeter wave radars. When these electromagnetic waves encounter objects, they are reflected back and received by the radar. Objects are objects in space, which can include people, objects, etc.

[0061] In some embodiments, the millimeter wave radar system can emit millimeter wave signals, which propagate to objects in space and are reflected back. The antenna or sensor at the receiving end of the millimeter wave radar system captures the reflected signals to obtain millimeter wave signals.

[0062] In some embodiments of the present invention, the target mainly refers to a human object. In some other embodiments, the target may also be other moving objects, such as a robot.

[0063] Step 102: Determine motion information of the target detected in the space based on the millimeter wave signal.

[0064] The motion information includes the position information and speed information of the target in the space. The position information refers to the specific coordinate position of the target within the radar detection range. The speed information refers to the magnitude and direction of the speed of the target during movement.

[0065] In some embodiments, the existence and preliminary motion information of the target can be inferred based on the strength, phase, frequency shift, etc. of the received millimeter wave signal. For example, the received millimeter wave signal can be processed and the position and speed of the target can be calculated using algorithms such as the Doppler effect and the signal arrival time difference.

[0066] The millimeter wave radar signal can be an intermediate frequency signal obtained. Through FFT processing, Doppler processing, incoherent accumulation and DOA (direction of arrival) estimation, the distance, speed and angle information of the target in space can be obtained. Since the human body has breathing and heartbeat, the chest movement caused by it will cause the human body to move slightly, so it is only necessary to detect moving targets and micro-moving targets. Completely stationary targets can be considered as interference objects in the space, such as walls, buildings, tables and chairs, and will not be detected. Moving targets can be obtained through Doppler processing. For stationary people, the micro-motion features caused by breathing and heartbeat can be obtained by incoherent accumulation detection.

[0067] Step 103: clustering the detected target based on the motion information, and performing dynamic estimation based on the result of the clustering process to obtain tracking information of the target.

[0068] Clustering refers to grouping multiple detected targets according to their positions and speeds so that targets in the same group have similar motion characteristics. The motion information of targets in the same group may be the positions and speeds of the same target detected at different times.

[0069] Dynamic estimation refers to continuously updating and predicting the target's motion trajectory based on the clustering results and the target's motion information. The target's tracking information can include the target's real-time position, speed, and motion trajectory.

[0070] In some embodiments, a clustering algorithm (such as K-Means, DBSCAN, etc.) can be used to group multiple detection targets. Then, a dynamic estimation method such as a Kalman filter algorithm or a particle filter algorithm is applied to track the position changes of the target through continuous signal detection based on historical and current detection data, and infer its motion trajectory and speed.

[0071] In this embodiment, clustering is performed on targets with similar positions and velocities in space, and combined with the Kalman filter algorithm, data association, trajectory prediction and target tracking are performed on the acquired targets to obtain tracking information of the space targets.

[0072] Step 104: Based on the tracking information, the target that can be stably detected is determined as a candidate tracking target.

[0073] A candidate tracking target refers to a target that persists in multiple detections and has stable motion information.

[0074] In some embodiments, based on the results of multiple detections within a time window, targets that can be identified in each detection can be screened out, and after filtering out noise and occasional false signals, stably existing targets can be used as candidate tracking targets.

[0075] Tracking targets include the motion trajectory of moving targets. At the same time, for static micro-moving targets that can be detected continuously and stably, their static trajectories can also be tracked. For targets that appear occasionally and cannot be tracked stably, they may be false targets caused by interference. Only processing targets that are stably tracked can avoid misjudgment caused by interference.

[0076] Step 105: Determine whether the candidate tracking targets include a stationary target that meets a first preset condition.

[0077] A stationary target is one that moves almost nothing relative to the radar system or other objects in the detection space.

[0078] The first preset condition refers to a specific criterion for selecting a stationary target, such as a speed close to zero and a position that remains almost unchanged within a certain period of time.

[0079] In some embodiments, the speed information of the candidate target can be checked to determine whether it is stationary, for example, the speed is less than a certain threshold, or the positional relationship with other stationary objects has not changed.

[0080] When the candidate tracking targets do not include a stationary target that meets the first preset condition, the process proceeds to step 106 ; when the candidate tracking targets include a stationary target that meets the first preset condition, the process proceeds to step 107 .

[0081] Step 106: If not, output the position and motion state of the moving target in the candidate tracking target in the space.

[0082] A moving target is a target that is characterized by significant movement in space in the tracking information. Position and motion state refer to the real-time coordinates and velocity information of the target.

[0083] In some embodiments, when no stationary target meets the conditions, the system may directly output the positions and motion states of all moving targets.

[0084] The output information can be displayed on the monitoring interface or transmitted to other systems for further processing.

[0085] Step 107: If yes, collect environmental information in the space at least based on a photosensitive sensor, and based on the environmental information, determine and output the position and motion state of the stationary target that meets the first preset condition in the space.

[0086] A photosensitive sensor is a sensor used to detect light changes and measure environmental information such as brightness and color. Environmental information refers to data about light, color, etc. in a space collected by a photosensitive sensor.

[0087] By collecting ambient light information through photosensitive sensors and combining the ambient light information with millimeter-wave radar data, the position and status of stationary targets can be further determined.

[0088] The description of the photosensitive sensor and further determining the position and motion state of the stationary target that meets the first preset condition can be found in the following text, for example, Figure 2 and its related description.

[0089] In some embodiments of the present invention, by using millimeter wave radar, photosensitive sensor and infrared sensor, the target can be tracked in real time and the state of the target can be determined. At the same time, the fusion of the three sensor data can realize the identification of environmental interference, and the environmental interference can also be saved as a reference for subsequent detection, with more intelligent environmental adaptability and excellent adaptability in different weather and different scenes. The three sensor data complement each other to obtain more comprehensive target information, which improves the accuracy, reliability and stability of detection.

[0090] Figure 2 A flowchart of another target detection method based on multi-sensor fusion provided in an embodiment of the present application. Figure 2The process shown can be executed by an electronic device (eg, a processor) or a target detection device based on multi-sensor fusion. Figure 2 As shown, process 200 may include the following operations.

[0091] Step 201: Collect environmental information in the space based on the photosensitive sensor.

[0092] In some embodiments, light data may be continuously or periodically collected by a photosensitive sensor deployed in the detection space, and the collected light data may be digitally processed to generate characteristic information of the ambient light, that is, environmental information.

[0093] Step 202: Based on the environmental information, determine whether to use an infrared sensor to perform temperature detection.

[0094] An infrared sensor is a sensor that can detect infrared radiation and is used to measure the temperature of an object's surface.

[0095] Temperature detection refers to measuring the temperature of a target object or its surroundings through an infrared sensor.

[0096] In some embodiments, the environmental information collected by the photosensitive sensor can be analyzed to determine whether further temperature detection is required. For example, based on the environmental information, determining whether to use an infrared sensor for temperature detection includes: based on the environmental information, determining whether the current environment of the space is a dark environment; if so, determining to use an infrared sensor for temperature detection.

[0097] For example, if the light data shows that the space is a bright environment, the temperature detection of the infrared sensor may not be performed. If the light data shows that the space is a dark environment, it is determined that the temperature detection of the infrared sensor is required.

[0098] If it is determined that the infrared sensor is not needed for temperature detection, the position and motion state of the stationary target can be directly output based on the comprehensive data of the photosensitive sensor and the millimeter wave radar, that is, entering step 203. The output information may include the precise position coordinates of the stationary target and the confirmed stationary state.

[0099] The infrared sensor detects the temperature difference at different times. If there is no human movement, the temperature difference at different times does not change much; if there is human movement, the sensor detects the temperature difference between the human body temperature and the background temperature. Set a temperature threshold. If the temperature difference detected by the infrared sensor exceeds the temperature threshold, it is considered that there is a heat source in the area. Since the temperature of the space rises due to sunlight during the day, the space temperature is close to or even higher than the human body temperature, which will cause detection errors. Therefore, the infrared sensor can only work under conditions where the light sensor determines that the light is dim, such as at night or on rainy days. For indoor scenes, there may be air conditioners, heaters, electric heaters and other equipment, which will affect the detection results.

[0100] If it is determined that the infrared sensor is required to perform temperature detection, the process proceeds to step 204 .

[0101] Step 203: If not, output the position and motion state of the stationary target that meets the first preset condition in the space.

[0102] Step 204: If yes, then update the position and motion state of the stationary target that meets the first preset condition based on the result of the temperature detection, and output the updated position and motion state of the stationary target that meets the first preset condition in the space.

[0103] Updating refers to re-evaluating and confirming the position and motion status of a stationary target based on new temperature detection data.

[0104] The output updated stationary target information includes the newly determined position and motion state, ensuring that the system detects stationary targets more reliably and accurately.

[0105] In this embodiment, the infrared sensor is activated to collect temperature information of the stationary target or its surrounding area, and the temperature information is combined with the data of the photosensitive sensor and the millimeter wave radar for comprehensive analysis. For example, the temperature data can be used to confirm the existence of the target, exclude false targets, and further accurately locate the target, thereby improving the accuracy and reliability of target detection.

[0106] Figure 3 A flowchart of another target detection method based on multi-sensor fusion provided in an embodiment of the present application. Figure 3 The process shown can be executed by an electronic device (eg, a processor) or a target detection device based on multi-sensor fusion. Figure 3 As shown, process 300 may include the following operations.

[0107] Step 301: Collect temperature information in the space based on the infrared sensor.

[0108] Temperature information refers to the temperature data of each point in the space detected by the infrared sensor.

[0109] Step 302: Based on the environmental information, determine whether there is a temperature change in the current environment of the space.

[0110] Temperature change refers to the change in temperature of certain points in the current temperature information compared with the previously collected temperature information.

[0111] Step 303: If yes, determine the infrared detection result based on the temperature information, and update the position and motion state of the stationary target that meets the first preset condition in the space according to the infrared detection result.

[0112] Infrared detection results refer to the analysis results based on temperature information to determine the target location and movement status.

[0113] In some embodiments, the infrared detection result may be determined in the following manner.

[0114] Determining the temperature value of the stationary target satisfying the first preset condition in different time periods;

[0115] determining whether a difference in the temperature values ​​at different time periods exceeds a temperature threshold;

[0116] Determine the stationary target that meets the first preset condition and does not exceed the temperature threshold as an abnormal target;

[0117] The position and the motion state of the stationary target that exceeds the temperature threshold and satisfies the first preset condition are determined.

[0118] The temperature value refers to the temperature of the target object measured by the infrared sensor within a specific time period. The temperature threshold refers to the preset temperature change range, which is used to determine whether the temperature change is significant. An abnormal target is a stationary target whose temperature value does not exceed the temperature threshold, which may be a false alarm or a false target.

[0119] Exemplarily, the temperature value of the stationary target that meets the first preset condition can be collected and recorded within a preset time interval. The temperature change is calculated by comparing the temperature values ​​in different time periods. If the temperature change does not exceed the threshold, the stationary target is determined as an abnormal target. If the temperature change exceeds the threshold, the position and motion state of the stationary target are updated and confirmed.

[0120] In some embodiments, when it is determined that there is an abnormal target, the abnormal target can also be determined as a false target, and the position of the false target is added to the false target set. Similarly, when there is no target with tracking information among the targets detected in the space, the target can also be added to the false target set.

[0121] The false target can be used to determine whether the candidate tracking targets include a stationary target that meets the first preset condition. For example, the judgment can be made through the following operations.

[0122] Determining a stationary target among the targets based on the tracking information of the targets;

[0123] Determine whether the distance between the position of the stationary target and the position of the pre-stored false target exceeds a distance threshold. The distance threshold is a preset standard value for determining the distance between the stationary target and the false target. For each stationary target, the distance between its current position and each false target position in the pre-stored false target set is calculated, and it is determined whether these distances exceed the preset distance threshold.

[0124] If so, it is considered that the stationary target meets the first preset condition.

[0125] A false target is usually an object that has been analyzed or considered to be a false alarm or a non-real target (a detected object, such as a person, etc.), and its location is saved in the false target set.

[0126] The false target set may be a pre-set false target storage area for storing non-human false targets detected during the working process for interference elimination. In the initial state, the number of false targets in the storage area is 0.

[0127] In this embodiment, the system can accurately identify and confirm stationary targets that meet the first preset condition under various environmental conditions by combining multiple sensor data, and at the same time ensure the reliability and accuracy of the detection results through effective false target filtering.

[0128] For example, for detected stationary targets, since there may be trees, flowers, curtains, etc. in the space that move with the wind, in order to avoid the target being misjudged as a person, the final judgment can be more accurate by fusing the information of multiple sensors. If the stationary target detected by the radar is not detected by the infrared sensor, it can be considered that the target may be a false target caused by the shaking of branches, flowers, etc.

[0129] The embodiments of the present invention can bring the following beneficial effects:

[0130] Improved the detection accuracy and stability of personnel detection. The solution of using photosensitive sensors, millimeter-wave radars and infrared sensors to enhance anti-interference performance, solve the misjudgment caused by swaying branches and leaves, air conditioning, etc., and missed judgments caused by insufficient temperature difference in high temperature environments, improve detection accuracy, and enhance detection stability.

[0131] The limitations of traditional personnel detection solutions on usage scenarios are solved, and the system is more adaptable. Millimeter-wave radar can penetrate rain and fog, and is not affected by weather, light, etc., which solves the dependence of the camera solution on weather and light. At the same time, these three sensors will not cause privacy issues, and are more suitable for offices, indoors, or other places that focus on privacy. Through the data fusion of multiple sensors, environmental interference can be effectively identified, and it can be used outdoors, indoors, in green spaces, factories, and other places. The present invention has better universality.

[0132] Improved detection reliability. The three sensors acquire millimeter wave data, light data, and infrared data. The three types of data support and complement each other, improve data quality, actively adapt to environmental changes, reduce interference, and more comprehensively acquire detection target information, thereby achieving more reliable and intelligent personnel detection and positioning solutions.

[0133] like Figure 4 As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0134] Memory 113, used for storing computer programs;

[0135] In one embodiment of the present application, the processor 111 is used to execute the program stored in the memory 113 to implement the method for target detection based on multi-sensor fusion provided by any of the aforementioned method embodiments, including:

[0136] Receive the millimeter wave signal that is emitted into space by the millimeter wave radar and then reflected from the object;

[0137] Based on the millimeter wave signal, determining motion information of the target detected in the space; the motion information includes position information and speed information of the target in the space;

[0138] Based on the motion information, clustering is performed on the detected target, and dynamic estimation is performed based on the result of the clustering process to obtain tracking information of the target;

[0139] Based on the tracking information, determining the target that can be stably detected as a candidate tracking target;

[0140] Determining whether the candidate tracking targets include a stationary target that meets a first preset condition;

[0141] If not, output the position and motion state of the moving target in the candidate tracking target in the space;

[0142] If so, at least based on the photosensitive sensor, the environmental information in the space is collected, and based on the environmental information, the position and motion state of the stationary target that meets the first preset condition in the space are determined and output.

[0143] The present application also provides a target detection device based on multi-sensor fusion, the device comprising:

[0144] A millimeter wave signal receiving module, used to receive the millimeter wave signal reflected from the object after being emitted into the space by the millimeter wave radar;

[0145] A motion information determination module, configured to determine motion information of a target detected in the space based on the millimeter wave signal; the motion information includes position information and speed information of the target in the space;

[0146] A tracking information determination module, configured to perform clustering processing on the detected target based on the motion information, and perform dynamic estimation based on the result of the clustering processing to obtain tracking information of the target;

[0147] A tracking target determination module, configured to determine the target that can be stably detected as a candidate tracking target based on the tracking information;

[0148] A stationary target determination module, used to determine whether the candidate tracking targets include a stationary target that meets a first preset condition;

[0149] The output module is used to output the position and motion state of the moving target among the candidate tracking targets in the space when the candidate tracking targets do not include a stationary target that meets the first preset condition; and, when the candidate tracking targets include a stationary target that meets the first preset condition, collect environmental information in the space based on at least a photosensitive sensor, and determine the position and motion state of the stationary target that meets the first preset condition in the space based on the environmental information and output it.

[0150] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the target detection method based on multi-sensor fusion provided in any of the aforementioned method embodiments are implemented.

[0151] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0152] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A target detection method based on multi-sensor fusion, characterized in that: The method comprises: Receive the millimeter wave signal that is emitted into space by the millimeter wave radar and then reflected from the object; Based on the millimeter wave signal, determining motion information of the target detected in the space; the motion information includes position information and speed information of the target in the space; Based on the motion information, clustering is performed on the detected target, and dynamic estimation is performed based on the result of the clustering process to obtain tracking information of the target; Based on the tracking information, determining the target that can be stably detected as a candidate tracking target; Determining whether the candidate tracking targets include a stationary target that meets a first preset condition; If not, output the position and motion state of the moving target in the candidate tracking target in the space; If so, at least based on the photosensitive sensor, the environmental information in the space is collected, and based on the environmental information, the position and motion state of the stationary target that meets the first preset condition in the space are determined and output.

2. The method according to claim 1, characterized in that The collecting of environmental information in the space at least based on a photosensitive sensor, and determining and outputting the position and motion state of the stationary target that meets the first preset condition in the space based on the environmental information, include: Collecting environmental information in the space based on the photosensitive sensor; Based on the environmental information, determining whether to use an infrared sensor to perform temperature detection; If not, outputting the position and motion state of the stationary target that meets the first preset condition in the space; If so, the position and motion state of the stationary target that meets the first preset condition are updated based on the result of the temperature detection, and the updated position and motion state of the stationary target that meets the first preset condition in the space are output.

3. The method according to claim 2, characterized in that The determining whether to use an infrared sensor to perform temperature detection based on the environmental information includes: Based on the environmental information, determining whether the current environment of the space is a dark environment; If yes, it is determined to use the infrared sensor for temperature detection.

4. The method according to claim 2, characterized in that: The method of using an infrared sensor to detect temperature includes: collecting temperature information in the space based on the infrared sensor; Based on the environmental information, determining whether there is a temperature change in the current environment of the space; If so, based on the temperature information, an infrared detection result is determined, and the position and motion state of the stationary target that meets the first preset condition in the space are updated according to the infrared detection result.

5. The method according to claim 4, characterized in that The step of determining the infrared detection result based on the temperature information includes: Determining the temperature value of the stationary target satisfying the first preset condition in different time periods; determining whether a difference in the temperature values ​​at different time periods exceeds a temperature threshold; Determine the stationary target that meets the first preset condition and does not exceed the temperature threshold as an abnormal target; The position and the motion state of the stationary target that exceeds the temperature threshold and satisfies the first preset condition are determined.

6. The method according to claim 5, characterized in that The method further comprises: Determine the target without the tracking information and the abnormal target among the targets as false targets; The position of the decoy target is added to the decoy target set.

7. The method according to claim 1, characterized in that The determining whether the candidate tracking targets include a stationary target that meets a first preset condition includes: Determining a stationary target among the targets based on the tracking information of the targets; Determine whether the distance between the position of the stationary target and the position of the pre-stored false target exceeds a distance threshold; If so, it is considered that the stationary target meets the first preset condition.

8. A target detection device based on multi-sensor fusion, characterized in that: The device comprises: A millimeter wave signal receiving module, used to receive the millimeter wave signal reflected from the object after being emitted into the space by the millimeter wave radar; A motion information determination module, configured to determine motion information of a target detected in the space based on the millimeter wave signal; the motion information includes position information and speed information of the target in the space; A tracking information determination module, configured to perform clustering processing on the detected target based on the motion information, and perform dynamic estimation based on the result of the clustering processing to obtain tracking information of the target; A tracking target determination module, configured to determine the target that can be stably detected as a candidate tracking target based on the tracking information; A stationary target determination module, used to determine whether the candidate tracking targets include a stationary target that meets a first preset condition; The output module is used to output the position and motion state of the moving target among the candidate tracking targets in the space when the candidate tracking targets do not include a stationary target that meets the first preset condition; and, when the candidate tracking targets include a stationary target that meets the first preset condition, collect environmental information in the space based on at least a photosensitive sensor, and determine the position and motion state of the stationary target that meets the first preset condition in the space based on the environmental information and output it.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the target detection method based on multi-sensor fusion described in any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the target detection method based on multi-sensor fusion as described in any one of claims 1 to 7 are implemented.

Images