Door and window anti-intrusion intelligent early warning device integrating millimeter wave radar and visual identification

Abstract

The invention relates to a door and window anti-intrusion intelligent early warning device integrating millimeter wave radar and visual identification, and belongs to the technical field of intelligent doors and windows, the door and window anti-intrusion intelligent early warning device comprises a control module, a radar module and a visual module, the radar module and the visual module are electrically connected with the control module, and the radar module is used for scanning a detection area and uploading a radar scanning result to the control module; the control module judges whether to instruct the visual module to start according to a radar scanning result, the visual module is used for scanning a detection area when the visual module is started and uploading a visual scanning result to the control module, and the control module is used for judging whether to send out an early warning signal according to the radar scanning result and the visual scanning result.

Description

Technical Field

[0001] This invention belongs to the field of intelligent door and window technology, specifically relating to an intelligent early warning device for preventing intrusion into doors and windows that integrates millimeter-wave radar and visual recognition. Background Technology

[0002] As an essential component of housing, doors and windows are becoming increasingly intelligent and multifunctional with technological advancements to meet diverse needs, such as utilizing various sensors and alarms to achieve home security or intrusion prevention functions.

[0003] Traditional early warning alarm schemes determine whether an intruder has entered the room through a door or window based on changes in the movement state of the door or window. Therefore, any change in the movement state of the door or window will report to the anti-theft system and trigger an alarm, which often results in false alarms and makes it difficult to ensure the accuracy of door and window intrusion detection reporting. To address this, Chinese Patent CN116013009B discloses an IoT-based smart door and window alarm method, system, and readable storage medium. The method includes: collecting environmental information in an outdoor area; generating a first control command corresponding to the environmental information, and controlling the movement state of the smart door or window according to the first control command; determining whether the cumulative timing duration corresponding to the timing command is less than a preset timing threshold; if the cumulative timing duration is less than the preset timing threshold, determining whether the difference between the number of people in the security boundary to be generated and the number of people in the current security boundary is greater than a quantity threshold; if the difference is greater than the preset quantity threshold, stopping the updating of the security boundary, analyzing the movement speed of people within the current security boundary, and updating the movement state of the smart door or window and generating an alarm signal based on the analysis results. This technology enables the timely closure of smart doors and windows while ensuring the accuracy of intrusion detection, effectively increasing the difficulty of intrusion. The above solution only involves how to compare and determine whether a person can intrude after obtaining information about a stranger. However, in actual use, smart doors may be installed in various environments, and the usage environment, such as weather, is quite diverse. In some usage environments, the functional modules used to obtain information may be affected by the environment. If the functional modules used to obtain information are single-type sensors, their anti-interference ability is low. Therefore, the judgment of a single-type sensor may lead to misjudgment and inaccurate detection results. For this reason, a multi-sensor fusion, accurate detection fusion millimeter-wave radar and visual recognition smart door and window intrusion early warning device is needed. Summary of the Invention

[0004] To address the aforementioned problems in the prior art, the present invention provides a method with accurate detection capabilities.

[0005] The objective of this invention can be achieved through the following technical solutions: A smart early warning device for preventing intrusion into doors and windows, integrating millimeter-wave radar and visual recognition, includes a control module and a radar module and a vision module electrically connected to the control module. The radar module scans the detection area and uploads the radar scan results to the control module. The control module determines whether to instruct the vision module to start based on the radar scan results. The vision module scans the detection area when activated and uploads the visual scan results to the control module. The control module determines whether to issue an early warning signal based on the radar scan results and the visual scan results.

[0006] As a preferred embodiment of the present invention, the control module is used to extract radar trajectory matching degree and size data from the radar scanning results, and to determine whether the radar trajectory matching degree and size data are greater than a threshold. When both determinations are greater than the threshold, the vision module is instructed to start. The control module extracts visual contour matching degree from the visual scanning results. The control module is used to calculate the fusion confidence degree based on the visual contour matching degree, radar trajectory matching degree and size data, and to issue a warning signal when the fusion confidence degree exceeds a preset confidence degree threshold.

[0007] As a preferred technical solution of the present invention, the control module is used to calculate the fusion confidence level R based on the visual contour matching degree, radar trajectory matching degree and size data, where R=(w1×visual contour matching degree+w2×radar trajectory matching degree+w3×size data) / (w1+w2+w3).

[0008] As a preferred embodiment of the present invention, the control module is electrically connected to a light sensor and a visibility sensor. The light sensor is used to detect the ambient illuminance and upload the illuminance data to the control module. The visibility sensor is used to upload the visibility data to the control module. The control module is used to calculate the comprehensive environmental coefficient based on the illuminance data and the visibility data and determine whether the comprehensive environmental coefficient is lower than a threshold. If the determination result is yes, the value of w1 is decreased and the values ​​of w2 and w3 are increased. If the determination result is no, the value of w1 is increased and the values ​​of w2 and w3 are decreased.

[0009] As a preferred technical solution of the present invention, the control module is used to obtain the distribution location of user terminals and determine the proportion of user terminals located indoors to the total number of terminals. When the proportion exceeds a threshold, the value of w1 is corrected downwards, and when the proportion is higher than the threshold, the value of w1 is corrected upwards.

[0010] As a preferred embodiment of the present invention, it further includes a user terminal, which is communicatively connected to the control module. When the control module issues a warning signal, it simultaneously sends the warning information to the user terminal.

[0011] As a preferred embodiment of the present invention, the radar module includes a millimeter-wave radar, and the vision module includes a camera.

[0012] The beneficial effects of this invention are as follows: (1) By simultaneously setting up radar and vision modules to detect the detection area, and making the control module determine whether to issue a warning signal based on the radar scanning results and vision scanning results, compared with the single sensor scheme, the probability of both sensors being interfered with at the same time is lower. When one sensor is interfered with by the environment and the detection result is inaccurate, the other sensor can play a role in determining whether to issue a warning signal based on its own more accurate detection result, thereby improving the accuracy of the detection result. (2) Preferably, the control module extracts radar trajectory matching degree and size data from the radar scanning results and determines whether the radar trajectory matching degree and size data are greater than the threshold. When both judgments are greater than the threshold, the vision module is instructed to start and extract the visual contour matching degree from the visual scanning results. This allows the control module to calculate the fusion confidence degree based on the visual contour matching degree, radar trajectory matching degree and size data, and issue an early warning signal when the fusion confidence degree exceeds the preset confidence degree threshold. This completes the comprehensive judgment of whether to issue an early warning signal based on the three data from the two sensors, further improving the accuracy of the detection results. (3) By using the control module to calculate the fusion confidence R based on the weighted average of visual contour matching degree, radar trajectory matching degree and size data, i.e. R=(w1×visual contour matching degree+w2×radar trajectory matching degree+w3×size data) / (w1+w2+w3), the weight of the three types of data in the detection result can be flexibly adjusted according to the confidence of the three types of data, reducing the weight of data with lower confidence, and further improving the accuracy of the detection result; (4) By having the control module calculate the comprehensive environmental coefficient based on illuminance data and visibility data, and by reducing the value of w1 and increasing the values ​​of w2 and w3 when the comprehensive environmental coefficient is lower than the threshold, the weights in the fusion confidence calculation are automatically adjusted. At the same time, the weights of the visual scheme data are reduced when the surrounding environment is unfavorable to the visual scheme, and the weights of the visual scheme data are increased when the surrounding environment is favorable to the visual scheme. This completes the automatic adjustment of weights based on the influence of the environment on the confidence of the data, further improving the accuracy of the detection results. Attached Figure Description

[0013] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0014] Figure 1 This is a block diagram of the control loop of the present invention. Detailed Implementation

[0015] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0016] Please see Figure 1 A smart early warning device for preventing intrusion into doors and windows, which integrates millimeter-wave radar and visual recognition, includes a control module and a radar module and a vision module electrically connected to the control module. The radar module is used to scan the detection area and upload the radar scan results to the control module. The control module determines whether to instruct the vision module to start based on the radar scan results. The vision module is used to scan the detection area when activated and upload the visual scan results to the control module. The control module determines whether to issue an early warning signal based on the radar scan results and the visual scan results. Specifically, before installing the radar module, the space to be monitored is pre-defined, and this monitored space is the detection zone. Then, during installation, ensure that the two modules face the detection zone and can cover it, so that the detection zone is within the effective detection range of the radar and vision modules. The control module controls the start and stop of the radar module and the vision module; The radar module includes at least a millimeter-wave radar for transmitting millimeter waves and receiving target reflection signals, and the vision module includes at least a camera; In use, the radar module is mainly used to scan and acquire the size and trajectory of objects in the detection area, and upload the object size and trajectory data to the control module. The control module determines whether there is an abnormal moving target according to the preset intrusion judgment rules. At the same time, the vision module is used to acquire image information in the detection area and transmit the image data to the control module. The control module is used to identify the visual outline of the target based on the image information. By simultaneously setting up radar and vision modules to detect the detection area, and having the control module determine whether to issue a warning signal based on the radar and vision scanning results, the probability of both sensors being interfered with at the same time is lower compared to a single sensor solution. When one sensor is affected by environmental interference and the detection result is inaccurate, the other sensor can use its more accurate detection result to play a role in determining whether to issue a warning signal, thus improving the accuracy of the detection result. The control module is used to extract radar trajectory matching degree and size data from radar scan results, and to determine whether the radar trajectory matching degree and size data are greater than the threshold. When both judgments are greater than the threshold, the vision module is instructed to start. The control module extracts visual contour matching degree from visual scan results. The control module is used to calculate fusion confidence based on visual contour matching degree, radar trajectory matching degree and size data, and to issue an early warning signal when the fusion confidence exceeds the preset confidence threshold. In this embodiment, the radar trajectory matching degree and size data are each set with their own thresholds, and the fusion confidence degree corresponds to the pre-set fusion confidence threshold. In the process of calculating the fusion confidence, specifically, the control module is used to calculate the fusion confidence R based on the visual contour matching degree, radar trajectory matching degree and size data, where R = (w1 × visual contour matching degree + w2 × radar trajectory matching degree + w3 × size data) / (w1 + w2 + w3). Preferably, the control module extracts radar trajectory matching degree and size data from the radar scan results, and determines whether the radar trajectory matching degree and size data are greater than a threshold. When both determinations are greater than the threshold, the vision module is instructed to start and extract the visual contour matching degree from the visual scan results. This allows the control module to calculate the fusion confidence degree based on the visual contour matching degree, radar trajectory matching degree, and size data, and issue an early warning signal when the fusion confidence degree exceeds a preset confidence degree threshold. This completes the comprehensive judgment based on the three data from the two sensors to determine whether to issue an early warning signal, further improving the accuracy of the detection results. Meanwhile, by using the control module to calculate the fusion confidence level R based on the weighted average of visual contour matching degree, radar trajectory matching degree, and size data, i.e., R=(w1×visual contour matching degree+w2×radar trajectory matching degree+w3×size data) / (w1+w2+w3), the weight of the three types of data in the detection result can be flexibly adjusted according to the confidence level of the three types of data, reducing the weight of data with lower confidence level, and further improving the accuracy of the detection result.

[0017] In extreme cases, the human eye can replace optical sensors for judgment, but cannot replace radar. Furthermore, the weight of optical sensors needs to be reduced to avoid misjudgments. Therefore, the control module is electrically connected to a light sensor and a visibility sensor. The light sensor detects ambient light intensity and uploads the data to the control module, while the visibility sensor uploads the data to the control module. The control module calculates a comprehensive environmental coefficient based on the light and visibility data and determines whether the coefficient is below a comprehensive threshold. When the comprehensive environmental coefficient is below the threshold, it indicates that at least one of visibility or light intensity is poor, failing to meet visual recognition requirements. In this case, the comprehensive environmental coefficient is below the threshold, indicating an unreliable working environment for the optical sensors. The control module automatically increases the weight of radar trajectory matching and size data while decreasing the weight of visual contour matching. At this point, the control module decreases the value of w1 and increases the values ​​of w2 and w3. When the judgment result is negative, that is, when the comprehensive environmental coefficient is not lower than the comprehensive threshold, it indicates that the illumination and visibility are both within a suitable range, the vision module can work reliably, and the control module maintains the preset weights w1, w2 and w3 unchanged to ensure that the three types of data participate in the fusion calculation according to their original confidence levels. Optionally, when the lighting conditions are good and the visual information is highly reliable, the control module can further increase the value of w1 to enhance the contribution of visual contour matching degree in the calculation of fusion confidence. The control module has a pre-set excellent threshold for the comprehensive environmental coefficient. The excellent threshold is higher than the comprehensive threshold. When the comprehensive environmental coefficient is greater than the excellent threshold, the control module increases the value of w1 and decreases the values ​​of w2 and w3. By having the control module calculate the comprehensive environmental coefficient based on illuminance and visibility data, and by reducing the value of w1 and increasing the values ​​of w2 and w3 when the comprehensive environmental coefficient is lower than the comprehensive threshold, the weights in the fusion confidence calculation are automatically adjusted. At the same time, the weights of the visual scheme data are reduced when the surrounding environment is unfavorable to the visual scheme, and increased when the surrounding environment is favorable to the visual scheme. This completes the automatic adjustment of weights according to the impact of the environment on the data confidence, further improving the accuracy of the detection results. Since the typical usage environment, namely the home, has many small obstacles, the radar system is prone to misjudgment due to obstruction. Therefore, it is necessary to verify the size matching. To this end, the control module is also used to calculate the radar size matching degree based on the visual scanning results and the radar scanning results, determine whether the radar size matching degree is less than the threshold, and reduce the value of w3 when it is less than the threshold. Specifically, each time the control module receives the radar scan results and the visual scan results, it draws the radar point cloud outline and the target outline in the visual image, and calculates the radar size matching degree based on the outline overlap rate and size deviation. When the radar size matching degree is less than the preset threshold, the radar echo characteristics of the target object are significantly different from the known obstacle model. This indicates that the confidence of the size data is low, so the value of w3 is reduced. It also includes a user terminal, which is connected to the control module. When the control module issues a warning signal, it simultaneously sends the warning information to the user terminal.

[0018] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A smart early warning device for preventing intrusion into doors and windows that integrates millimeter-wave radar and visual recognition, characterized in that: The system includes a control module and a radar module and a vision module electrically connected to the control module. The radar module scans the detection area and uploads the radar scan results to the control module. The control module determines whether to instruct the vision module to start based on the radar scan results. The vision module scans the detection area when activated and uploads the visual scan results to the control module. The control module determines whether to issue a warning signal based on the radar scan results and the visual scan results.

2. The intelligent early warning device for door and window intrusion prevention that integrates millimeter-wave radar and visual recognition according to claim 1, characterized in that: The control module is used to extract radar trajectory matching degree and size data from radar scan results, and determine whether the radar trajectory matching degree and size data are greater than a threshold. When both judgments are greater than the threshold, the vision module is instructed to start. The control module extracts visual contour matching degree from visual scan results. The control module is used to calculate fusion confidence based on visual contour matching degree, radar trajectory matching degree and size data, and issue an early warning signal when the fusion confidence exceeds a preset confidence threshold.

3. The intelligent early warning device for door and window intrusion prevention that integrates millimeter-wave radar and visual recognition according to claim 2, characterized in that: The control module is used to calculate the fusion confidence level R based on the visual contour matching degree, radar trajectory matching degree and size data, where R = (w1 × visual contour matching degree + w2 × radar trajectory matching degree + w3 × size data) / (w1 + w2 + w3), and w1, w2 and w3 are pre-input weights.

4. The intelligent early warning device for door and window intrusion prevention that integrates millimeter-wave radar and visual recognition according to claim 3, characterized in that: The control module is electrically connected to a light sensor and a visibility sensor. The light sensor is used to detect the ambient light intensity and upload the light intensity data to the control module. The visibility sensor is used to upload the visibility data to the control module. The control module is used to calculate the comprehensive environmental coefficient based on the light intensity data and visibility data and determine whether the comprehensive environmental coefficient is lower than the threshold. If the determination result is yes, the value of w1 is decreased and the values ​​of w2 and w3 are increased.

5. The intelligent early warning device for door and window intrusion prevention that integrates millimeter-wave radar and visual recognition according to claim 4, characterized in that: The control module is used to obtain the distribution location of user terminals and determine the proportion of user terminals located indoors to the total number of terminals. When the proportion exceeds a threshold, the value of w1 is adjusted downwards; when the proportion is higher than the threshold, the value of w1 is adjusted upwards.

6. The intelligent early warning device for door and window intrusion prevention that integrates millimeter-wave radar and visual recognition according to claim 5, characterized in that: The control module is also used to calculate the radar size matching degree based on the visual scanning results and the radar scanning results, determine whether the radar size matching degree is less than the threshold, and reduce the value of w3 when it is less than the threshold.

7. The intelligent early warning device for door and window intrusion prevention that integrates millimeter-wave radar and visual recognition according to claim 6, characterized in that: It also includes a user terminal, which is communicatively connected to the control module. When the control module issues a warning signal, it simultaneously sends the warning information to the user terminal.

8. The intelligent early warning device for door and window intrusion prevention that integrates millimeter-wave radar and visual recognition according to claim 7, characterized in that: The radar module includes a millimeter-wave radar, and the vision module includes a camera.

Images