Door and window abnormal movement monitoring method and device based on millimeter wave radar and electronic equipment

Abstract

The invention discloses a door and window abnormal movement monitoring method and device based on a millimeter wave radar and electronic equipment, and relates to the technical field of radar target detection, and the method comprises the steps: employing the millimeter wave radar to transmit a frequency modulation continuous wave (FMCW) signal to a target door and window, and obtaining a reflection signal reflected by the target door and window; determining an intermediate frequency signal related to the signal propagation distance based on the reflected signal and the FMCW signal; determining motion state information of the target door and window according to the intermediate frequency signal; according to the motion state information, point cloud information representing the positions of the target door and window at multiple moments is calculated; constructing a door and window plane model of the target door and window according to the point cloud information, and determining opening and closing state information corresponding to the door and window plane model; determining monitoring parameters and parameter thresholds of door and window types corresponding to the target doors and windows from the opening and closing state information; and based on a comparison result of the monitoring parameter and the parameter threshold value, determining whether the target door and window has abnormal movement so as to realize monitoring of the abnormal movement of the door and window.

Description

Technical Field

[0001] The present invention relates to the field of radar target detection technology, and in particular to a method, device and electronic equipment for monitoring door and window movement based on millimeter wave radar. Background Art

[0002] With the development of science and technology and the improvement of people's living standards, smart home products have penetrated into all aspects of family life. As an important part of the smart home security system, the door and window movement monitoring device can provide rapid response and alarm for family doors and windows in emergency situations to ensure the safety of family residents. At present, the common door and window movement monitoring devices on the market are mainly divided into three types: magnetic suction type, laser beam type and infrared grating type. Among them, the magnetic suction sensor door and window movement monitoring device is the mainstream product in the market. This type of monitoring device is small in size, simple in structure, low in cost, and can be connected to the smart home system, but it only has the function of monitoring the switch status of doors and windows, and has no monitoring ability for abnormal vibration of doors and windows and invasive behavior by destroying glass. On the other hand, the monitoring equipment based on laser beam and infrared grating uses multiple pairs of laser / infrared based transmitting / receiving devices to monitor the switch status of doors and windows and external intrusion conditions at multiple points, but it should also be noted that the coverage range of such equipment is fixed, there are monitoring blind spots, poor environmental adaptability, and it cannot be used to monitor abnormal vibration conditions of doors and windows that may cause safety risks. Summary of the invention

[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, one object of the present invention is to provide a method, device and electronic device for monitoring door and window movement based on millimeter wave radar to achieve monitoring of door and window movement.

[0004] According to a first aspect of an embodiment of the present invention, a method for monitoring door and window movement based on millimeter wave radar is provided, the method comprising:

[0005] A millimeter wave radar is used to send a frequency modulated continuous wave (FMCW) signal to a target door or window, and a reflected signal reflected by the target door or window is obtained;

[0006] Determine an intermediate frequency signal related to a signal propagation distance based on the reflected signal and the FMCW signal;

[0007] Determine the motion state information of the target door and window according to the intermediate frequency signal;

[0008] Calculating point cloud information representing the locations of the target doors and windows at multiple moments according to the motion state information;

[0009] Constructing a door and window plane model of the target door and window according to the point cloud information, and determining switch state information corresponding to the door and window plane model;

[0010] Determining monitoring parameters and parameter thresholds corresponding to the door and window type of the target door and window from the switch state information;

[0011] Based on the comparison result between the monitoring parameter and the parameter threshold, it is determined whether there is any abnormal movement of the target door or window.

[0012] According to a second aspect of an embodiment of the present invention, a door and window movement monitoring device based on millimeter wave radar is provided, and the device includes:

[0013] A millimeter wave radar module is used to send a frequency modulated continuous wave (FMCW) signal to a target door or window using a millimeter wave radar, and obtain a reflected signal reflected by the target door or window; determine an intermediate frequency signal related to a signal propagation distance based on the reflected signal and the FMCW signal; determine motion state information of the target door or window according to the intermediate frequency signal; and calculate point cloud information representing the location of the target door or window at multiple times according to the motion state information;

[0014] A target tracking module constructs a door and window plane model of the target door and window according to the point cloud information, and determines the switch state information corresponding to the door and window plane model;

[0015] The target alarm module is used to determine the monitoring parameters and parameter thresholds of the door and window types corresponding to the target doors and windows from the switch status information; and determine whether there is any abnormal movement of the target doors and windows based on the comparison results of the monitoring parameters and the parameter thresholds.

[0016] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned method for monitoring abnormal movement of doors and windows based on millimeter-wave radar is implemented.

[0017] In the solution provided by the embodiment of the present invention, a millimeter wave radar is used to transmit an FMCW signal and receive a reflected signal to obtain an intermediate frequency signal, so that the motion state information can be determined and the point cloud information can be calculated, and the position changes and change amplitudes of each target point of the target door and window at different times can be accurately captured, and the opening state information can be determined. By comparing with the parameter threshold of the corresponding door and window type, it is determined whether the change state of the door and window opening and closing process is consistent with this door and window type, so as to monitor whether there is any abnormal movement. In this way, when an abnormal movement is detected, an alarm can be issued to the user, thereby realizing the monitoring of the abnormal movement of doors and windows and improving the safety of the home environment.

[0018] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of a method for monitoring abnormal movement of doors and windows based on millimeter wave radar provided by an embodiment of the present invention;

[0020] Figure 2 It is a structural schematic diagram of a door and window movement monitoring device based on millimeter wave radar provided by an embodiment of the present invention;

[0021] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0023] The following describes a method, device and electronic device for monitoring door and window movement based on millimeter wave radar according to an embodiment of the present invention with reference to the accompanying drawings.

[0024] In one embodiment of the present invention, see Figure 1 , provides a door and window movement monitoring method based on millimeter wave radar, including the following steps S101-S107:

[0025] S101: using a millimeter wave radar to send a frequency modulated continuous wave FMCW signal to a target door or window, and obtaining a reflected signal reflected by the target door or window;

[0026] S102: Determine an intermediate frequency signal related to a signal propagation distance based on the reflected signal and the FMCW signal;

[0027] S103: Determine the motion state information of the target door and window according to the intermediate frequency signal;

[0028] S104: Calculating point cloud information representing the locations of the target doors and windows at multiple times according to the motion state information;

[0029] S105: constructing a door and window plane model of the target door and window according to the point cloud information, and determining the switch state information corresponding to the door and window plane model;

[0030] S106: Determine monitoring parameters and parameter thresholds of the door and window type corresponding to the target door and window from the switch state information;

[0031] S107: Determine whether there is any abnormal movement in the target door or window based on the comparison result between the monitoring parameter and the parameter threshold.

[0032] Specifically, the RF front-end module transmits a modulated FMCW (Frequency Modulated Continuous Wave) signal to the target doors and windows through the antenna array. The FMCW signal propagates to the monitored object, i.e., the target doors and windows, and then reflects and is received by the antenna array again. The reflected signal refers to the signal received by the antenna array.

[0033] Before implementing the above-mentioned door and window movement monitoring method based on millimeter wave radar, the signal transceiver of the millimeter wave radar can be deployed in advance at a fixed point where the target door and window can be monitored. In this way, the target door and window can be located through the FMCW signal and the reflected signal. Specifically, the intermediate frequency signal can be a signal obtained by mixing the FMCW signal and the reflected signal. The specific mixing method is shown in the subsequent embodiments. Therefore, the intermediate frequency signal carries information related to the transmission distance of the FMCW signal and the reflected signal.

[0034] In step S103, after the intermediate frequency signal is obtained, the motion state information is determined according to the intermediate frequency signal, including information on distance, angle, speed, etc. When monitoring the target doors and windows, other objects other than the doors and windows may also contact the FMCW signal and generate a reflection signal. At this time, the distance measurement of the doors and windows will be interfered by other objects, and accordingly, there will be noise in the intermediate frequency signal.

[0035] Therefore, in one embodiment, the intermediate frequency signal can be filtered out and sampled by ADC (Analog-to-Digital Converter) to obtain a digital intermediate frequency signal corresponding to the input analog signal, that is, a digital signal corresponding to the intermediate frequency signal, and the parameter values ​​in the digital signal can be used for calculation.

[0036] In one embodiment of the present invention, the motion state information includes distance information, angle information and speed information; wherein,

[0037] The distance information is determined as follows:

[0038]

[0039] Among them, S c,n represents the distance between the target point on the target door or window and the millimeter wave radar; c represents the propagation speed of the electromagnetic wave signal; f c,n represents the center frequency of each frequency modulation cycle within n frequency modulation cycles; T c represents the signal frequency modulation period; B represents the system bandwidth;

[0040] The angle information is determined as follows:

[0041]

[0042] Wherein, θ represents the receiving and transmitting angle of the FMCW signal; λ represents the wavelength of the FMCW signal; represents the phase difference between the FMCW signal and the reflected signal; d represents the distance between the target point and the millimeter-wave radar.

[0043] A target point represents the actual position on a target door or window. By selecting multiple target points, a point cloud corresponding to the target door or window can be formed. Correspondingly, the position of each target point can be recorded in the point cloud information, and according to the above calculation formula, the distance information and angle information of each target point can also be recorded. Among them, the receiving and transmitting angle can be the angle formed by the direction of receiving and transmitting FMCW signals and the ground plane of the house where the target door or window is located.

[0044] Distance information can be obtained in the above process. Accordingly, the distance information at multiple moments can be used to calculate the moving speed of different target points to form speed information, which may include the speed in a straight line direction or the angular velocity of the target door or window rotating along the axis.

[0045] For the subset of point cloud data corresponding to the actual doors and windows on site, it is necessary to establish the corresponding relationship between different time periods, that is, to form matching point cloud information at the positions of the point clouds collected at each time point to achieve tracking of the target doors and windows.

[0046] From the above, we can get the position of the target point, and the distance and angle can represent the coordinates of the target point:

[0047]

[0048] in, represents the distance of the i-th target point in the point cloud at time t in the radar coordinate system, It represents the transmitting and receiving angle of the i-th target point in the point cloud at time t in the radar coordinate system.

[0049] In addition, the coordinates of the target points can be converted to the preset world coordinate system to form the point cloud information of each target point. The coordinates in the world coordinate system are:

[0050]

[0051] in, and They are the positions of the i-th target point at time t along the X-axis, Y-axis and Z-axis in the world coordinate system.

[0052] The calculated original point cloud image may have some singular values ​​caused by mismatch or sensor noise. For this, a preprocessing operation may be performed first, that is, noise and outliers may be removed. Specifically, a radius-based filtering method may be used, as described in the subsequent embodiments.

[0053] Since the target doors and windows can be approximately regarded as planes in the world coordinate system, the door and window plane model can be established by selecting three target points and making the three points coplanar. In this case, the geometric features of the plane can be used to eliminate points irrelevant to the target doors and windows again, and the point cloud data that conforms to the geometric features of the plane can be segmented to obtain a point cloud data subset including the target doors and windows, that is, the target inner point set calculated in the following embodiment, so as to determine the state information based on the target point information of the inner point of the plane model. For details, please refer to the subsequent embodiments, which will not be described in detail here.

[0054] Similarly, this method can also be used to monitor flat targets such as furniture and walls.

[0055] The specific implementation method of selecting the target point can refer to the following embodiment.

[0056] The switch status information refers to the status information of the change state caused by the switch of the target door or window, which may include the following information:

[0057]

[0058] Among them, (A i (t),B i (t),C i (t)) is the i-th point cloud data subset, corresponding to the plane model parameters at time t, and each point cloud data subset is formed in the cyclic calculation of the following embodiments. Plane model parameters refer to the coefficients of the equation of the plane model, because the equation of the plane model can be expressed as Ax+By+Cz+D=0; or, when D≠0, the left side of the equation is divided by D and converted to the form of Ax+By+Cz+1=0, A, B, C indicate the existence of coefficients, but do not refer to specific values, that is, A in Ax+By+Cz+1=0 and A in Ax+By+Cz+D=0 can take different values; A i (t),B i (t),C i (t) corresponds to the parameters A, B, and C of the equation.

[0059] Randomly selecting three target points from the point cloud data subset can form a plane model, thereby obtaining the plane model parameters.

[0060] is the change in the coordinate position of the center point of the plane model of the i-th point cloud data subset at time t.

[0061] is the change value of the center point position of the plane model of the i-th point cloud data subset at time t, which can be used to track the status of the push-pull switches of doors and windows.

[0062] It is the change value of the plane model's angle around the three axes at time t for the i-th point cloud data subset, which is used to track the rotation of doors and windows.

[0063] Because the point cloud information includes the position of each target point at different times, the above change value, that is, the difference in angle and position, can be determined according to the change of the target point at different times, for example It is the angular difference between time t and the previous observation time t.

[0064] Different types of doors and windows have different switching and rotation methods, so the monitoring parameters and corresponding parameter threshold settings are different. According to the opening method, they are divided into push-pull type, casement type, and fixed type.

[0065] For sliding doors and windows, there will be no large changes in angle during the opening and closing process, that is, the angle of the door and window plane model corresponding to the sliding doors and windows will be at a smaller value at different time points, and the point cloud distribution in the point cloud information corresponding to the sliding doors and windows will show regular changes; while casement doors and windows usually rotate around a fixed axis in a larger range during the opening and closing process, and the door and window plane models corresponding to these doors and windows will intersect on the same straight line at different time points; fixed doors and windows cannot be opened and closed, so the range of change in any direction or angle is smaller.

[0066] After comparing the door and window types, it can be determined whether there is any abnormality. For details, please refer to the subsequent embodiments.

[0067] It can be seen from the above that according to the solution provided by the embodiment of the present invention, a millimeter wave radar is used to transmit an FMCW signal and receive a reflected signal to obtain an intermediate frequency signal, so that the motion state information can be determined and the point cloud information can be calculated, and the position changes and change amplitudes of each target point of the target door and window at different times can be accurately captured, and the opening state information can be determined. By comparing with the parameter threshold of the corresponding door and window type, it is determined whether the change state of the door and window opening and closing process is consistent with this door and window type, so as to monitor whether there is any abnormal movement. In this way, when an abnormal movement is detected, an alarm can be issued to the user, thereby realizing the monitoring of the abnormal movement of doors and windows and improving the safety of the home environment.

[0068] The specific implementation method of determining the intermediate frequency signal in step S102 is described below.

[0069] In one embodiment of the present invention, the reflected signal and the FMCW signal may be mixed to obtain a mixed signal; and the mixed signal may be low-pass filtered to obtain an intermediate frequency signal.

[0070] Specifically, low-pass filtering refers to a filtering method that sets rules so that low-frequency signals can pass normally, while high-frequency signals that exceed a set critical value are blocked and weakened.

[0071] In one embodiment of the present invention, the mixed signal is obtained in the following manner:

[0072] u m (t) = u s (t)·u e (t)

[0073] Among them, u m (t) represents the amplitude of the mixing signal, u s (t) represents the amplitude of the FMCW signal, u e (t) represents the amplitude of the reflected signal;

[0074] The amplitude of the FMCW signal is determined as follows:

[0075]

[0076] Where f0 is the starting frequency of the FMCW signal, T is the signal frequency modulation time, T c is the signal frequency modulation period, B is the system bandwidth, j is the imaginary unit, k is the preset coefficient, and t represents time.

[0077] The specific implementation of the radius-based filtering method is described below.

[0078] In one embodiment of the present invention, for any target point in the point cloud information, whether the target point is a singular value point is determined based on the number of adjacent target points within a preset radius of the target point.

[0079] If the target point is a singular value point, the target point information of the target point is deleted from the point cloud information.

[0080] Specifically, a quantity threshold may be set. If the number of adjacent target points within a preset radius is less than the quantity threshold, the point is determined to be a singular value point, that is, it is highly likely to be a target point generated by noise or an outlier point.

[0081] The target point information is the relevant information of the target point in the point cloud information, including its position, angle, distance, etc., which is equivalent to removing outliers and noise points from the point cloud information.

[0082] In one embodiment of the present invention, constructing a door and window plane model of a target door and window according to point cloud information includes:

[0083] Extracting target points from the point cloud information to establish a first plane model;

[0084] Determine a first inner point set according to the distance relationship between each target point in the point cloud information and the first plane model;

[0085] Extracting target points from the first interior point set to establish a second plane model, and updating the first interior point set according to the distance relationship between each target point in the first interior point set and the second plane model;

[0086] The door and window plane model of the target door and window is determined based on the updated first inner point set.

[0087] Specifically, the calculation is performed using a clustering method based on RANSAC (Random Sample Consensus), as shown below.

[0088] For example, the position information of three target points in the target internal point set may be randomly extracted to establish the first plane model;

[0089] Calculating first distances between other target points in the point cloud information except the three extracted target points and the first plane model, and determining a first inner point set relative to the first plane model according to the first distances;

[0090] Furthermore, a first ratio of the number of inliers in the first inlier point set to the number of target points in the point cloud information may be determined;

[0091] Randomly select three target points from the internal point set to establish the second plane model;

[0092] Calculating second distances between other target points in the first interior point set except the three extracted target points and the second plane model, and determining a second interior point set relative to the second plane model according to the second distances;

[0093] determining a second ratio of the number of inliers in the second inlier set to the number of inliers in the first inlier set;

[0094] Based on the size relationship between the first ratio and the second ratio, selecting a target inlier point set with a larger ratio from the first inlier point set and the second inlier point set as the updated first inlier point set;

[0095] This target inner point set can replace the initial point cloud information, and the steps of establishing the first plane model are continued in a loop until the preset number of loops is reached. The model established by randomly selecting three inner points from the target inner point set obtained at the end of the loop can be considered as the door and window plane model of the target door and window.

[0096] Each target inner point set obtained in the loop is a subset of suspected observation targets. The remaining points not in these inner point sets are considered to be noise points that are irrelevant to the target doors and windows. In the process of multiple cycles, the noise points are gradually eliminated, and the remaining inner points are more representative of the target doors and windows, making the determined door and window plane model more accurate.

[0097] The following example illustrates how to determine monitoring parameters and parameter thresholds based on door and window types.

[0098] If the door and window type is a sliding door and window type, the center point position parameter, speed parameter, and angle parameter of the target door and window are obtained as monitoring parameters based on the switch state information, and the corresponding parameter thresholds are: the change amplitude threshold of the center point position, the change amplitude threshold of the speed, and the change amplitude threshold of the angle;

[0099] In this case, the monitoring parameters and parameter thresholds are compared as follows:

[0100] If the center point coordinates in the center point position parameters and the speed represented by the speed parameters change in their pre-calibrated directions beyond preset thresholds, which are respectively the center point position change amplitude threshold and the speed change amplitude threshold, then it is considered that there is an abnormal movement; or, if the center point coordinates of the door and window plane model in two directions orthogonal to its movement direction show a change amplitude exceeding the change amplitude threshold, then it is determined that there is an abnormal movement; or, if the angle parameters show that the rotation angle of the door and window plane model is greater than the angle change amplitude threshold, then it is determined that there is an abnormal movement.

[0101] If the door and window type is a casement door and window type, the center point position parameters and angle parameters of the target door and window are obtained as monitoring parameters based on the switch state information. The corresponding parameter thresholds are: the change amplitude threshold of the center point position, the change amplitude threshold of the angle, and the change amplitude threshold of the angular velocity;

[0102] In this case, the monitoring parameters and parameter thresholds are compared as follows:

[0103] If the change amplitude of the center point position of the door and window plane model exceeds the preset threshold, or shows irregular changes, it is determined that there is an abnormal movement; or, if the angle parameter shows that the rotation angle and angular velocity of the casement door and window along the rotation center exceed the angle change amplitude threshold, or the casement door and window shows an angle change greater than the angle change amplitude threshold in the direction orthogonal to the rotation center, it is determined that there is an abnormal movement.

[0104] If the door and window type is a fixed door and window type, the plane model parameters and angle parameters of the target door and window are obtained as monitoring parameters based on the switch status information, and the corresponding parameter thresholds are: the model parameter change amplitude threshold and the angle change amplitude threshold.

[0105] In this case, the monitoring parameters and parameter thresholds are compared as follows:

[0106] If the center point position of the door and window plane model changes more than the model parameter change amplitude threshold, it is determined that there is an abnormality; if the change amplitude of the plane model parameter corresponding to the door and window plane model is greater than the model parameter change amplitude threshold, or the rotation angle is greater than the angle change amplitude threshold, it is determined that there is an abnormality;

[0107] On the contrary, if it is contrary to the above determination, it is determined that there is no abnormal movement or damage, and it can be determined as a static state without alarm, or a normal opening / closing state without alarm.

[0108] Among them, for the target doors and windows in a static and non-alarm state, the changes in their monitoring parameters at different times are kept in a small range. Thresholds can be preset for different changes for judgment. At this time, the doors and windows are regarded as being in a static or slightly vibrating state under the influence of the environment, and the abnormal movement monitoring equipment does not issue an alarm signal.

[0109] For normal opening / closing without alarm, if the target door or window is a sliding door or window, the coordinates of the center point of the corresponding door or window plane model will show uniform changes within a certain amplitude range along its pre-calibrated movement direction, and the coordinates of the center point in the two directions orthogonal to its movement direction will only change slightly, and the angle of the corresponding door or window plane model around the three axes will remain basically unchanged; for casement doors and windows, the coordinates of the center point of the corresponding door or window plane model will rotate around its pre-calibrated rotation center at an angular velocity within a certain amplitude range, and the angle along the rotation direction of the rotation center should also show uniform changes within a certain period of time, while the angle in the two directions orthogonal to its rotation direction will remain basically unchanged. The monitoring device will not issue an alarm signal for the observed target door or window in the normal opening / closing state.

[0110] In one embodiment, if the comparison result of the monitoring parameter and the parameter threshold determines that the target door or window has abnormal movement, an abnormal alarm message is sent;

[0111] In addition, a specific destruction alarm signal can be sent when doors and windows are damaged. For example, in another embodiment, if the state change of the switch state information and the distribution change of the target point in the point cloud information are greater than a preset change threshold, or the door and window plane model cannot be constructed based on the point cloud information, a destruction alarm signal is sent.

[0112] In this case, the state change can be determined by the change amplitude of the model center point position, angle, and model parameters in a short period of time. If the change amplitude of any parameter is greater than the preset change threshold corresponding to the parameter type and shows a drastic change state, it is considered that the target door and window are damaged, and a damage alarm signal is sent.

[0113] After the monitoring platform receives the abnormal alarm signal and damage alarm signal sent by the door and window movement monitoring device, it will quickly notify the monitoring personnel by alarm, text message, emergency call, etc. according to the type of alarm signal to ensure the safety of the monitored object / area and avoid unnecessary risks.

[0114] In one embodiment of the present invention, a door and window movement monitoring device based on millimeter wave radar is provided. Figure 2 , the device comprises:

[0115] A millimeter wave radar module is used to send a frequency modulated continuous wave (FMCW) signal to a target door or window using a millimeter wave radar, and obtain a reflected signal reflected by the target door or window; determine an intermediate frequency signal related to a signal propagation distance based on the reflected signal and the FMCW signal; determine motion state information of the target door or window according to the intermediate frequency signal; and calculate point cloud information representing the location of the target door or window at multiple times according to the motion state information;

[0116] A target tracking module constructs a door and window plane model of the target door and window according to the point cloud information, and determines the switch state information corresponding to the door and window plane model;

[0117] The target alarm module is used to determine the monitoring parameters and parameter thresholds of the door and window types corresponding to the target doors and windows from the switch status information; and determine whether there is any abnormal movement of the target doors and windows based on the comparison results of the monitoring parameters and the parameter thresholds.

[0118] For details, see Figure 2 After transmitting the FMCW signal and receiving the reflected signal, the millimeter-wave radar module can perform mixing processing and low-frequency filtering processing, obtain the distance between the target point on the target door or window and the millimeter-wave radar through distance calculation, and obtain the angle information through direction calculation, that is, determine the motion state information, and then obtain 4D point cloud information.

[0119] The target tracking module can first perform singular value filtering, delete the target points corresponding to the noise points and outliers in the point cloud information, perform point cloud segmentation and model fitting, segment to obtain the internal point set, and iterate the door and window plane model according to the above embodiment to achieve target recognition. The door and window plane model represents the target position / attitude information of the target door and window.

[0120] The target alarm module can calculate the motion state, obtain monitoring parameters, and realize alarm state analysis by comparing with parameter thresholds, and make alarm state judgment to determine whether there is abnormal movement or damage. If so, an alarm signal is sent.

[0121] In one embodiment of the present invention, an electronic device is provided, comprising: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the method for monitoring door and window movement based on millimeter-wave radar as described in any of the aforementioned embodiments is implemented.

[0122] Figure 3 It is a structural block diagram of an electronic device according to an embodiment of the present invention.

[0123] like Figure 3 As shown, the electronic device 300 includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the electronic device 300 does not constitute a limitation on the embodiments of the present invention.

[0124] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0125] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0126] The memory 303 is used to store a computer program corresponding to the method for monitoring door and window movement based on millimeter wave radar according to the above embodiment of the present invention, and the computer program is controlled and executed by the processor 301. The processor 301 is used to execute the computer program stored in the memory 303 to implement the contents shown in the above method embodiment.

[0127] Among them, the electronic device 300 includes but is not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device 300 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0128] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0129] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0130] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0131] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “clockwise”, “counterclockwise”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0132] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0133] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0134] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0135] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for monitoring abnormal movement of doors and windows based on millimeter wave radar, characterized in that: The method comprises: A millimeter wave radar is used to send a frequency modulated continuous wave (FMCW) signal to a target door or window, and a reflected signal reflected by the target door or window is obtained; Determine an intermediate frequency signal related to a signal propagation distance based on the reflected signal and the FMCW signal; Determine the motion state information of the target door and window according to the intermediate frequency signal; Calculating point cloud information representing the locations of the target doors and windows at multiple moments according to the motion state information; Constructing a door and window plane model of the target door and window according to the point cloud information, and determining switch state information corresponding to the door and window plane model; Determining monitoring parameters and parameter thresholds corresponding to the door and window type of the target door and window from the switch state information; Based on the comparison result between the monitoring parameter and the parameter threshold, it is determined whether there is any abnormal movement of the target door or window.

2. The method according to claim 1, characterized in that The determining of an intermediate frequency signal related to a signal propagation distance based on the reflected signal and the FMCW signal includes: Mixing the reflected signal with the FMCW signal to obtain a mixed signal; The mixed signal is subjected to low-pass filtering to obtain an intermediate frequency signal.

3. The method according to claim 2, characterized in that The mixed signal is obtained in the following manner: u m (t)=u s (t)·u e (t) Among them, u m (t) represents the amplitude of the mixing signal, u s (t) represents the amplitude of the FMCW signal, u e (t) represents the amplitude of the reflected signal; The amplitude of the FMCW signal is determined as follows: Wherein, f0 is the starting frequency of the FMCW signal, T is the signal frequency modulation time, T c is the signal frequency modulation period, B is the system bandwidth, j is the imaginary unit, k is the preset coefficient, and t represents time.

4. The method according to claim 1, characterized in that The motion state information includes distance information, angle information and speed information; wherein, The distance information is determined in the following manner: Among them, S c,n represents the distance between the target point on the target door or window and the millimeter wave radar; c represents the propagation speed of the electromagnetic wave signal; f c,n represents the center frequency of each frequency modulation cycle within n frequency modulation cycles; T c represents the signal frequency modulation period; B represents the system bandwidth; The angle information is determined in the following manner: Wherein, θ represents the receiving and transmitting angle of the FMCW signal; λ represents the wavelength of the FMCW signal; represents the phase difference between the FMCW signal and the reflected signal; d represents the distance between the target point and the millimeter wave radar.

5. The method according to claim 1, characterized in that Before constructing the door and window plane model of the target door and window according to the point cloud information and determining the switch state information corresponding to the door and window plane model, the method further includes: For any target point in the point cloud information, whether the target point is a singular value point is determined based on the number of adjacent target points within a preset radius of the target point. If the target point is a singular value point, the target point information of the target point is deleted from the point cloud information.

6. The method according to claim 1, characterized in that The step of constructing the door and window plane model of the target door and window according to the point cloud information comprises: Extracting target points from the point cloud information to establish a first plane model; Determine a first inner point set according to the distance relationship between each target point in the point cloud information and the first plane model; Extracting target points from the first interior point set to establish a second plane model, and updating the first interior point set according to the distance relationship between each target point in the first interior point set and the second plane model; The door and window plane model of the target door and window is determined based on the updated first interior point set.

7. The method according to claim 1, characterized in that The step of determining the monitoring parameters of the door and window type corresponding to the target door and window from the switch state information includes: If the door and window type is a sliding door and window type, the center point position parameter, speed parameter, and angle parameter of the target door and window are obtained as monitoring parameters based on the switch state information, and the corresponding parameter thresholds are: the change amplitude threshold of the center point position, the change amplitude threshold of the speed, and the change amplitude threshold of the angle; or, If the door and window type is a casement door and window type, then the center point position parameter and angle parameter of the target door and window are obtained as monitoring parameters based on the switch state information, and the corresponding parameter thresholds are: the change amplitude threshold of the center point position, the change amplitude threshold of the angle, and the change amplitude threshold of the angular velocity; or, If the door and window type is a fixed door and window type, the plane model parameters and angle parameters of the target door and window are obtained as monitoring parameters based on the switch state information, and the corresponding parameter thresholds are: the model parameter change amplitude threshold and the angle change amplitude threshold.

8. The method according to claim 1, characterized in that The method further comprises: If the comparison result of the monitoring parameter and the parameter threshold determines that the target door or window has abnormal movement, an abnormal alarm message is sent; and / or, If the state change of the switch state information and the distribution change of the target points in the point cloud information are greater than a preset change threshold, or the door and window plane model cannot be constructed based on the point cloud information, a damage alarm signal is sent.

9. A door and window movement monitoring device based on millimeter wave radar, characterized in that: The device comprises: A millimeter wave radar module is used to send a frequency modulated continuous wave (FMCW) signal to a target door or window using a millimeter wave radar, and obtain a reflected signal reflected by the target door or window; determine an intermediate frequency signal related to a signal propagation distance based on the reflected signal and the FMCW signal; determine motion state information of the target door or window according to the intermediate frequency signal; and calculate point cloud information representing the location of the target door or window at multiple times according to the motion state information; A target tracking module constructs a door and window plane model of the target door and window according to the point cloud information, and determines the switch state information corresponding to the door and window plane model; The target alarm module is used to determine the monitoring parameters and parameter thresholds of the door and window types corresponding to the target doors and windows from the switch status information; and determine whether there is any abnormal movement of the target doors and windows based on the comparison results of the monitoring parameters and the parameter thresholds.

10. An electronic device, characterized in that: include: Memory, processor; The memory stores a computer program, and when the computer program is executed by the processor, the method for monitoring door and window movement based on millimeter-wave radar as described in any one of claims 1 to 8 is implemented.

Images