Outdoor safety monitoring method, equipment and system based on millimeter wave radar

Abstract

The invention discloses an outdoor safety monitoring method based on a millimeter wave radar, a program product and electronic equipment, and the method comprises the steps: building the pairing relation between the electronic equipment and each detection equipment, and building the communication connection with each detection equipment, and the detection equipment is one or more detection equipment; area ranges corresponding to the detection devices are obtained, each detection device comprises a millimeter wave radar sensor, and the area ranges corresponding to one or more detection devices form a monitoring range of the protection area; acquiring radar echo data received by each detection device through the millimeter wave radar sensor; and identifying a target and target information based on radar echo data received by each detection device through the millimeter wave radar sensor, and executing an early warning operation based on the target information.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an outdoor safety monitoring method based on millimeter wave radar, electronic equipment, and an outdoor safety monitoring system based on millimeter wave radar. Background Art

[0002] At present, most outdoor camping warning systems use infrared sensor detection and visual detection systems. The infrared sensor detection method is easily affected by environmental factors, such as sunlight, so the false alarm rate is high in outdoor scenes. The visual detection system using visual sensors is also easily affected by environmental factors. In bad weather environments such as night, rain, snow, fog, etc., the accuracy of target recognition is low, so the false alarm rate is also high, and the user experience is low. Summary of the invention

[0003] In order to solve the existing technical problems, the present invention provides an outdoor safety monitoring method based on millimeter wave radar, an electronic device and an outdoor safety monitoring system based on millimeter wave radar, which can improve the accuracy of early warning.

[0004] In a first aspect, a millimeter-wave radar-based outdoor safety monitoring method is provided, comprising: establishing a pairing relationship between the electronic device and each detection device, and establishing a communication connection with each of the detection devices, wherein the detection device is one or more detection devices; obtaining an area range corresponding to each of the detection devices, wherein each detection device includes a millimeter-wave radar sensor, and the area range corresponding to one or more detection devices forms a monitoring range of a protection area; obtaining radar echo data received by each of the detection devices through the millimeter-wave radar sensor; identifying targets and target information based on the radar echo data received by each of the detection devices through the millimeter-wave radar sensor, and performing an early warning operation based on the target information.

[0005] In a second aspect, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the outdoor safety monitoring method based on millimeter wave radar as described in any embodiment of the present application.

[0006] In a third aspect, a millimeter-wave radar-based outdoor safety monitoring system is provided, comprising an electronic device as described in any embodiment of the present application, and at least one detection device, wherein the electronic device can establish a pairing relationship with the detection device to establish a communication connection, and the detection device comprises a millimeter-wave radar sensor, and the millimeter-wave radar sensor is used to perform detection within the area corresponding to the detection device and obtain radar echo data.

[0007] The embodiment of the present application establishes a pairing relationship between an electronic device and each detection device, establishes a communication connection with each of the detection devices, and obtains the area range corresponding to each of the detection devices, thereby forming an all-round monitoring range around the protection area, and obtains radar echo data received by each detection device, and based on the radar echo data received by each detection device, identifies the target and obtains target information, and performs early warning operations according to the target information, thereby realizing all-round outdoor monitoring. Moreover, since the radar echo data obtained by the millimeter-wave radar is used to identify the target information, the millimeter-wave radar has high resolution, good anti-interference ability, small size, light weight, and can work in severe weather conditions, so it can improve the accuracy of the early warning and is easy to carry. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a diagram of an application environment of an outdoor safety monitoring method based on millimeter wave radar in an embodiment;

[0009] Figure 2 is a schematic diagram of an electronic device in one embodiment;

[0010] Figure 3 is a schematic diagram of a detection device in one embodiment;

[0011] Figure 4 is a flow chart of an outdoor safety monitoring method based on millimeter wave radar in one embodiment;

[0012] Figure 5 A schematic diagram of a region range corresponding to a detection device in an embodiment;

[0013] Figure 6 A schematic diagram of setting a region range for a detection device in an embodiment;

[0014] Figure 7 is a schematic diagram of a virtual map display in one embodiment;

[0015] Figure 8 is a schematic diagram of an outdoor safety monitoring device based on millimeter wave radar in one embodiment;

[0016] Fig. 9 FIG. 1 is a schematic diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0017] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the technical field of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0019] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0020] See also Figure 1 , is an application environment diagram of an outdoor safety monitoring method based on millimeter wave radar in one embodiment. The application environment diagram includes an electronic device 10 and one or more detection devices 20, the detection device 20 includes a millimeter wave radar sensor 201, and one or more detection devices 20 are respectively arranged in the surrounding area of ​​the protection area for all-round monitoring of real-time environmental data around the protection area. The electronic device 10 is used to identify the target according to the real-time environmental data monitored by each detection device 20, and timely warn when a dangerous situation is found. The protection area indicates an area that the identified target cannot approach, including but not limited to the area where the tent is located, the area where the vehicle is located, the camping area, etc.

[0021] The electronic device 10 is provided with a plurality of detection device identifiers, which are used to bind with the detection device 20, so that each detection device 20 can correspond to a detection device identifier. Figure 2 As shown, it is a schematic diagram of an electronic device in one embodiment, and the detection device identification is represented by digital numbers 1, 2, 3, 4, 5, 6. By selecting the number and the pairing button set on the electronic device 10, the detection device can be paired, that is, the detection device 20 can correspond to a digital number, so that when the electronic device 10 communicates with multiple detection devices 20 to send data, the detection device 20 can be identified by the detection device identification. The area range of a detection device 20 can be configured by the detection device identification and the area setting button. The electronic device 10 is also provided with a variety of indicator lights, including but not limited to status indicator lights, detection device identification indicator lights, pairing key indicator lights, area setting indicator lights, etc. The indicator lights can perform different prompt operations in different states, wherein the prompt operations include but are not limited to: prompting by displaying different colors, prompting by flashing times, etc. The detection device identification, switch button, area setting button, pairing button can be a physical key, or a touch key on the user interface, etc., which is not limited here. The expression form of the electronic device 10 is not limited to Figure 2As shown, the electronic device 10 may also be a device installed with a millimeter wave radar-based outdoor safety monitoring program, for example, it may include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a terminal device (e.g., a mobile phone, etc.), a wearable device (e.g., a pair of smart glasses or a smart watch), a handheld device, etc. Through the user interface provided by the millimeter wave radar-based outdoor safety monitoring program, the user interface with the outdoor safety monitoring program is realized. Figure 2 The same various key functions.

[0022] like Figure 3 FIG. 2 is a schematic diagram of a detection device in an embodiment. The detection device 20 is provided with a switch button and a pairing button, and a pairing request signal can be sent to the electronic device through the pairing button. It can be understood that the detection device 20 can also include a variety of indicator lights, and the indicator lights can perform different prompt operations in different states, wherein the prompt operations include but are not limited to: prompting by displaying different colors, prompting by flashing times, etc.

[0023] The millimeter wave radar sensor 201 is used to send electromagnetic wave signals to the area corresponding to the detection device. When there are targets in the area, these multiple targets will reflect the electromagnetic wave signals, and the millimeter wave radar sensor 201 can receive the echo signals reflected by the targets. The electronic device 20 detects the targets in the area based on the echo signals detected by the millimeter wave radar sensor 201.

[0024] The millimeter wave radar sensor 201 is a radar system that uses electromagnetic waves in the millimeter wave band to detect targets. The wavelength of millimeter waves ranges from 1 mm to 10 mm, which is between microwaves and terahertz waves. The millimeter wave radar sensor acquires information about the target by emitting millimeter wave signals and receiving signals reflected from the target. The target information includes but is not limited to the distance between the target and the detection device 20, the target's location data, the target's radar point cloud data, and the like.

[0025] See also Figure 4 , is a flow chart of an outdoor safety monitoring method based on millimeter wave radar provided in an embodiment of the present application. The outdoor safety monitoring method based on millimeter wave radar is applied to electronic equipment, and the outdoor safety monitoring method based on millimeter wave radar includes the following steps:

[0026] S11. Establishing a pairing relationship between the electronic device and each detection device, and establishing a communication connection with each detection device.

[0027] In this embodiment, generally, the detection devices correspond to their respective detection ranges, and one detection device cannot detect in all directions. Therefore, generally in outdoor environments, multiple detection devices are required. Therefore, when there are multiple detection devices, it is necessary to configure the pairing relationship between each detection device and the electronic device. Through the pairing relationship, a communication connection can be established between the electronic device and the detection device, and the detection device can also be identified, so that when the electronic device receives data, it can determine the detection device that sent the data. The communication connection method between the electronic device and the detection device can be a wireless or wired network connection method.

[0028] S12. Acquire the area range corresponding to each detection device, wherein each detection device includes a millimeter wave radar sensor, and the area range corresponding to one or more detection devices forms a monitoring range of the protection area.

[0029] In this embodiment, for a protection area, multiple detection devices can be placed around the protection area, each detection device corresponds to an area range, wherein the area ranges corresponding to the detection devices may intersect or not intersect. Based on the area ranges corresponding to the multiple detection devices, a monitoring range around the protection area can be formed. Specifically, the area ranges corresponding to the multiple detection devices are spliced ​​to obtain a monitoring range around the protection area. Figure 5 As shown, Figure 5 It is a schematic diagram of a region range corresponding to a detection device in an embodiment. The region range of a detection device is similar to a fan-shaped region range, and targets can be identified within the region range.

[0030] S13. Acquire radar echo data received by each detection device through a millimeter wave radar sensor.

[0031] In this embodiment, each detection device is responsible for detection within its own area. The millimeter wave radar sensor in each detection device sends electromagnetic waves to the corresponding area at a preset time interval. When the electromagnetic waves encounter obstacles, they can reflect echo signals, so that the detection device can receive radar echo data. By analyzing these radar echo data, the distance, speed, angle and other information of the target can be determined.

[0032] S14. Identify the target and target information based on the radar echo data received by each detection device through the millimeter wave radar sensor, and perform an early warning operation based on the target information.

[0033] In this embodiment, for any detection device, the radar echo data can be pre-processed to obtain pre-processed signal data. The pre-processing includes at least one of the following: static clutter elimination, mixer-based processing operations, analog-to-digital converter-based sampling operations, and fast-time Fourier transform-based transformation operations. The echo signal is first pre-processed to obtain pre-processed echo data, wherein the pre-processing operations include but are not limited to static clutter elimination, mixer-based processing operations, analog-to-digital converter-based sampling operations, and fast-time Fourier transform-based transformation operations. The echo signal performs static clutter elimination, which can reduce the impact of clutter signals generated by large static objects in the room on the target echo signal. The transmission signal and the echo signal are mixed based on the processing operation of the mixer to obtain a signal with a new frequency, which is the intermediate frequency signal. The frequency and phase of the echo signal and the transmission signal are different. Therefore, the distance, speed, angle and other data can be extracted and analyzed from the superimposed intermediate frequency signal. The sampling operation based on the analog-to-digital converter is used to sample the processed echo signal, and the sampled signal is Fourier transformed to obtain a discrete echo signal x(m,n) containing distance dimension and slow time dimension information, where m represents the slow time dimension, which is the mth pulse echo, and n represents the distance dimension, which is the nth distance unit.

[0034] Based on the pre-processed signal data, the target is identified and the target information is determined by using a target recognition algorithm, wherein the target recognition algorithm includes but is not limited to: a recognition method based on the directional gradient histogram (HOG) feature of two-dimensional FFT, target recognition based on machine learning algorithms such as support vector machine (SVM) or convolutional neural network (CNN), etc. The target information includes but is limited to the speed of the target, the distance of the target, the angle of the target, the position of the target, etc. The targets include but are not limited to: pedestrians, animals, vehicles, etc. For different outdoor environments, target options corresponding to the outdoor environment can be provided on the user interface, and targets matching the current outdoor environment can be configured based on the target options. For example, if the user is camping in a place where animals often appear, multiple animal categories can be configured as targets on the user interface, so that the target can be identified more accurately. The warning operation includes but is not limited to one or more of the following combinations: image warning, indicator light warning, voice warning, etc.

[0035] In this embodiment, by comparing the transmitted signal and the received signal of the millimeter radar wave, the flight time (Time of Flight, ToF) or frequency difference of the signal is calculated to determine the distance and speed of the target. Using the Doppler effect, the radar can measure the speed of the target. When the target moves toward the radar, the frequency of the echo signal will be higher than the frequency of the transmitted signal; when the target moves away from the radar, the frequency of the echo signal will be lower than the frequency of the transmitted signal. By calculating the change in frequency, the speed of the target can be obtained. Millimeter wave radars usually use antenna arrays to determine the angle of arrival (Angle of Arrival, AoA) of the echo signal. By analyzing the phase difference of the signals received by different antennas in the array, the angular position of the target can be estimated.

[0036] In the above embodiment, by establishing a pairing relationship between an electronic device and each detection device, a communication connection with each of the detection devices is established, and the area range corresponding to each of the detection devices is obtained, thereby forming an all-round monitoring range around the protection area, and obtaining radar echo data received by each detection device, and based on the radar echo data received by each detection device, identifying the target and obtaining target information, and performing early warning operations according to the target information, thereby achieving all-round outdoor monitoring. Moreover, since the radar echo data obtained by the millimeter-wave radar is used to identify the target information, the millimeter-wave radar has high resolution, good anti-interference ability, small size, light weight, and can work in severe weather conditions, so it can improve the accuracy of the early warning and is easy to carry.

[0037] In some embodiments, establishing a pairing relationship between the electronic device and each detection device includes:

[0038] For any of the detection devices, obtaining a pairing request sent by the detection device;

[0039] Based on the electronic device obtaining a detection device identification selection operation corresponding to the detection device and a pairing confirmation operation for pairing with the detection device, a pairing confirmation signal corresponding to the pairing confirmation operation and a detection device identification corresponding to the detection device identification selection operation are sent to the detection device;

[0040] A feedback signal sent by the detection device to confirm successful pairing is obtained.

[0041] In this embodiment, the electronic device provides a detection device identification control and a pairing confirmation control. The user can select the desired detection device identification by triggering the detection device identification control, and can confirm entering the pairing state through the pairing confirmation control. It can be understood that the types of the detection device identification control and the pairing confirmation control can be one or more of the following combinations: virtual touch buttons, physical buttons, text boxes, drop-down boxes, check boxes, icons, combo boxes, etc. When the detection device identification control is triggered, the detection device identification selection operation is obtained, and when the pairing confirmation control is triggered, the pairing confirmation operation is obtained, and the pairing state is entered, so that the detection device identification corresponding to the detection device identification selection operation is sent to the requesting detection device. After the requesting detection device receives the detection device identification, it sends a feedback signal to the electronic device, thereby realizing the pairing relationship between the electronic device and the requested detection device. For any detection device, pairing can be performed according to the above method.

[0042] In the above embodiment, a request is sent through the detection device, a detection device identifier corresponding to the detection device is set on the electronic device, and the identifier is sent to the requesting detection device to implement pairing with the detection device, thereby achieving a communication connection, which facilitates the subsequent configuration of the area range of the detection device, so as to facilitate all-round monitoring of the protection area.

[0043] In an optional embodiment, based on the electronic device obtaining a detection device identification selection operation corresponding to the detection device and a pairing confirmation operation for pairing with the detection device, sending a pairing confirmation signal corresponding to the pairing confirmation operation and a detection device identification corresponding to the detection device identification selection operation to the detection device, including:

[0044] When the detection device identification selection operation is acquired, performing a prompt operation through at least one indicator light on the electronic device to prompt that the detection device identification has been selected;

[0045] When the pairing confirmation operation is obtained, a prompt operation is performed through at least one indicator light on the electronic device to confirm entering a pairing state with the detection device, and the detection device identifier is sent to the detection device;

[0046] The step of obtaining a feedback signal sent by the detection device to confirm successful pairing includes:

[0047] When the feedback signal is obtained, a prompt operation is performed through at least one indicator light on the electronic device to prompt the user that the pairing is successful.

[0048] In this embodiment, if Figure 2As shown, when a detection device identification button is clicked, the detection device identification selection operation is obtained. For example, if the number "2" is clicked, it means that the detection device identification selection operation indicates that the number "2" has been selected. Then the detection device identification indicator under the number "2" can flash to prompt the user that the number "2" has been selected. Then click the pairing button, and the electronic device can obtain the pairing confirmation operation, so that the pairing key indicator can display yellow and flash, prompting the user that the requested detection device has entered the pairing state. After obtaining the feedback signal, the pairing key indicator is used to prompt in a way different from the yellow flashing method, and the pairing is confirmed to be successful by displaying red and flashing.

[0049] In the above embodiment, in different task states, at least one indicator light is used to prompt the user that different tasks have been entered, so that the user can more intuitively understand the pairing process, thereby achieving communication connection and facilitating subsequent configuration of the area range of the detection equipment to facilitate all-round monitoring of the protection area.

[0050] In some embodiments, obtaining the area range corresponding to each of the detection devices includes:

[0051] For any of the detection devices, upon obtaining a confirmation selection operation of the detection device identification corresponding to the detection device and obtaining a region setting operation, confirming to enter the region setting state of the detection device;

[0052] Acquire the trajectory data of the user detected by the detection device, and determine the area range corresponding to the detection device according to the trajectory data.

[0053] In this embodiment, the detection device identification control is triggered to obtain a confirmation selection operation, and different operations are configured for the detection device identification control. For example, a short press on the detection device identification control is used to obtain a detection device identification selection operation, and a long press on the detection device identification control is used to obtain a region setting operation. The same is true for other controls. Different triggering methods can be set to correspond to different operations, and the types of controls can also be diverse, as in the above embodiments. The electronic device is also provided with a region setting control. After the user triggers the region setting control, the electronic device can obtain the region setting operation, and then the electronic device enters the region setting state. Among them, Figure 2 As shown, the locale control can be Figure 2 The regional settings button in .

[0054] The trajectory data represents the trajectory formed by the user from the beginning of the movement to the end of the movement. The shape of the trajectory is not limited to any type of regular shape, or a combination of multiple regular shapes, etc. Figure 6 As shown, Figure 6This is a schematic diagram of setting a regional range for a detection device in an embodiment; for example, after entering the regional setting state, the user drives at a constant speed within the detection range of the detection device to form trajectory data, and after returning to the starting point of the movement, the regional setting control can be triggered again, and the electronic device obtains the trajectory setting end operation, and then determines the regional range corresponding to the detection device according to the trajectory data. For other detection devices, the regional range can be set according to the above steps in sequence.

[0055] In an optional embodiment, the acquiring the trajectory data of the user detected by the detection device and determining the area range corresponding to the detection device according to the trajectory data includes:

[0056] Obtaining the projection point of the detection device on the horizontal plane;

[0057] According to the trajectory data, extract the maximum horizontal point in the horizontal direction, and extract the minimum vertical point and the maximum vertical point in the vertical direction;

[0058] Based on the projection point, horizontal maximum point, vertical minimum point and vertical maximum point, the area range corresponding to the detection device is determined, wherein the horizontal direction indicates the direction of establishing a horizontal line with the projection point of the detection device on the horizontal plane, and the vertical direction is the direction perpendicular to the horizontal direction.

[0059] In this embodiment, the horizontal maximum point, the vertical minimum point and the vertical maximum point are boundary points. Connecting the projection point with these boundary points can form a range similar to a fan-shaped area. Figure 5 The area range shown in FIG. According to the trajectory data, the horizontal maximum point, the vertical minimum point and the vertical maximum point are extracted by using a feature extraction method, wherein the feature extraction method includes but is not limited to an edge extraction method.

[0060] In an optional embodiment, the acquiring the trajectory data of the user detected by the detection device and determining the area range corresponding to the detection device according to the trajectory data includes:

[0061] When the area range corresponding to the detection device is not obtained according to the trajectory data, a prompt operation is performed through at least one indicator light on the electronic device to prompt that the area range setting of the detection device fails.

[0062] In this embodiment, after the electronic device obtains the trajectory setting and ends the operation, the situation where the area range corresponding to the detection device is not obtained according to the trajectory data includes but is not limited to not extracting edge points, not extracting all boundary points, etc. When the area range corresponding to the detection device is not obtained, a prompt can be given through the area setting indicator to prompt that the area range setting of the detection device has failed.

[0063] In the above embodiment, based on the confirmation selection operation of the detection device identification corresponding to each detection device and the acquisition of the area setting operation, the area setting state of the detection device can be entered, and then the trajectory data of the users of each detection device can be obtained, and the area range corresponding to the detection device can be determined according to the trajectory data, thereby facilitating accurate configuration of the area range of the detection device, realizing all-round monitoring of the protection area, and improving the accuracy of early warning.

[0064] In some embodiments, identifying a target and target information based on the radar echo data received by each of the detection devices through the millimeter wave radar sensor, and performing an early warning operation based on the target information includes at least one of the following:

[0065] When a target is detected based on the radar echo data received by the detection device, controlling at least one indicator light to perform a warning operation to prompt a user that a target is approaching within the area of ​​the detection device;

[0066] When a target is detected based on the radar echo data received by the detection device, the target category is detected, and different early warning operations are performed according to the danger level of the target category, wherein the higher the danger level, the stronger the warning corresponding to the early warning operation;

[0067] When a target is detected based on the radar echo data received by the detection device, the area level of the target is determined according to the position of the target, and different early warning operations are performed at different area levels, wherein the higher the area level and the closer it is to the protection area, the stronger the warning corresponding to the early warning operation.

[0068] In this embodiment, in an optional embodiment, when a target is detected within the area of ​​the detection device, an early warning operation is performed. For example, if a target is detected within the area of ​​the detection device A, the detection device identification indicator light of the detection device A can be controlled to prompt, for example, continuously flashing to prompt the user that a target appears in the direction of the detection device A. In an optional embodiment, when identifying a target, the target category can also be identified, for example, using a pre-trained target detection model to identify the target category, using sample targets of different categories to form a training data set, using the training data set to train the target detection model, and pre-configuring the danger level of each target category. When the target category is identified, different warnings are given according to the danger level corresponding to the target category, so that the user can clearly understand the dangerous state outside the protection area in the protection area, thereby improving the accuracy of the early warning. In an optional implementation method, the protection area can be used as the center point, and different monitoring areas can be divided with different radius lengths from the center point. The closer the area is to the center point, the stronger the area level is. In this way, when the target gradually approaches the protection area, warnings of different intensities are given to remind the user.

[0069] In the above embodiments, when a target is identified within the area range of the detection device, a timely warning is issued, or different warnings are given according to the degree of danger of the target, or different warnings are given according to the distance of the target from the protection area, so that the user can understand the status of the target outside the protection area in time within the protection area, provide warning accuracy, and thus improve outdoor safety.

[0070] In some embodiments, the method further comprises:

[0071] For any of the detection devices, deleting the pairing relationship between the detection device and the electronic device;

[0072] Deleting the pairing relationship between the detection device and the electronic device includes:

[0073] When the detection device identification selection operation is acquired, performing a prompt operation through at least one indicator light on the electronic device to prompt that the detection device identification has been selected;

[0074] When a deletion confirmation operation is obtained, confirming that the deletion state for deletion pairing with the detection device is entered, and sending a deletion signal corresponding to the deletion state to the detection device;

[0075] After obtaining a deletion success signal fed back by the detection device, it is confirmed that the pairing relationship with the detection device is successfully deleted.

[0076] In this embodiment, when the user needs to delete the pairing relationship of a certain detection device, the detection device identification selection operation can also be obtained through the detection device identification control, and the deletion confirmation operation can be obtained by triggering the pairing button, so that the electronic device enters the deletion state, and the deletion signal of the pairing relationship is sent to the detection device corresponding to the detection device identification selection operation, thereby realizing the deletion of the pairing relationship.

[0077] In the above embodiment, the pairing relationship between the detection device and the electronic device can be deleted according to demand, so as to flexibly configure the monitoring area around the protection area and improve the user experience.

[0078] In some embodiments, the target and target information are identified based on the radar echo data received by each of the detection devices through the millimeter wave radar sensor, and an early warning operation is performed based on the target information, including:

[0079] Establishing a virtual map based on the area range corresponding to each of the detection devices, and displaying the virtual map on a user interface;

[0080] When a target is detected based on the radar echo data received by the detection device, a moving track of the target is displayed in real time on the virtual map based on the radar echo data received in real time by the detection device;

[0081] When the moving trajectory of the target indicates that the target is close to the protection area, an early warning operation is performed.

[0082] In this embodiment, since the detection device is built based on the protection area, a virtual map can be established according to the area range and installation orientation of the detection device and the location of the protection area, wherein the method of establishing the virtual map includes but is not limited to the Simultanous Localization and Mapping (SLAM) algorithm. The form of the virtual map includes but is not limited to the grid map, etc. After the virtual map is established, the virtual map can be visualized, such as Figure 7 The figure shows a schematic diagram of the display of a virtual map in one embodiment. The location of the protection area, the location of the target, and the location of each detection device relative to the protection area can be displayed on the virtual map. The radar echo data received in real time can also be used to track the moving trajectory of the target using a radar tracking algorithm, and the moving trajectory of the target can be displayed on the virtual map, so that the user can clearly know the movement of the target outside the protection area through this user interface, thereby improving outdoor safety. It is also possible to predict whether the target is approaching or moving away from the protection area based on the tracked moving trajectory. When the moving trajectory of the target indicates that the target is approaching the protection area, an early warning operation is performed, thereby providing an early warning. The trajectory prediction method includes but is not limited to the Kalman filter algorithm.

[0083] In the above embodiment, a virtual map can be established based on the area range corresponding to each of the detection devices, and the virtual map can be visualized, so that the movement of targets outside the protection area can be clearly known through the displayed virtual map, thereby improving outdoor safety.

[0084] On the other hand, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the outdoor safety monitoring method based on millimeter wave radar described in any embodiment of the present application.

[0085] Among them, in the computer program product, an optional implementation form of the program module architecture of the computer program that implements each step of the target identification method can be an outdoor safety monitoring device based on millimeter wave radar.

[0086] See also Figure 8An embodiment of the present application provides an outdoor safety monitoring device based on millimeter-wave radar, including: an establishment module 81, used to establish a pairing relationship between the electronic device and each detection device, and establish a communication connection with each of the detection devices, wherein the detection device is one or more detection devices; an acquisition module 82, used to obtain the area range corresponding to each of the detection devices, wherein each detection device includes a millimeter-wave radar sensor, and the area range corresponding to one or more detection devices forms a monitoring range of a protection area; the acquisition module 82 is also used to obtain radar echo data received by each of the detection devices through the millimeter-wave radar sensor; an early warning module 83, used to identify targets and target information based on the radar echo data received by each of the detection devices through the millimeter-wave radar sensor, and perform early warning operations based on the target information.

[0087] Optionally, the establishment module 81 is further used for:

[0088] For any of the detection devices, obtaining a pairing request sent by the detection device;

[0089] Based on the electronic device obtaining a detection device identification selection operation corresponding to the detection device and a pairing confirmation operation for pairing with the detection device, a pairing confirmation signal corresponding to the pairing confirmation operation and a detection device identification corresponding to the detection device identification selection operation are sent to the detection device;

[0090] A feedback signal sent by the detection device to confirm successful pairing is obtained.

[0091] Optionally, the establishment module 81 is further used for:

[0092] When the detection device identification selection operation is acquired, performing a prompt operation through at least one indicator light on the electronic device to prompt that the detection device identification has been selected;

[0093] When the pairing confirmation operation is obtained, a prompt operation is performed through at least one indicator light on the electronic device to confirm entering a pairing state with the detection device, and the detection device identifier is sent to the detection device;

[0094] The step of obtaining a feedback signal sent by the detection device to confirm successful pairing includes:

[0095] When the feedback signal is obtained, a prompt operation is performed through at least one indicator light on the electronic device to prompt the user that the pairing is successful.

[0096] Optionally, the acquisition module 82 is further used for:

[0097] For any of the detection devices, upon obtaining a confirmation selection operation of the detection device identification corresponding to the detection device and obtaining a region setting operation, confirming to enter the region setting state of the detection device;

[0098] Acquire the trajectory data of the user detected by the detection device, and determine the area range corresponding to the detection device according to the trajectory data.

[0099] Optionally, the acquisition module 82 is further used for:

[0100] Obtaining the projection point of the detection device on the horizontal plane;

[0101] According to the trajectory data, extract the maximum horizontal point in the horizontal direction, and extract the minimum vertical point and the maximum vertical point in the vertical direction;

[0102] Based on the projection point, horizontal maximum point, vertical minimum point and vertical maximum point, the area range corresponding to the detection device is determined, wherein the horizontal direction indicates the direction of establishing a horizontal line with the projection point of the detection device on the horizontal plane, and the vertical direction is the direction perpendicular to the horizontal direction.

[0103] Optionally, the acquisition module 82 is further used for:

[0104] When the area range corresponding to the detection device is not obtained according to the trajectory data, a prompt operation is performed through at least one indicator light on the electronic device to prompt that the area range setting of the detection device fails.

[0105] Optionally, the early warning module 83 is further used for:

[0106] When a target is detected based on the radar echo data received by the detection device, controlling at least one indicator light to perform a warning operation to prompt a user that a target is approaching within the area of ​​the detection device;

[0107] When a target is detected based on the radar echo data received by the detection device, the target category is detected, and different early warning operations are performed according to the danger level of the target category, wherein the higher the danger level, the stronger the warning corresponding to the early warning operation;

[0108] When a target is detected based on the radar echo data received by the detection device, the area level of the target is determined according to the position of the target, and different early warning operations are performed at different area levels, wherein the higher the area level and the closer it is to the protection area, the stronger the warning corresponding to the early warning operation.

[0109] Optionally, the early warning module 83 is further used for:

[0110] For any of the detection devices, deleting the pairing relationship between the detection device and the electronic device;

[0111] Deleting the pairing relationship between the detection device and the electronic device includes:

[0112] When the detection device identification selection operation is acquired, performing a prompt operation through at least one indicator light on the electronic device to prompt that the detection device identification has been selected;

[0113] When a deletion confirmation operation is obtained, confirming that the deletion state for deletion pairing with the detection device is entered, and sending a deletion signal corresponding to the deletion state to the detection device;

[0114] After obtaining a deletion success signal fed back by the detection device, it is confirmed that the pairing relationship with the detection device is successfully deleted.

[0115] Optionally, the early warning module 83 is further used for:

[0116] Establishing a virtual map based on the area range corresponding to each of the detection devices, and displaying the virtual map on a user interface;

[0117] When a target is detected based on the radar echo data received by the detection device, a moving track of the target is displayed in real time on the virtual map based on the radar echo data received in real time by the detection device;

[0118] When the moving trajectory of the target indicates that the target is close to the protection area, an early warning operation is performed.

[0119] It can be understood by those skilled in the art that Figure 8 The structure of the outdoor safety monitoring device based on millimeter wave radar does not constitute a limitation on the outdoor safety monitoring device based on millimeter wave radar, and each module can be implemented in whole or in part by software, hardware, and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. In other embodiments, the outdoor safety monitoring device based on millimeter wave radar may include more or fewer modules than shown in the figure.

[0120] See also Fig. 9In another aspect of the embodiment of the present application, an electronic device 10 is provided, including a processor 13 and a memory 14, wherein the memory 14 stores a computer program, and when the computer program is executed by the processor, the processor 13 executes the steps of the outdoor safety monitoring method based on millimeter wave radar provided in any of the above embodiments of the present application. The electronic device 10 may include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a terminal device (e.g., a mobile phone, etc.), a wearable device (e.g., a pair of smart glasses or a smart watch), a handheld device, or a similar device.

[0121] The processor 13 is the control center, which uses various interfaces and lines to connect various parts of the entire computer device, and executes various functions of the computer device and processes data by running or executing software programs and / or modules stored in the memory 14, and calling data stored in the memory 14. Optionally, the processor 13 may include one or more processing cores; preferably, the processor 13 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 13.

[0122] The memory 14 can be used to store software programs and modules. The processor 13 executes various functional applications and data processing by running the software programs and modules stored in the memory 14. The memory 14 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 14 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 14 can also include a memory processor to provide the processor 13 with access to the memory 14.

[0123] On the other hand, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the outdoor safety monitoring method based on millimeter wave radar provided in any of the above embodiments of the present application.

[0124] On the other hand, an embodiment of the present application provides an outdoor safety monitoring system based on millimeter-wave radar, including an electronic device as described in any embodiment of the present application, and at least one detection device, wherein the electronic device can establish a pairing relationship with the detection device to establish a communication connection, and the detection device includes a millimeter-wave radar sensor, and the millimeter-wave radar sensor is used to perform detection within the area corresponding to the detection device and obtain radar echo data.

[0125] Those skilled in the art can understand that all or part of the processes in the methods provided in the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0126] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A millimeter wave radar-based outdoor safety monitoring method, characterized in that: Used in electronic equipment, including: Establishing a pairing relationship between the electronic device and each detection device, and establishing a communication connection with each of the detection devices, wherein the detection device is one or more detection devices; Acquire the area range corresponding to each of the detection devices, wherein each detection device includes a millimeter wave radar sensor, and the area range corresponding to one or more detection devices forms a monitoring range of the protection area; Acquiring radar echo data received by each of the detection devices through the millimeter wave radar sensor; Based on the radar echo data received by each detection device through the millimeter wave radar sensor, the target and target information are identified, and based on the target information, an early warning operation is performed.

2. The outdoor safety monitoring method based on millimeter wave radar according to claim 1, characterized in that: The establishing of the pairing relationship between the electronic device and each detection device comprises: For any of the detection devices, obtaining a pairing request sent by the detection device; Based on the electronic device obtaining a detection device identification selection operation corresponding to the detection device and a pairing confirmation operation for pairing with the detection device, a pairing confirmation signal corresponding to the pairing confirmation operation and a detection device identification corresponding to the detection device identification selection operation are sent to the detection device; A feedback signal sent by the detection device to confirm successful pairing is obtained.

3. The outdoor safety monitoring method based on millimeter wave radar as claimed in claim 2, characterized in that: The method of acquiring, based on the electronic device, a detection device identification selection operation corresponding to the detection device and a pairing confirmation operation for pairing with the detection device, sending a pairing confirmation signal corresponding to the pairing confirmation operation and a detection device identification corresponding to the detection device identification selection operation to the detection device comprises: When the detection device identification selection operation is acquired, performing a prompt operation through at least one indicator light on the electronic device to prompt that the detection device identification has been selected; When the pairing confirmation operation is obtained, a prompt operation is performed through at least one indicator light on the electronic device to confirm entering a pairing state for pairing with the detection device, and the detection device identifier is sent to the detection device; The step of obtaining a feedback signal sent by the detection device to confirm successful pairing includes: When the feedback signal is obtained, a prompt operation is performed through at least one indicator light on the electronic device to prompt the user that the pairing is successful.

4. The outdoor safety monitoring method based on millimeter wave radar as claimed in claim 1, characterized in that: The obtaining of the area range corresponding to each of the detection devices comprises: For any of the detection devices, upon obtaining a confirmation selection operation of the detection device identification corresponding to the detection device and obtaining a region setting operation, confirming to enter the region setting state of the detection device; Acquire the trajectory data of the user detected by the detection device, and determine the area range corresponding to the detection device according to the trajectory data.

5. The outdoor safety monitoring method based on millimeter wave radar as claimed in claim 4, characterized in that: The acquiring the trajectory data of the user detected by the detection device, and determining the area range corresponding to the detection device according to the trajectory data, includes: Obtaining the projection point of the detection device on the horizontal plane; According to the trajectory data, extract the maximum horizontal point in the horizontal direction, and extract the minimum vertical point and the maximum vertical point in the vertical direction; Based on the projection point, horizontal maximum point, vertical minimum point and vertical maximum point, the area range corresponding to the detection device is determined, wherein the horizontal direction indicates the direction of establishing a horizontal line with the projection point of the detection device on the horizontal plane, and the vertical direction is the direction perpendicular to the horizontal direction.

6. The outdoor safety monitoring method based on millimeter wave radar as claimed in claim 4, characterized in that: The acquiring the trajectory data of the user detected by the detection device, and determining the area range corresponding to the detection device according to the trajectory data, includes: When the area range corresponding to the detection device is not obtained according to the trajectory data, a prompt operation is performed through at least one indicator light on the electronic device to prompt that the area range setting of the detection device fails.

7. The outdoor safety monitoring method based on millimeter wave radar according to claim 1, characterized in that: Identifying targets and target information based on radar echo data received by each of the detection devices through the millimeter wave radar sensor, and performing early warning operations based on the target information, including at least one of the following: When a target is detected based on the radar echo data received by the detection device, controlling at least one indicator light to perform a warning operation to prompt a user that a target is approaching within the area of ​​the detection device; When a target is detected based on the radar echo data received by the detection device, the target category is detected, and different early warning operations are performed according to the danger level of the target category, wherein the higher the danger level, the stronger the warning corresponding to the early warning operation; When a target is detected based on the radar echo data received by the detection device, the area level of the target is determined according to the position of the target, and different early warning operations are performed at different area levels, wherein the higher the area level and the closer it is to the protection area, the stronger the warning corresponding to the early warning operation.

8. The outdoor safety monitoring method based on millimeter wave radar as claimed in claim 1, characterized in that: The method further comprises: For any of the detection devices, deleting the pairing relationship between the detection device and the electronic device; Deleting the pairing relationship between the detection device and the electronic device includes: When the detection device identification selection operation is acquired, performing a prompt operation through at least one indicator light on the electronic device to prompt that the detection device identification has been selected; When a deletion confirmation operation is obtained, confirming that the deletion state for deletion pairing with the detection device is entered, and sending a deletion signal corresponding to the deletion state to the detection device; After obtaining a deletion success signal fed back by the detection device, it is confirmed that the pairing relationship with the detection device is successfully deleted.

9. The outdoor safety monitoring method based on millimeter wave radar as claimed in claim 1, characterized in that: Identifying targets and target information based on radar echo data received by each of the detection devices through the millimeter wave radar sensor, and performing early warning operations based on the target information, including: Establishing a virtual map based on the area range corresponding to each of the detection devices, and displaying the virtual map on a user interface; When a target is detected based on the radar echo data received by the detection device, a moving track of the target is displayed in real time on the virtual map based on the radar echo data received in real time by the detection device; When the moving trajectory of the target indicates that the target is close to the protection area, an early warning operation is performed.

10. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the outdoor safety monitoring method based on millimeter wave radar as claimed in any one of claims 1 to 9.

11. An outdoor safety monitoring system based on millimeter wave radar, characterized in that: It includes an electronic device as described in any one of claims 1 to 10, and at least one detection device, the electronic device can establish a pairing relationship with the detection device to establish a communication connection, the detection device includes a millimeter wave radar sensor, and the millimeter wave radar sensor is used to perform detection within the area corresponding to the detection device and obtain radar echo data.

Images