Target object detection data acquisition method, radar equipment, visual equipment and system

Abstract

The invention relates to the technical field of data processing, and discloses a target object detection data acquisition method, radar equipment, visual equipment and a target object detection data acquisition system.The method is applied to the radar equipment, and the radar equipment and the visual equipment are connected and used for detecting a target object. The method comprises the steps of obtaining position information of a target object in a target area, sending the position information to visual equipment, receiving first angle information sent by the visual equipment, detecting the target object according to the first angle information to obtain first target position information corresponding to the target object, and sending the first target position information to the visual equipment; and acquiring detection data in the target area collected at the current moment, and extracting the detection data according to the first target position information to obtain detection data corresponding to the target object. Through the above mode, the radar equipment can extract the required radar original data from the detected data in a targeted and accurate manner.

Description

Technical Field

[0001] Embodiments of the present application relate to the technical field of data processing, and in particular, to a method for collecting target detection data, a radar device, a vision device, and a target detection data collection system. Background Art

[0002] A radar device is an electronic device that uses radio waves for detection and positioning, and is widely used in fields such as military, aviation, and navigation. The radar device mainly obtains the detection data corresponding to the target by capturing the micro-movements of the target through the micro-Doppler effect. The collected and labeled detection data is of great significance for improving the performance of the radar device. For example, the labeled detection data can be used to train models, test, and improve radar algorithms, etc.

[0003] However, the detection data collected by the radar device usually includes data corresponding to all objects within the detection area. For example, data corresponding to objects such as buildings, trees, and birds. The amount of radar raw data is large and the interpretability is poor, making it difficult to accurately obtain the data corresponding to a specific target from the detection data. Summary of the Invention

[0004] In view of the above problems, embodiments of the present application provide a method for collecting target detection data, a radar device, a vision device, and a target detection data collection system, which are used to solve the problem in the prior art that it is difficult to accurately obtain the data corresponding to a specific target from the detection data due to the large amount and poor interpretability of the radar raw data.

[0005] According to the first aspect of the embodiments of the present application, a method for collecting target detection data is provided. The method is applied to a radar device, and the radar device is connected to a vision device, and is respectively used to detect a target. The method includes: obtaining the position information of the target in the target area; sending the position information to the vision device, so that the vision device collects image information according to the position information, and determines the type information of the target corresponding to the position information according to the image information, and sends the first angle information of the target to the radar device when the type information is a preset type; receiving the first angle information sent by the vision device, and detecting the target according to the first angle information to obtain the first target position information corresponding to the target; obtaining the detection data within the target area collected at the current moment, and extracting the detection data corresponding to the target according to the first target position information.

[0006] In an optional manner, extracting the detection data corresponding to the target according to the first target position information specifically includes: obtaining a preset data extraction quantity; extracting a plurality of detection data adjacent to the first target position information from the detection data according to the data extraction quantity to obtain the detection data corresponding to the target.

[0007] In an alternative manner, the second angle information sent by the vision device is obtained in real time, and the target object is tracked and detected according to the second angle information to obtain the second target position information corresponding to the target object; the detection data collected at the current moment is obtained, and the detection data is extracted according to the second target position information to obtain the detection data corresponding to the target object.

[0008] In an alternative manner, the detection data corresponding to the target object is analyzed to obtain the attribute information corresponding to the target object; the time-frequency processing is performed on the detection data corresponding to the target object to obtain the micro-Doppler feature data corresponding to the target object; the micro-Doppler feature data corresponding to the target object is labeled according to the attribute information.

[0009] According to the second aspect of the embodiments of the present application, a method for collecting detection data of a target object is provided, which is applied to a vision device. The vision device is connected to a radar device and is respectively used for detecting a target object. The method includes: receiving the position information of the target object sent by the radar device, and collecting image information according to the position information; determining the type information of the target object corresponding to the position information according to the image information; if the type information is a preset type, sending the first angle information of the target object to the radar device, so that the radar device detects the target object according to the first angle information to obtain the first target position information corresponding to the target object, and enabling the radar device to extract the detection data collected at the current moment according to the first target position information to obtain the detection data corresponding to the target object.

[0010] In an alternative manner, after sending the first angle information of the target object to the radar device, the method further includes: locking and tracking the target object to obtain the second angle information of the target object; sending the second angle information to the radar device in real time, so that the radar device tracks and detects the target object according to the second angle information to obtain the second target position information corresponding to the target object, and extracting the detection data collected at the current moment according to the second target position information to obtain the detection data corresponding to the target object.

[0011] In an alternative manner, the method further includes: collecting the target image information of the target object, and identifying the target image information to determine the pose information of the target object; receiving the track information corresponding to the target object sent by the radar device; determining the motion trend of the target object according to the track information; labeling the target image information according to the pose information and the motion trend.

[0012] According to the third aspect of the embodiments of the present application, a radar device is provided, including a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the data collection method according to any one of the above first aspects.

[0013] According to a fourth aspect of the embodiments of the present application, a vision device is provided, including a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the data acquisition method according to any one of the above second aspects.

[0014] According to a fifth aspect of the embodiments of the present application, a target detection data acquisition system is provided. The system includes a radar device and a vision device. The radar device and the vision device are connected and are respectively used for detecting a target. The radar device is used to obtain the position information of the target in the target area and send the position information to the vision device. The vision device is used to receive the position information of the target sent by the radar device, collect image information according to the position information, and determine the type information of the target corresponding to the position information according to the image information. The vision device is further used to send the angle information of the target to the radar device when the type information is a preset type. The radar device is further used to receive the angle information sent by the vision device and detect the target according to the angle information to obtain the target position information corresponding to the target. The radar device is further used to obtain the detection data collected at the current moment and extract the detection data according to the target position information to obtain the detection data corresponding to the target.

[0015] In the embodiments of the present application, by sending the position information of the target to the vision device, the vision device is used to identify the target in the target area and determine whether the target is a target that needs to perform data acquisition. This not only enables the radar device to specifically and accurately extract the required data from all the data it collects, that is, accurately obtain the detection data corresponding to a specific target from all the detection data, improving the efficiency and accuracy of data acquisition, but also can extract the required detection data according to the first angle information for storage, rather than storing all the data detected by the radar device, effectively reducing the amount of data saved and reducing the subsequent data annotation task volume.

[0016] The above description is only an overview of the technical solutions of the embodiments of the present application. In order to be able to understand the technical means of the embodiments of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the embodiments of the present application more obvious and understandable, the following specific embodiments of the present application are specifically given. Description of the Drawings

[0017] The drawings are only used to illustrate the embodiments and are not considered to be a limitation of the present application. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0018] Figure 1 A structural block diagram of the target detection data acquisition system provided by the embodiments of the present application is shown;

[0019] Figure 2 The structural schematic diagram of the radar device provided by the embodiment of the present application is shown;

[0020] Figure 3 The flowchart of the method for collecting target detection data provided by the embodiment of the present application is shown;

[0021] Figure 4 The structural schematic diagram of the vision device provided by the embodiment of the present application is shown;

[0022] Figure 5 The flowchart of another method for collecting target detection data provided by the embodiment of the present application is shown. Detailed implementation manners

[0023] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0024] Accurately obtaining the detection data corresponding to a specific target from the data collected by the radar device and applying this data to tasks such as model training, testing, and improving radar algorithms can effectively improve the performance of the radar device. For example, with the development of deep learning technology, applying a deep learning model to the radar device can promote the leap of radar technology from traditional signal processing to intelligent perception and decision-making. This can not only improve the performance of the radar device but also expand the application scenarios of the radar system.

[0025] In addition, for models or radar algorithms in different fields, detection data corresponding to different targets are required. For example, for a radar device in the field of target detection, detection data corresponding to multiple targets need to be collected. By improving the radar algorithm with the detection data corresponding to different targets, the ability of the radar device to distinguish different types of targets can be improved; for a radar device in the field of UAV detection, detection data corresponding to the UAV target need to be collected. By improving the radar algorithm with the detection data corresponding to the UAV, the accuracy of the radar device in identifying UAVs can be improved.

[0026] However, the detection data is the data obtained by the radar device detecting the objects in the target area, which usually includes all the electromagnetic wave information reflected from the environment, that is, the data corresponding to all the objects in the detection area (for example, buildings, trees, vehicles, drones, birds, etc.), and the detection data includes analog-to-digital conversion data, range-Doppler spectrum, range-angle spectrum data, etc. This results in a large amount of detection data, and these data are usually stored in the form of numbers and matrices, making it difficult to directly determine the meaning behind these data. For example, it is difficult for annotators to directly determine what the corresponding target object is through these data. This makes it difficult to accurately obtain the required data when collecting the detection data of specific target objects, thereby affecting the effects of subsequent tasks (such as model training, algorithm improvement, etc.).

[0027] Based on this, in order to accurately obtain the detection data corresponding to specific target objects, the present application provides a method for collecting target object detection data. The data is collected through the cooperation of a radar device and a vision device. By utilizing the ability of the vision device to accurately identify the type of target object, the detection data corresponding to specific target objects is accurately extracted from the detection data collected by the radar, effectively reducing the amount of data and improving the annotation efficiency of the detection data. Specifically, after the radar device obtains the position information of the target object, it sends the position information to the vision device to identify the type of the target object through the vision device. When the type of the target object is a preset type, the vision device sends the angle of the target object to the radar device, so that the radar device detects the target object according to the angle and extracts the detection data corresponding to the target object from the collected detection data.

[0028] According to the first aspect of the embodiments of the present application, a system for collecting target object detection data is provided, as Figure 1 shown Figure 1 FIG. shows a structural block diagram of the system for collecting target object detection data provided by the embodiments of the present application. The system includes a radar device 1 and a vision device 2. The radar device 1 and the vision device 2 are connected and are respectively used to detect target objects.

[0029] The radar device 1 is an electronic device that uses radio waves to detect information such as the position, speed, and azimuth of target objects. Specifically, the radar device 1 calculates parameters such as the distance, speed, and angle of target objects by emitting radio waves and receiving the echoes reflected by the target objects. The radar device 1 can be a mechanical scanning radar, a phased array radar, a frequency scanning radar, etc. The radar device 1 can be fixedly arranged in the target area to detect the target area, or can move and detect in the target area by means of patrol.

[0030] The vision device 2 refers to a system that uses optical sensors (such as cameras) and computing technologies to capture, process, and understand environmental information. Among them, the computing technologies can be intelligent algorithms such as object detection and recognition, enabling the vision device 2 to analyze the images collected by the optical sensor through the computing technology, so as to realize the detection, classification, and tracking of specific objects, people, or scenes. The vision device 2 can be a surveillance camera, a network camera, a PTZ camera, etc. The vision device 2 can be fixedly set together with the radar device 1 to form a whole to detect the target objects in the target area, or can be separated from the radar device 1 and independently set in the target area.

[0031] The radar device 1 and the vision device 2 can be connected through a network, including but not limited to one or more of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a 4G / 5G network, WIFI, Bluetooth, and a peer-to-peer (P2P) communication network, or can also be connected through wires such as optical fibers, cables, and twisted pairs.

[0032] Specifically, first, the radar device 1 is used to detect the target area, obtain the position information of the target objects in the target area, and send the position information to the vision device 2. Then, after receiving the position information of the target objects sent by the radar device 1, the vision device 2 will adjust the detection direction of the vision device 2 according to the position information to collect the image information corresponding to the position information. Next, the vision device 2 determines the type of the target object corresponding to the position information according to the image information, and judges whether the type of the target object is a preset type. When the type of the target object is the preset type, the vision device 2 sends the angle information of the target object to the radar device 1. Finally, the radar device 1 detects the target object according to the received angle information to obtain the target position information corresponding to the target object, and extracts the detection data corresponding to the target object from the detection data collected at the current moment according to the detected target position information.

[0033] The target object detection data acquisition system includes a radar device as Figure 2 shown. Figure 2 FIG. shows a schematic structural diagram of the radar device provided by an embodiment of the present application. The specific implementation of the radar device is not limited in the specific embodiments of the present application.

[0034] The radar device 1 accurately extracts the detection data corresponding to specific target objects from all the detection data collected by it by executing the following target detection data acquisition method. Specifically, as Figure 2 shown, the radar device may include: a processor 11 and a memory 12.

[0035] Among them, the memory 12 is used to store the computer program 13. The memory 12 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory. The computer program 13 may include computer-executable instructions.

[0036] The processor 11 is used to execute the computer program 13 to implement the embodiments of the target detection data acquisition method described below.

[0037] The processor 11 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the radar device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0038] Figure 3 The flowchart of the target detection data acquisition method provided by the embodiments of the present application is shown. This method is applied to the radar device in the above target detection data acquisition system and is executed by the controller of the radar device (such as the processor of the radar device). The radar device is connected to the vision device and is respectively used to detect the target object, such as Figure 3 As shown, the method includes the following steps:

[0039] Step S110: Obtain the position information of the target object in the target area.

[0040] Among them, the target area is the detection area of the radar device, and the target object is the target detected by the radar device in the target area, which may be a target such as a drone, a person, a vehicle, etc. The position information is used to characterize the position of the target object, and may be the azimuth angle, elevation angle, distance information, etc. of the target object. Specifically, taking the phased array radar as an example of the radar device, when the radar device is working, it will control the beam to scan the target area according to a certain rule, and collect the electromagnetic wave data reflected by each target object in the target area. By processing the electromagnetic wave data, real number fast Fourier transform data or pulse compression data can be formed, that is, the detection data corresponding to the target object. By analyzing the detection data, the position information corresponding to each target object can be obtained, such as azimuth angle, elevation angle, distance, etc.

[0041] Step S120: Send the position information to the vision device, so that the vision device collects image information according to the position information, and determines the type information of the target object corresponding to the position information. When the type information is the preset type, send the first angle information of the target object to the radar device.

[0042] Among them, the vision device is used to collect the image information corresponding to the target object, analyze the image information, and determine the type information of the target object, that is, determine the type of the target object according to the image information. Specifically, algorithms such as target recognition and target detection can be used. For example, traditional computer vision methods such as support vector machines and random forests, or deep learning methods such as convolutional neural networks, residual networks, and YOLO models. If the type information is a preset type, it means that the target object corresponding to the position information is the target object for which data collection is required, that is, the detection data corresponding to this target object needs to be extracted from the detection data collected by the radar device. The first angle information is used to characterize the position information of the target object, that is, the radar device controls the beam to irradiate the position corresponding to the first angle information, and the detection data corresponding to the target object can be obtained.

[0043] Taking the detection data collection of an unmanned aerial vehicle (UAV) as an example, the vision device takes pictures of the target object according to the position information, collects the image information containing the target object, and then analyzes the image information to identify the type information of the target object in the image information, that is, determines whether the target object corresponding to the position information is a UAV or a bird. Then, it judges whether the target object corresponding to the position information is a UAV. If the target object corresponding to the position information is a UAV, the first angle information corresponding to this target object is sent to the radar device to tell the radar device that the target object corresponding to the first angle information is the target object that the radar device needs to focus on.

[0044] In addition, in order to collect clear images, when the vision device collects image information according to the position information, it can be automatically adjusted according to the collected images, that is, adjust the zoom and focus of the vision device to appropriate parameters to obtain clearer image information. Further, since there may be measurement errors in the radar device, or there may be deviations in the position information received by the vision device when the target object is in a moving state, the vision device can search within a preset range centered on the position information when collecting image information until the target appears in the image collected by the vision device, thereby ensuring that the target object is in the image information and guaranteeing the accuracy of the type information.

[0045] Step S130: Receive the first angle information sent by the vision device, and detect the target object according to the first angle information to obtain the first target position information corresponding to the target object.

[0046] Among them, when the radar device receives the first angle information, it means that the target object corresponding to the first angle information is the target for which data collection is required. Taking the data collection of a UAV as an example, when the radar device receives the first angle information, it can determine that the target object corresponding to the first angle information is a UAV. Only by collecting the detection data of the target object corresponding to the first angle information can the detection data of the UAV be collected.

[0047] Specifically, after the radar device receives the first angle information sent by the vision device, it can emit a beam towards the first angle information to irradiate the target corresponding to the first angle information, obtain the first target position information corresponding to the target, and finally extract the detection data corresponding to the target from the detection data collected by the radar device, so as to collect the detection data of the UAV.

[0048] The first target position information may include data such as azimuth, elevation angle, and distance, so that the radar device can accurately extract data. In addition, if the radar device is a phased array radar or other devices, these devices mainly obtain the detection data of the target by changing the beam direction and can only scan one direction at the same time. When the radar device performs detection according to the first angle information, it can only collect the detection data of the target located at the first angle information. At this time, the first position information may only include distance data, that is, the detection data corresponding to the target can be extracted from all the detection data collected by the radar device according to the distance data.

[0049] Step S140: Obtain the detection data within the target area collected at the current moment, and extract the detection data corresponding to the target according to the first target position information.

[0050] Among them, the detection data is the original data collected by the radar device. These data include mode conversion data, range-Doppler spectrum, and range-angle spectrum data. Not only is the amount of data large, but these data are usually stored in the form of numbers and matrices, with poor interpretability, that is, it is difficult for users to directly determine which target these data belong to.

[0051] Taking the collection of the detection data corresponding to the UAV as an example, the radar device can detect all flying objects (such as UAVs, birds, etc.) within the target area and obtain the detection data corresponding to these flying objects. Moreover, these detection data are stored in the same structure, resulting in the detection data corresponding to different flying objects being relatively similar. Just observing these detection data, it is difficult to accurately judge which data are the detection data corresponding to the required UAV and which data are the detection data corresponding to the birds to be discarded.

[0052] However, after the target is identified by the vision device, it can be determined which flying objects are UAVs and which are birds. When the radar device receives the first angle information, it can determine that the target corresponding to the first angle information is the UAV. Only by obtaining the first target position information of the target corresponding to the first angle information and then extracting the detection data corresponding to the first target position information from the data collected by the radar device can the detection data corresponding to the UAV be collected.

[0053] In the above embodiments, by sending the position information of the target object to the vision device, the vision device can identify the target object in the target area and determine whether the target object is the target object for which data collection is required. This not only enables the radar device to extract the required data from all the data it collects in a targeted and accurate manner, that is, accurately obtain the detection data corresponding to a specific target object from all the detection data, improving the efficiency and accuracy of data collection, but also can extract the required detection data according to the first angle information for storage, rather than storing all the data detected by the radar device, effectively reducing the amount of stored data and reducing the subsequent data annotation task volume.

[0054] Further, in order to completely collect the detection data corresponding to the target object, step S140 specifically includes the following steps:

[0055] Step S141: Obtain the preset data extraction quantity.

[0056] Step S142: According to the data extraction quantity, extract multiple detection data adjacent to the first target position information from the detection data to obtain the detection data corresponding to the target object.

[0057] Among them, the data extraction quantity is the number of detection data extracted from the detection data according to the first target position information. The data extraction quantity can be set according to the size of the target object and the distance at which the radar device collects data. Specifically, taking the detection data collection of an unmanned aerial vehicle as an example, if the width or length of the unmanned aerial vehicle is 2m and the distance at which the radar device collects data is 1m, then there are at least two detection data in the detection data collected by the radar device that are the detection data corresponding to the unmanned aerial vehicle. Therefore, the data extraction quantity can be set to 3, 4, 5, etc.

[0058] In addition, since the radar device collects detection data at a preset distance, when the position of the target object is not the position where the radar device collects detection data, the data extraction quantity can extract all the detection data near the target object to ensure the integrity of the detection data corresponding to the target object. As an example, assume that the radar device collects detection data at a distance of 1m. If the detection distance of the radar device is 10m, then the radar device can collect the detection data corresponding to the 10 positions of 0m, 1m, 2m, 3m, 4m, 5m, 6m, 7m, 8m, 9m, and 10m. When the first target position information of the target object is 1.5m, the detection data corresponding to the two positions of 1m and 2m may both be the detection data of the target object. Through the data extraction quantity, the detection data corresponding to these two positions can be extracted to avoid omission and ensure the integrity of the detection data of the target object.

[0059] In the above embodiments, by setting the data extraction quantity, all the detection data corresponding to the target object can be extracted from the detection data, ensuring that the detection data of the target object can be obtained completely, and further guaranteeing the accuracy of subsequent tasks.

[0060] Further, in order to improve the efficiency of data collection, the method further includes:

[0061] Step S151: Obtain the second angle information sent by the vision device in real time, and perform tracking detection on the target object according to the second angle information to obtain the second target position information corresponding to the target object.

[0062] Step S152: Obtain the detection data collected at the current moment, and extract the detection data corresponding to the target object according to the second target position information.

[0063] Among them, for a radar device, such as a phased array radar device, the radar device will control the beam to scan the target area according to a certain rule during operation. When a target object is found, the radar device will switch to the tracking mode to establish a target track, that is, the radar device will intermittently control the beam to irradiate the target object at a specific frequency to maintain the update of the information of the target object. At this time, the data collected by the radar device during the rest of the time when it irradiates outside the target object are all data irrelevant to the target object, and these data will affect the efficiency of data collection. For example, the radar device will first emit a frame of beam towards the target object, and the waveform can be N pulses or N CHIRPs. After the emission is completed, it will pause for a period of time, and then emit the next frame of beam towards the target object. During the paused period, the detection data collected by the radar device is irrelevant to the target object, that is, these data are data that do not need to be collected.

[0064] After the vision device discovers the target object, it can lock and autonomously track the target object, so that the second angle information corresponding to the target object can be continuously obtained. As long as the radar device emits a beam according to the second angle information, it can realize the locking and tracking of the target object. Therefore, the ability of the vision device to lock and track can be utilized. After the radar device discovers the target object, it can receive the second angle information sent by the vision device in real time and control the beam to continuously irradiate the target object, that is, perform tracking detection on the target object, so that the detection data collected by the radar device at each moment contains the detection data of the target object.

[0065] Specifically, the tracking and detection of a target by a radar device means that the radar device continuously emits beams towards the target, such that there is no additional time interval between each frame of the beams emitted by the radar device towards the target. The radar device can receive the second angle information sent by the vision device in real time and control the beam to continuously emit beams towards the second angle information. For example, the radar device can adjust the beams it emits. It can adjust the pulse regime or the continuous wave regime. After the radar device receives the second angle information, it adjusts the waveform of the beam and continuously emits M frames of beams towards the second angle information, and there is no additional time interval between frames.

[0066] In the above embodiment, by receiving the second angle information sent by the vision device in real time, the radar device can track and detect the target, so that the radar device can continuously detect the target and collect the detection data corresponding to the target, effectively improving the efficiency of data collection.

[0067] Further, in order to improve the efficiency of data collection, the method further includes:

[0068] Step S161: Analyze the detection data corresponding to the target to obtain the attribute information corresponding to the target.

[0069] Step S162: Perform time-frequency processing on the detection data corresponding to the target to obtain the micro-Doppler feature data corresponding to the target.

[0070] Step S163: Label the micro-Doppler feature data corresponding to the target according to the attribute information.

[0071] Among them, the attribute information is used to explain what the detection data represents. That is, after specific analysis of the detection data, its corresponding attribute information can be obtained. For example, information such as the position where the target is located, the moving speed, and the moving direction. Specifically, methods such as rule matching and pre-trained deep learning models can be used to determine the attribute information corresponding to the detection data, that is, the attribute information of the target. Micro-Doppler features refer to the Doppler frequency modulation information caused by the micro-movement of the target. These micro-movements include vibrations, rotations, or the movement of non-rigid parts of the target. For example, the rotor movement outside the UAV body, the arm swing outside the human body, etc.

[0072] Specifically, time-frequency processing can be performed on the detection data by means of a sliding window. For example, short-time Fourier transform, wavelet transform, etc. The size of the sliding window can be set according to the size of the target, the distance at which the radar device collects data, etc. In this way, the micro-Doppler time-frequency diagram corresponding to the target can be obtained, that is, the micro-Doppler feature data corresponding to the target. In addition, the detection data corresponding to the target can also be processed offline by setting sliding windows of multiple sizes to generate multiple micro-Doppler time-frequency diagrams with different distance dimensions.

[0073] Finally, by annotating the micro-Doppler feature data corresponding to the target object with the attribute information corresponding to the target object (i.e., the attribute information obtained after analyzing the detection data), a data set with annotation information can be formed, enabling subsequent tasks to be carried out through this data set, such as model training, radar algorithm improvement, and other tasks.

[0074] In the above embodiments, by utilizing the ability of the radar device itself to accurately obtain the attribute information of the target object, after extracting the detection data of the target object, the attribute information such as the position, distance, and speed corresponding to the detection data can be directly analyzed from the data, and the micro-Doppler feature data corresponding to the detection data can be annotated with these attribute information. This can not only achieve automatic annotation of the data, improve the efficiency of data annotation, but also have high annotation accuracy, providing guarantee for subsequent tasks.

[0075] Furthermore, the target object detection data acquisition system further includes a visual device as Figure 4 shown. Figure 4 FIG. shows a schematic structural diagram of the visual device provided in the embodiment of the present application. The specific implementation of the visual device is not limited in the specific embodiments of the present application.

[0076] The visual device 2 enables the detection device to accurately extract the detection data corresponding to a specific target object from all the detection data collected by it by executing the following target detection data acquisition method. Specifically, as Figure 4 shown, the visual device 2 may include: a processor 21 and a memory 22.

[0077] Among them, the memory 22 is used to store a computer program 23. The memory 22 may include a high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory. The computer program 23 may include computer-executable instructions.

[0078] The processor 21 is used to execute the computer program 23 to implement the embodiment of the target object detection data acquisition method provided in the fourth aspect of the present application embodiment above.

[0079] The processor 21 may be a CPU, or an ASIC, or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the visual device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0080] Figure 5The flowchart of another method for collecting target detection data provided by an embodiment of the present application is shown. This method is applied to the vision device in the above-mentioned target detection data collection system and is executed by the controller of the vision device (such as the processor of the vision device). The vision device is connected to the radar device and is respectively used for detecting the target. As Figure 5 shown, the method includes the following steps:

[0081] Step S210: Receive the position information of the target sent by the radar device, and collect image information according to the position information.

[0082] Among them, after receiving the position information, the vision device controls the camera to take a picture of the position information to collect the image of the position information. Of course, due to possible errors in the radar detection device, or when the target is in a moving state, the position information may be deviated. The vision device can search for the target centered on the position information to ensure that the collected image information contains the target corresponding to the position information. In addition, after the vision device collects the image information containing the target, it can also detect the position and clarity of the target in the image, and adjust the shooting angle, as well as the parameters of zooming and focusing of the vision device according to the detection results, so that the target can be located in the middle of the image and the target in the image is clearer.

[0083] Step S220: Determine the type information of the target corresponding to the position information according to the image information.

[0084] Specifically, algorithms such as target recognition and target detection (for example, traditional computer vision methods such as support vector machines and random forests, or deep learning methods such as convolutional neural networks, residual networks, and YOLO models) can be used to identify the target in the image information to determine whether the target is an object for which data needs to be collected. Taking the data collection of an unmanned aerial vehicle as an example, by analyzing the image information, it can be determined whether the target in the image is an unmanned aerial vehicle or a bird. If the target in the image is an unmanned aerial vehicle, it means that the target corresponding to the position information is the target for which data needs to be collected, that is, the radar device needs to collect the detection data corresponding to this target to generate a data set corresponding to the unmanned aerial vehicle. If the target in the image is a bird, it means that the detection data corresponding to this target is not needed and these detection data can be discarded.

[0085] Step S230: If the type information is a preset type, send the first angle information of the target to the radar device, so that the radar device detects the target according to the first angle information to obtain the first target position information corresponding to the target, and the radar device extracts the detection data collected at the current moment according to the first target position information to obtain the detection data corresponding to the target.

[0086] Among them, if the type of the target object is a preset type, the first angle information of the target object is sent to the radar device, so as to tell the radar device that the target object corresponding to the first angle information is the target object for which data collection is required, and the detection data of the target object can be extracted from the data collected by the radar device to complete the data collection of the target object.

[0087] In the above embodiment, an image of the position information is collected by the vision device, and the image is analyzed to determine whether the target object in the image is the object for which data collection is required. When the target object is the object for which data collection is required, the first angle information of the target object is sent to the radar device. On the one hand, it enables the radar device to specifically and accurately extract the required data from all the data it has collected, improving the efficiency and accuracy of data collection. On the other hand, the radar device can retain only the required detection data according to the first angle information, rather than saving all the data detected by the radar device, effectively reducing the amount of data saved and reducing the task of data annotation.

[0088] Further, to improve the efficiency of data collection, the method further includes the following steps:

[0089] Step S240: Lock and track the target object to obtain the second angle information of the target object.

[0090] Step S250: Send the second angle information to the radar device in real time, so that the radar device tracks and detects the target object according to the second angle information, obtains the second target position information corresponding to the target object, and extracts the detection data corresponding to the target object from the detection data collected at the current moment.

[0091] Specifically, the lock and track of the vision device refers to the technology of real-time identifying and continuously following a specific target through vision algorithms or sensor technologies. When the vision device determines that the target object is the object for which data collection is required (that is, the type information of the target object is a preset type), the target object can be locked and tracked to continuously obtain the second angle information of the target object and send the second angle information to the radar device in real time, so that the radar device enters the staring mode (that is, the radar device continuously emits beams at a certain angle), and tracks and detects the target object, so that there is no extra time interval between each frame of beams emitted by the radar device towards the target object, that is, each frame of beams emitted by the radar device can collect the detection data corresponding to the target object.

[0092] In the above embodiment, by locking and tracking the target object and sending the continuously obtained second angle information of the target object to the radar device in real time, the radar device can continuously collect the detection data corresponding to the target object, effectively improving the efficiency of data collection.

[0093] Furthermore, to enrich the annotation information of the image information, the method further includes the following steps:

[0094] Step S261: Collect the target image information of the target object, and identify the target image information to determine the pose information of the target object.

[0095] Among them, the pose information is used to characterize the posture of the target object. For example, the forward tilt and backward tilt of a drone, the standing and bending postures of a human body, or the direction the target object is facing, etc. Specifically, traditional computer vision methods such as support vector machines and random forests, or deep learning methods such as convolutional neural networks, residual networks, and YOLO models can be used to identify the target object in the target image information to determine the state of the target object. For example, information such as the tilt direction and tilt angle of a drone, and the orientation and offset angle of a human body.

[0096] Step S262: Receive the trajectory information corresponding to the target object sent by the radar device.

[0097] Step S263: Determine the motion trend of the target object according to the trajectory information.

[0098] Among them, the motion trend of the target object is the motion state of the target object. For example, information such as the target object approaching the vision device, the target object moving away from the vision device, or the target object stopping moving. Only the pose of the target can be recognized from the image information collected by the vision device, but the motion state of the target object cannot be recognized. Taking the target object as a drone as an example, the vision device can only recognize whether the drone is in a forward tilt state or a backward tilt state, but cannot determine whether the drone is in a flying trend of moving away, approaching, flying horizontally, or hovering. The radar device can continuously obtain the position information of the target object according to a certain rule, and generate the trajectory corresponding to the target object based on the position information of the target object. The motion trend of the target object can be obtained from the trajectory. Therefore, the vision device can determine the motion trend of the target object by obtaining the trajectory information of the target object.

[0099] Step S264: Annotate the target image information according to the pose information and the motion trend.

[0100] Finally, annotate the target image information according to the pose information and the motion trend to enrich the annotation information corresponding to the target image information. Taking the target object as a drone as an example, the annotation information corresponding to the drone not only includes posture information such as forward tilt or backward tilt, but also includes flight states such as moving away, approaching, flying horizontally, or hovering.

[0101] In the above embodiments, the vision device determines the motion trend of the target object in the target image information by receiving the track information sent by the radar device. Furthermore, not only can the pose information of the target object be marked on the target image information, but also the motion trend can be marked, enriching the annotation information of the target image information and providing more information content for subsequent tasks.

[0102] An embodiment of the present application provides a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned embodiment of the method for collecting target object detection data applied to the radar device, or when the computer program is executed by a processor, it implements the above-mentioned embodiment of the method for collecting target object detection data applied to the vision device.

[0103] An embodiment of the present application provides a computer program. The computer program can be executed by a processor to implement the above-mentioned embodiment of the method for collecting target object detection data applied to the radar device, or the computer program can be executed by a processor to implement the above-mentioned embodiment of the method for collecting target object detection data applied to the vision device.

[0104] An embodiment of the present application provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the above-mentioned embodiment of the method for collecting target object detection data applied to the radar device, or when the computer program is executed by a processor, it implements the above-mentioned embodiment of the method for collecting target object detection data applied to the vision device.

[0105] In several embodiments provided by the present application, if any function is implemented in the form of a software function module / unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, part or all of the technical solutions of the present application can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be an electronic device such as a personal computer or a server) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store computer program code.

[0106] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. A variety of general-purpose systems may also be used in conjunction with the teachings based herein. The structure required to construct such systems will be apparent from the above description. Additionally, the embodiments of the present application are not directed to any particular programming language. It should be understood that the content of the present application described herein can be implemented using a variety of programming languages, and the descriptions of specific languages above are for disclosing the best mode of the present application.

[0107] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a claim listing several devices, several units or modules of these devices may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

[0108] The above-described embodiments merely represent several implementation manners of the present application, and the description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for collecting target detection data, characterized in that, Applied to a radar device, the radar device is connected to a vision device, and they are respectively used to detect a target object. The method includes: Obtain the position information of the target object in the target area; Send the position information to the vision device, so that the vision device acquires image information according to the position information, determines the type information of the target object corresponding to the position information according to the image information, and sends the first angle information of the target object to the radar device when the type information is a preset type; Receive the first angle information sent by the vision device, and detect the target object according to the first angle information to obtain the first target position information corresponding to the target object; Obtain the detection data within the target area collected at the current moment, and extract the detection data corresponding to the target object from the detection data according to the first target position information.

2. The method for collecting target detection data according to claim 1, wherein The extracting the detection data corresponding to the target object from the detection data according to the first target position information specifically includes: Obtain a preset data extraction quantity; According to the data extraction quantity, extract a plurality of detection data adjacent to the first target position information from the detection data to obtain the detection data corresponding to the target object.

3. The method for collecting target detection data according to claim 1, wherein The method further includes: Obtain the second angle information sent by the vision device in real time, and perform tracking detection on the target object according to the second angle information to obtain the second target position information corresponding to the target object; Obtain the detection data collected at the current moment, and extract the detection data corresponding to the target object from the detection data according to the second target position information.

4. The method for collecting target detection data according to claim 1, wherein The method further includes: Analyze the detection data corresponding to the target object to obtain the attribute information corresponding to the target object; Perform time-frequency processing on the detection data corresponding to the target object to obtain the micro-Doppler feature data corresponding to the target object; Label the micro-Doppler feature data corresponding to the target object according to the attribute information.

5. A method for collecting target detection data, characterized in that, Applied to a vision device, the vision device is connected to a radar device, and they are respectively used to detect a target object. The method includes: Receive the position information of the target object sent by the radar device, and acquire image information according to the position information; Determine the type information of the target object corresponding to the position information according to the image information; If the type information is a preset type, send the first angle information of the target object to the radar device, so that the radar device detects the target object according to the first angle information to obtain the first target position information corresponding to the target object, and enables the radar device to extract the detection data corresponding to the target object from the detection data collected at the current moment according to the first target position information.

6. The method for collecting object detection data according to claim 5, wherein After sending the first angle information of the target object to the radar device, the method further includes: Lock and track the target object, and obtain the second angle information of the target object; Send the second angle information to the radar device in real time, so that the radar device tracks and detects the target object according to the second angle information, obtains the second target position information corresponding to the target object, and extracts the detection data collected at the current moment according to the second target position information to obtain the detection data corresponding to the target object.

7. The method for collecting object detection data according to claim 6, wherein The method further includes: Collect the target image information of the target object, and identify the target image information to determine the pose information of the target object; Receive the track information corresponding to the target object sent by the radar device; Determine the motion trend of the target object according to the track information; Annotate the target image information according to the pose information and the motion trend.

8. A radar device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the target object detection data acquisition method according to any one of claims 1 to 4.

9. A vision device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the target object detection data acquisition method according to any one of claims 5 to 7.

10. A target detection data acquisition system, characterized in that, The system includes a radar device and a vision device. The radar device and the vision device are connected and are respectively used for detecting a target object; The radar device is used to obtain the position information of the target object in the target area and send the position information to the vision device; The vision device is used to receive the position information of the target object sent by the radar device, collect image information according to the position information, and determine the type information of the target object corresponding to the position information according to the image information; The vision device is further used to send the angle information of the target object to the radar device when the type information is a preset type; The radar device is further used to receive the angle information sent by the vision device and detect the target object according to the angle information to obtain the target position information corresponding to the target object; The radar device is further used to obtain the detection data collected at the current moment and extract the detection data according to the target position information to obtain the detection data corresponding to the target object.

Images