Target detection method, device, electronic device, and storage medium

By combining radar and camera data, using position, distance and depth information to match targets, the problem of inaccurate matching of radar point clouds and visual cameras is solved, and the accuracy and reliability of target detection are improved.

CN113673569BActive Publication Date: 2024-11-05ZHEJIANG DAHUA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110825923.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-21
Publication Date
2024-11-05
Estimated Expiration
2041-07-21

AI Technical Summary

Technical Problem

In the prior art, when detecting targets, the matching between the radar point cloud and the visual camera is not accurate enough, resulting in inaccurate detection results.

Method used

By combining radar data and camera data, target matching is used using position coordinates, distance information and depth information, the depth information is calculated using the binocular camera parallax algorithm, and traversal matching is performed through weighted summing, and radar and camera data are fused to improve detection accuracy.

Benefits of technology

It improves the accuracy of object detection, reduces the dependence on coordinate system mapping function, and enhances the reliability of detection results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113673569B_ABST
    Figure CN113673569B_ABST
Patent Text Reader

Abstract

The present invention provides a target detection method, device, electronic device, and storage medium, wherein the target detection method comprises: obtaining first position coordinates and distance information of each first detection target in the first detection target set based on radar data of the first detection target set, and obtaining second position coordinates and depth information of each second detection target in the second detection target set based on camera data of the second detection target set; matching the first detection target with the second detection target using the first position coordinates, the second position coordinates, the distance information, and the depth information. This can improve the accuracy of target detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of target recognition technology, and in particular to a target detection method, device, electronic equipment, and storage medium. Background Art

[0002] When performing target detection, it is necessary to fuse the target's appearance information and the target's motion information. In the prior art, the commonly used method is to traverse the external rectangular box of the target detected by vision, and the point closest to the projection point of the radar point cloud target in the visual camera coordinate system is determined as the target detected by the radar and the camera together, but the detection results of this method are not accurate enough. Summary of the invention

[0003] The present invention provides a target detection method, device, electronic equipment, and storage medium, which can improve the accuracy of target detection.

[0004] To solve the above technical problems, the first technical solution provided by the present invention is: providing a target detection method, comprising: obtaining first position coordinates and distance information of each first detection target in the first detection target set based on radar data of the first detection target set, and obtaining second position coordinates and depth information of each second detection target in the second detection target set based on camera data of the second detection target set; matching the first detection target with the second detection target using the first position coordinates, the second position coordinates, the distance information and the depth information.

[0005] The depth information of each second detection target in the second detection target set is obtained based on the camera data of the second detection target set, including: using a binocular camera parallax algorithm to calculate the depth information of each second detection target based on the camera data of the second detection target set.

[0006] Among them, the step of matching the first detection target with the second detection target using the first position coordinates, the second position coordinates, the distance information and the depth information includes: calculating the distance between the first detection target and the second detection target based on the first position coordinates and the second position coordinates; and calculating the depth difference between the first detection target and the second detection target based on the distance information and the depth information; matching the first detection target with the second detection target using the distance and the depth difference.

[0007] Among them, the step of matching the first detection target with the second detection target using the distance and the depth difference includes: traversing and matching each first detection target in the first detection target set with each second detection target in the second detection target set using the distance and the depth difference.

[0008] Among them, the step of traversing and matching each first detection target in the first detection target set with each second detection target in the second detection target set by using the distance and depth difference includes: performing weighted summation on the distance and the depth difference, and traversing and matching each first detection target in the first detection target set with each second detection target in the second detection target set based on the result of the weighted summation.

[0009] Among them, the step of performing weighted summation on the distance and the depth difference, and traversing and matching each first detection target in the first detection target set with each second detection target in the second detection target set based on the result of the weighted summation includes: determining the first detection target and the second detection target corresponding to the result with the smallest weighted summation as the same detection target.

[0010] Among them, the first position coordinates of each first detection target in the first detection target set are obtained based on the radar data of the first detection target set, including: mapping the radar data to a reference pixel coordinate system to obtain a pixel coordinate point set, and the radar data is radar point cloud data; obtaining the first position coordinates of each first detection target based on the pixel coordinate point set.

[0011] Among them, obtaining the second position coordinates of each second detection target in the second detection target set based on the camera data of the second detection target set includes: extracting the area to be detected from the image to be detected based on the first position coordinates, and the image to be detected is the camera data; performing target detection on the area to be detected to obtain the second position coordinates of each second detection target in the area to be detected.

[0012] The method further includes: fusing radar data of a matched first detection target and camera data of a matched second detection target, wherein the matched first detection target and the matched second detection target are determined to be the same detection target.

[0013] Among them, before the steps of obtaining the first position coordinates and distance information of each first detection target in the first detection target set based on the radar data of the first detection target set, and obtaining the second position coordinates and depth information of each second detection target in the second detection target set based on the camera data of the second detection target set, it also includes: acquiring the radar data and the camera data by using a time synchronization method.

[0014] Among them, the step of obtaining radar data and camera data using the time synchronization method includes: obtaining radar data and camera data using the software time synchronization method; or obtaining radar data and camera data using the hardware time synchronization method.

[0015] Among them, the step of obtaining radar data and camera data using the time synchronization method includes: in response to the movement speed of the detection target being less than the preset speed, obtaining radar data and camera data using the software time synchronization method; in response to the movement speed of the detection target being greater than the preset speed, obtaining radar data and camera data using the hardware time synchronization method.

[0016] Among them, before the steps of obtaining the first position coordinates and distance information of each first detection target in the first detection target set based on the radar data of the first detection target set, and obtaining the second position coordinates and depth information of each second detection target in the second detection target set based on the camera data of the second detection target set, it also includes: performing Kalman filtering on the radar data.

[0017] The first position coordinates are the center point coordinates of the first detection target, and the second position coordinates are the center point coordinates of the second detection target.

[0018] To solve the above technical problems, the second technical solution provided by the present invention is: to provide a target detection device, including: an acquisition module, used to obtain the first position coordinates and distance information of each first detection target in the first detection target set based on the radar data of the first detection target set, and to obtain the second position coordinates and depth information of each second detection target in the second detection target set based on the camera data of the second detection target set; a matching module, using the first position coordinates, the second position coordinates, the distance information and the depth information to match the first detection target with the second detection target.

[0019] To solve the above technical problems, the third technical solution provided by the present invention is: to provide an electronic device, comprising: a memory and a processor, wherein the memory stores program instructions, and the processor calls the program instructions from the memory to execute any of the above methods.

[0020] In order to solve the above technical problems, the fourth technical solution provided by the present invention is: providing a computer-readable storage medium storing a program file, and the program file can be executed to implement any of the above methods.

[0021] The beneficial effect of the present invention is different from that of the prior art. When matching a first detection target with a second detection target, the present invention not only considers the positional relationship between the first detection target and the second detection target, but also introduces depth information of the first detection target and the second detection target, thereby improving the accuracy of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0023] Figure 1 It is a schematic diagram of the flow chart of the first embodiment of the target detection method of the present invention;

[0024] Figure 2 is a flow chart of a second embodiment of the target detection method of the present invention;

[0025] Figure 3 is a schematic flow chart of a third embodiment of the target detection method of the present invention;

[0026] Figure 4 A schematic diagram of the structure of an embodiment of a target detection device of the present invention;

[0027] Figure 5 It is a structural schematic diagram of an embodiment of an electronic device of the present invention;

[0028] Figure 6 It is a schematic diagram of the structure of an embodiment of a computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0029] In the prior art, the generally adopted method is to traverse the circumscribed rectangular box of the target detected visually, and the point closest to the projection point of the radar point cloud target in the visual camera coordinate system is determined as the target detected by the radar and the camera. This method requires the radar point cloud target to be projected onto the camera coordinate system, which involves a mapping function. The accuracy of the function directly affects the detection accuracy, making the target detection results in the prior art inaccurate. The present invention provides a target detection method that introduces depth information to make the target detection results more accurate. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0030] See also Figure 1 , is a flow chart of a first embodiment of the target detection method of the present invention, which specifically includes:

[0031] Step S11: obtaining first position coordinates and distance information of each first detection target in the first detection target set based on the radar data of the first detection target set, and obtaining second position coordinates and depth information of each second detection target in the second detection target set based on the camera data of the second detection target set.

[0032] In modern society, security is increasingly valued by the public, security products are emerging in an endless stream, the application field of security is constantly expanding, and security technology is developing rapidly. Regional monitoring technology based on millimeter-wave radar has been a hot topic in the past five years. Traditional security terminal equipment is mainly visible light cameras, but visible light cameras cannot work at night. Although infrared cameras can supplement the defects of visible light cameras, this move undoubtedly increases the cost and difficulty of operation. In addition, optical sensors are also affected by weather. In foggy or rainy and snowy days, the monitoring effect is not satisfactory. Millimeter-wave radar actively emits electromagnetic waves and receives signals of the same frequency. It has a very high detection probability for moving objects or objects with large RCS (Radar Cross Section). Millimeter-wave radar can work 24 hours a day and is less affected by weather. Therefore, there is a strong demand for monitoring products based on millimeter-wave radar in the market. Radar can monitor multiple targets, extract targets of interest to users from multiple targets, and terminate / filter targets that users are not interested in or false targets as soon as possible. One of the purposes of target trajectory classification is to screen / filter targets. For example, in a park, a level 3 wind blows occasionally, and the trees sway, forming a low-speed and small-range target track. The target type is a non-human, non-car, and non-animal target. This type of target does not need to be reported, or the track termination method should be called as soon as possible to delete it. If it is a puppy passing through the garden, since it is not the target of the user's attention (the target of the user's attention is a person or a car), the track should also be terminated in time. If it is a track formed by a pedestrian, the radar will output the pedestrian track information to the camera, and the camera will take pictures or videos of it according to the track space position coordinates provided by the radar. The radar can obtain the target's measurement information including: distance, angle, radial speed (RadialSpeed), and radar reflection area (RCS). The radar can set the measurement cycle (signal transmission and reception cycle) as needed. The measurement cycle is generally set to 0.1 seconds, that is, the operating frequency is 10HZ.

[0033] Video / images can obtain high-accuracy identification information of the target, such as the target's appearance information, but cannot obtain the target's motion information and spatial position coordinates. By fusing radar data with camera data, the target's appearance information, motion information and spatial position coordinates can be obtained.

[0034] In this embodiment, when matching the first detection target with the second detection target, it is necessary to first obtain the first position coordinates of each first detection target in the first detection target set based on the radar data of the first detection target set, and obtain the second position coordinates of each second detection target in the second detection target set based on the camera data of the second detection target set.

[0035] For example, the radar data is point cloud data, and the first position coordinates of each first detection target can be obtained based on the radar data. Specifically, the radar data is mapped to a reference pixel coordinate system to obtain a pixel coordinate point set. When mapping the radar data to the reference pixel coordinate system, a mapping function is applied. The first position coordinates of the first detection target are obtained based on the pixel coordinate point set.

[0036] In one embodiment, the radar data may also be subjected to Kalman filtering to filter out interference signals, and then the filtered radar data may be mapped to a reference pixel coordinate system to obtain a set of pixel coordinate points, thereby making the obtained set of pixel coordinate points more accurate, and further, the obtained first position coordinates may also be more accurate.

[0037] For another example, when the camera device performs target detection, it needs to capture an image to obtain the image to be detected, and the image to be detected is the camera data. The area to be detected can be extracted from the image to be detected based on the first position coordinates. In a specific embodiment, the area to be detected can be extracted from the image to be detected based on the attention mechanism. Target detection is performed on the area to be detected to obtain the second position coordinates of each second detection target in the area to be detected. In a specific embodiment, a deep neural network algorithm can be used to perform target detection on the area to be detected, and then the second position coordinates of each second detection target can be obtained.

[0038] For example, in one embodiment, the radar data of the first detection target set obtained by the radar device includes radar data of two first detection targets, namely, radar data A1 of the first detection target A and radar data B1 of the first detection target B. The first position coordinates of the first detection targets A and B are calculated based on the radar data. For example, the radar data A1 and B1 of the first detection targets A and B are respectively mapped to the reference coordinate system, thereby obtaining a pixel coordinate point set of the first detection target A and a pixel coordinate point set of the first detection target B. Based on the pixel coordinate point set of the first detection target A and the pixel coordinate point set of the first detection target B, the first position coordinates M1 of the first detection target A and the first position coordinates M2 of the first detection target B are respectively obtained.

[0039] Furthermore, the area to be detected can be extracted from the image to be detected based on the first position coordinates. For example, the first area to be detected is extracted from the image to be detected using the first position coordinates M1; and the second area to be detected is extracted from the image to be detected using the first position coordinates M2. Target detection is performed on each area to be detected to obtain the second position coordinates of each second detection target in the area to be detected. For example, the first area to be detected detects two second detection targets, namely, the second detection target a and the second detection target b, wherein the second position coordinates of the second detection target a are N1, and the second position coordinates of the second detection target b are N2; the second area to be detected detects two second detection targets, namely, the second detection target b and the second detection target c, wherein the second position coordinates of the second detection target b are N2, and the second position coordinates of the second detection target c are N3.

[0040] In one embodiment, the first position coordinates are the position coordinates of the center point of the first detection target, and the second position coordinates are the position coordinates of the center point of the second detection target. In a feasible embodiment, the first position coordinates are the position coordinates of the upper left point of the first detection target, and the second position coordinates are the position coordinates of the upper left point of the second detection target.

[0041] Specifically, the radar can measure the depth information of the first detection target, which is the distance information from the first detection target to the measurement sensor. Therefore, the distance information of the first detection target, that is, the depth information, can be directly obtained based on the radar data corresponding to the first detection target.

[0042] In one embodiment, the binocular camera parallax algorithm is used to calculate the depth information D2 of the detection target based on the camera data of the second detection target set. Specifically, the binocular camera can be regarded as two pinhole camera models. According to the parallax principle, the same target has different imaging positions in the left and right eye cameras. According to the parallax, the distance of the second detection target, that is, the depth information D2, can be calculated.

[0043] Step S12: Matching the first detection target with the second detection target using the first position coordinates, the second position coordinates, the distance information, and the depth information.

[0044] Specifically, the distance between each first detection target and each second detection target can be calculated based on the first position coordinate and the second position coordinate. For example, the distance between the first position coordinate M1 of the first detection target A and the second position coordinate N1 of the second detection target a is calculated, and the distance between the first position coordinate M1 of the first detection target A and the second position coordinate N2 of the second detection target b is calculated. For another example, the distance between the first position coordinate M2 of the first detection target B and the second position coordinate N2 of the second detection target b can be calculated, and the distance between the first position coordinate M2 of the first detection target B and the second position coordinate N3 of the second detection target c can be calculated. The specific calculation method is as follows:

[0045]

[0046] Among them, (u ci , v ci ) is the first position coordinate, (u ri , v ri ) is the second position coordinate.

[0047] The depth difference between the first detection target and the second detection target is calculated based on the distance information D1 and the depth information D2. The specific calculation method is as follows:

[0048]

[0049] The first detection target is matched with the second detection target by using the distance and the depth difference. Specifically, each first detection target in the first detection target set is traversed and matched with each second detection target in the second detection target set by using the distance and the depth difference. In a specific embodiment, the distance and the depth difference are weighted and summed, and each first detection target in the first detection target set is traversed and matched with each second detection target in the second detection target set based on the result of the weighted summation. The weighted summation method is as follows:

[0050]

[0051] Among them, α and β are weight coefficients.

[0052] In a specific embodiment, the first detection target and the second detection target corresponding to the minimum weighted sum result are determined to be the same detection target. For example, if the weighted sum of the distance and depth difference between the first detection target A and the second detection target a is the smallest, the second detection target a and the first detection target A are the same detection target; if the weighted sum of the distance and depth difference between the second detection target b and the first detection target B is the smallest, the second detection target b and the first detection target B are determined to be the same detection target.

[0053] The method of this embodiment introduces a fusion method of distance calculation and depth calculation when matching the first detection target detected by the radar with the second detection target detected by the camera, and traverses and matches the first detection target with the second detection target by the weighted summation method. In this way, both the radar data corresponding to the detection target and the camera data of the detection target can be obtained, and the detection result is more accurate. In the matching process, the depth information in the radar data of the first detection target and the depth information in the camera data of the second detection target are fused, which reduces the dependence on the coordinate system mapping function to a certain extent, making the detection accuracy higher.

[0054] See also Figure 2 , is a flow chart of a second embodiment of the target detection method of the present invention. In this embodiment, steps S21 and S22 are the same as those described above. Figure 1 In the first embodiment shown, step S11 and step S12 are the same, except that, after step S22, this embodiment further includes:

[0055] Step S23: Fusing the matched radar data of the first detection target and the camera data of the second detection target.

[0056] Specifically, the matched first detection target and the second detection target are determined to be the same detection target. If the judgment result is that the second detection target a matches the first detection target A, and the second detection target b matches the first detection target B, it means that the second detection target a and the first detection target A are the same detection target, and the second detection target b and the first detection target B are the same detection target. At this time, the radar data of the first detection target and the camera data of the second detection target are fused. Since the first detection target and the second detection target are unified detection targets, and the camera data is image data, it can obtain the appearance information of the detection target, and the radar data can represent the spatial information and motion information of the detection target. By fusing the radar data with the camera data, the appearance information, spatial information, motion information, etc. of the detection target can be obtained.

[0057] The method of this embodiment introduces a fusion method of distance calculation and depth calculation when matching the first detection target detected by the radar with the second detection target detected by the camera, and traverses and matches the first detection target with the second detection target by the weighted summation method. In this way, both the radar data corresponding to the detection target and the camera data of the detection target can be obtained, and the detection result is more accurate. In the matching process, the depth information in the radar data of the first detection target and the depth information in the camera data of the second detection target are fused, which reduces the dependence on the coordinate system mapping function to a certain extent, making the detection accuracy higher.

[0058] See also Figure 3, is a flow chart of a second embodiment of the target detection method of the present invention. In this embodiment, steps S32 and S33 are the same as those described above. Figure 1 In the first embodiment shown, step S11 and step S12 are the same, except that, before step S32, this embodiment further includes:

[0059] Step S31: Acquire radar data and camera data using a time synchronization method.

[0060] Specifically, data delays in the process of collecting radar data and camera data have a significant impact on the detection results. Therefore, it is necessary to reduce the number of intermediate nodes from image sensors, such as shooting devices, radar sensors to computer processors as much as possible to improve data transmission efficiency, reduce time delays, and ensure that radar data and camera data can be obtained simultaneously using time synchronization methods.

[0061] In one embodiment, time synchronization can be divided into software time synchronization and hardware time synchronization. The software time synchronization method has a large error, but it is low cost and flexible and configurable. The hardware time synchronization method has a small error, but it is high cost, requires additional circuit design, and is not easy to modify. Both methods have their own advantages and disadvantages, but both can improve the transmission efficiency of radar data and camera data, and can further improve the accuracy of detection results. Therefore, the software time synchronization method can be used to obtain radar data and camera data; the hardware time synchronization method can also be used to obtain radar data and camera data.

[0062] In a preferred embodiment, in response to the detection target's movement speed being less than a preset speed, the radar data and camera data are acquired using a software time synchronization method; in response to the detection target's movement speed being greater than a preset speed, the radar data and camera data are acquired using a hardware time synchronization method. The preset speed can be customized, that is, in this application, when the detection target is moving at a low speed, the radar data and camera data are acquired using a software time synchronization method, and when the detection target is moving at a high speed, the radar data and camera data are acquired using a hardware time synchronization method. That is, for low-speed target tracking scenarios, it is recommended to use a software time synchronization method; for high-speed target tracking scenarios, it is recommended to use a hardware time synchronization method. After the time synchronization process is executed, the information in the radar data and the camera data is almost the target information at the same time (the time error between the two is minimal), which can ensure the accuracy of the detection results.

[0063] The method of this embodiment introduces a fusion method of distance calculation and depth calculation when matching the first detection target detected by the radar with the second detection target detected by the camera, and traverses and matches the first detection target with the second detection target by the weighted summation method. In this way, both the radar data corresponding to the detection target and the camera data of the detection target can be obtained, and the detection result is more accurate. In the matching process, the depth information in the radar data of the first detection target and the depth information in the camera data of the second detection target are fused, which reduces the dependence on the coordinate system mapping function to a certain extent, making the detection accuracy higher. In addition, this embodiment uses a time synchronization method to obtain the radar data and camera data of the detection target at the same time, which can ensure the accuracy of the detection result.

[0064] See also Figure 4 , is a schematic diagram of the structure of an embodiment of a target detection device of the present invention. Specifically, the target detection device includes: an acquisition module 41 and a matching module 42.

[0065] Among them, the acquisition module 41 is used to obtain the first position coordinates and distance information of each first detection target in the first detection target set based on the radar data of the first detection target set, and to obtain the second position coordinates and depth information of each second detection target in the second detection target set based on the camera data of the second detection target set.

[0066] In one embodiment, the acquisition module 41 is used to map the radar data to a reference pixel coordinate system, thereby obtaining a pixel coordinate point set, and the radar data is radar point cloud data; based on the pixel coordinate point set, the first position coordinates of each first detection target are obtained. Specifically, the acquisition module 41 is also used to perform Kalman filtering on the radar data, map the filtered radar data to a reference pixel coordinate system, and thereby obtain a pixel coordinate point set.

[0067] In one embodiment, the acquisition module 41 is used to extract the area to be detected from the image to be detected based on the first position coordinates, and the image to be detected is camera data; perform target detection on the area to be detected to obtain the second position coordinates of each second detection target in the area to be detected.

[0068] In one embodiment, the first position coordinates are the center point coordinates of the first detection target, and the second position coordinates are the center point coordinates of the second detection target.

[0069] In one embodiment, the acquisition module 41 calculates the depth information of each second detection target based on the camera data of the second detection target set using a binocular camera parallax algorithm.

[0070] The matching module 42 is used to match the first detection target with the second detection target by using the first position coordinates, the second position coordinates, the distance information and the depth information.

[0071] In one embodiment, the matching module 42 calculates the distance between the first detection target and the second detection target based on the first position coordinates and the second position coordinates; and calculates the depth difference between the first detection target and the second detection target based on the distance information and the depth information; the first detection target is matched with the second detection target using the distance and the depth difference. Specifically, the matching module 42 uses the distance and the depth difference to traverse and match each first detection target in the first detection target set with each second detection target in the second detection target set. Specifically, the matching module 42 performs a weighted summation on the distance and the depth difference, and traverses and matches each first detection target in the first detection target set with each second detection target in the second detection target set based on the result of the weighted summation. In a specific embodiment, the matching module 42 determines the first detection target and the second detection target corresponding to the result with the smallest weighted summation as the same detection target.

[0072] In one embodiment of the present invention, the acquisition module 41 is also used to acquire radar data and camera data using a time synchronization method. For example, the acquisition module 41 can acquire radar data and camera data using a software time synchronization method; or acquire radar data and camera data using a hardware time synchronization method. In a preferred embodiment, if the movement speed of the detection target is less than a preset speed, the acquisition module 41 acquires radar data and camera data using a software time synchronization method. If the movement speed of the detection target is greater than the preset speed, the acquisition module 41 acquires radar data and camera data using a hardware time synchronization method.

[0073] The device of this embodiment introduces a fusion method of distance calculation and depth calculation when matching the first detection target detected by the radar with the second detection target detected by the camera, and traverses and matches the first detection target with the second detection target by the weighted summation method. In this way, both the radar data corresponding to the detection target and the camera data of the detection target can be obtained, and the detection result is more accurate. In the matching process, the depth information in the radar data of the first detection target and the depth information in the camera data of the second detection target are fused, which reduces the dependence on the coordinate system mapping function to a certain extent, making the detection accuracy higher. In addition, this embodiment uses a time synchronization method to obtain the radar data and camera data of the detection target at the same time, which can ensure the accuracy of the detection result.

[0074] See also Figure 5 , is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, the electronic device includes a memory 202 and a processor 201 connected to each other.

[0075] The memory 202 is used to store program instructions for implementing any of the above-mentioned device methods.

[0076] The processor 201 is used to execute program instructions stored in the memory 202 .

[0077] The processor 201 may also be referred to as a CPU (Central Processing Unit). The processor 201 may be an integrated circuit chip having signal processing capabilities. The processor 201 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0078] The memory 202 can be a memory stick, a TF card, etc., which can store all the information in the electronic device of the device, including the input raw data, computer programs, intermediate operation results and final operation results are all stored in the memory. It stores and retrieves information according to the location specified by the controller. With the memory, the electronic device has a memory function to ensure normal operation. The memory of the electronic device can be divided into main memory (internal memory) and auxiliary memory (external memory) according to its use, and there is also a classification method of dividing it into external memory and internal memory. External memory is usually a magnetic medium or an optical disk, etc., which can store information for a long time. Memory refers to the storage component on the motherboard, which is used to store the data and programs currently being executed, but is only used to temporarily store programs and data. If the power is turned off or the power is cut off, the data will be lost.

[0079] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0080] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0081] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0082] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a system server, or a network device, etc.) or a processor to execute all or part of the steps of the various implementation methods of the present application.

[0083] See also Figure 6 , which is a schematic diagram of the structure of the computer-readable storage medium of the present invention. The storage medium of the present application stores a program file 203 that can implement all the above methods, wherein the program file 203 can be stored in the above storage medium in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of each implementation method of the present application. The aforementioned storage device includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, etc., which can store program codes, or terminal devices such as computers, servers, mobile phones, and tablets.

[0084] The above are only implementation modes of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A target detection method, characterized in that: include: Mapping the radar data of the first detection target set to a reference pixel coordinate system to obtain a pixel coordinate point set, obtaining first position coordinates and distance information of each first detection target in the first detection target set in the image to be detected based on the pixel coordinate point set, and obtaining second position coordinates and depth information of each second detection target in the second detection target set in the image to be detected based on the camera data of the second detection target set; Matching the first detection target with the second detection target using the first position coordinates, the second position coordinates, the distance information, and the depth information; The step of matching the first detection target with the second detection target using the first position coordinates, the second position coordinates, the distance information and the depth information includes: calculating the distance information between the first detection target and the second detection target based on the first position coordinates and the second position coordinates; and calculating the depth difference between the first detection target and the second detection target based on the distance information and the depth information; performing weighted summation on the distance information and the depth difference, and traversally matching each of the first detection targets in the first detection target set with each of the second detection targets in the second detection target set based on the result of the weighted summation.

2. The method according to claim 1, characterized in that Obtaining depth information of each second detection target in the second detection target set based on camera data of the second detection target set includes: Depth information of each of the second detection targets is obtained by calculating based on the camera data of the second detection target set using a binocular camera parallax algorithm.

3. The target detection method according to claim 1, characterized in that: The step of matching the first detection target with the second detection target by using the distance and the depth difference comprises: Each first detection target in the first detection target set is traversally matched with each second detection target in the second detection target set by using the distance and the depth difference.

4. The method according to claim 1, characterized in that: The step of performing weighted summation on the distance and the depth difference, and traversing and matching each first detection target in the first detection target set with each second detection target in the second detection target set based on the result of the weighted summation, comprises: The first detection target and the second detection target corresponding to the result with the smallest weighted sum are determined to be the same detection target.

5. The method according to any one of claims 1 to 4, characterized in that: Obtaining first position coordinates of each first detection target in the first detection target set based on radar data of the first detection target set includes: Mapping the radar data to a reference pixel coordinate system to obtain a pixel coordinate point set, wherein the radar data is radar point cloud data; The first position coordinates of each of the first detection targets are obtained based on the pixel coordinate point set.

6. The method according to claim 5, characterized in that Obtaining second position coordinates of each second detection target in the second detection target set based on camera data of the second detection target set includes: Extracting a to-be-detected area from an to-be-detected image based on the first position coordinates, where the to-be-detected image is the camera data; Perform target detection on the area to be detected to obtain the second position coordinates of each second detection target in the area to be detected.

7. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The matched radar data of the first detection target and the camera data of the second detection target are fused, wherein the matched first detection target and the second detection target are determined to be the same detection target.

8. The method according to any one of claims 1 to 4, characterized in that: Before the step of obtaining the first position coordinates and distance information of each first detection target in the first detection target set based on the radar data of the first detection target set, and obtaining the second position coordinates and depth information of each second detection target in the second detection target set based on the camera data of the second detection target set, the method further includes: The radar data and the camera data are acquired using a time synchronization method.

9. The method according to claim 8, characterized in that The step of acquiring the radar data and the camera data by using a time synchronization method comprises: The radar data and the camera data are acquired by using a software time synchronization method; or the radar data and the camera data are acquired by using a hardware time synchronization method.

10. The method according to claim 8, characterized in that The step of acquiring the radar data and the camera data by using a time synchronization method comprises: In response to the movement speed of the detected target being less than a preset speed, acquiring the radar data and the camera data by using a software time synchronization method; In response to a movement speed of the detection target being greater than a preset speed, the radar data and the camera data are acquired using a hardware time synchronization method.

11. The method according to any one of claims 1 to 4, characterized in that: Before the step of obtaining the first position coordinates and distance information of each first detection target in the first detection target set based on the radar data of the first detection target set, and obtaining the second position coordinates and depth information of each second detection target in the second detection target set based on the camera data of the second detection target set, the method further includes: Kalman filtering is performed on the radar data.

12. The method according to any one of claims 1 to 4, characterized in that: The first position coordinates are the center point coordinates of the first detection target, and the second position coordinates are the center point coordinates of the second detection target.

13. A target detection device, characterized in that: include: an acquisition module, configured to map the radar data of the first detection target set to a reference pixel coordinate system, thereby obtaining a pixel coordinate point set, obtaining first position coordinates and distance information of each first detection target in the first detection target set in the image to be detected based on the pixel coordinate point set, and obtaining second position coordinates and depth information of each second detection target in the second detection target set in the image to be detected based on the camera data of the second detection target set; a matching module, matching the first detection target with the second detection target by using the first position coordinates, the second position coordinates, the distance information, and the depth information; The step of matching the first detection target with the second detection target using the first position coordinates, the second position coordinates, the distance information and the depth information includes: calculating the distance information between the first detection target and the second detection target based on the first position coordinates and the second position coordinates; and calculating the depth difference between the first detection target and the second detection target based on the distance information and the depth information; performing weighted summation on the distance information and the depth difference, and traversally matching each of the first detection targets in the first detection target set with each of the second detection targets in the second detection target set based on the result of the weighted summation.

14. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the target detection method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: A program file is stored, and the program file can be executed to implement the target detection method according to any one of claims 1 to 12.