Target detection method, device, system, equipment and program product

Abstract

The invention provides a target detection method, device, system, equipment and program product, and the method comprises the steps: obtaining a first target detection result and at least one group of second target detection results of a target region in a current frame image, the first target detection result is determined based on the current frame image, and the current frame image is collected by a visual sensor; the second target detection result is determined based on radar data collected by a radar sensor, and the target area is determined based on first detection error distribution information of a visual sensor and second detection error distribution information of the radar sensor; and carrying out fusion processing on the first target detection result and the second target detection result of the target area to obtain a target detection result of the target area. The target detection scheme has higher target detection precision, so that the accuracy and robustness of environmental perception can be effectively ensured.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a target detection method, device, system, equipment and program product. Background Art

[0002] In fields such as autonomous driving, environmental perception is an important link. In order to avoid the impact of the limitations of a single sensor on environmental perception, the accuracy and robustness of environmental perception are usually improved by fusing the perception results of multiple sensors such as visual sensors and radar sensors.

[0003] At present, in the process of fusing the perception results of visual sensors and radar sensors, there is a problem of poor fusion accuracy, which makes it impossible to effectively improve the accuracy and robustness of environmental perception. Summary of the invention

[0004] Based on the above needs, the present application proposes a target detection method, device, system, equipment and program product, which can effectively improve the fusion accuracy in the process of fusing the perception results of visual sensors and radar sensors, thereby improving the detection accuracy of the target.

[0005] In order to achieve the above technical objectives, this application specifically proposes the following technical solutions:

[0006] The first aspect of the present application provides a target detection method, comprising:

[0007] Obtaining a first target detection result and at least one set of second target detection results of a target area in a current frame image; the first target detection result is determined based on the current frame image, and the current frame image is collected by a visual sensor; the second target detection result is determined based on radar data collected by a radar sensor; the target area is determined based on first detection error distribution information of the visual sensor and second detection error distribution information of the radar sensor;

[0008] The first target detection result and the second target detection result are fused to obtain a target detection result of the target area.

[0009] A second aspect of the present application provides a target detection device, comprising:

[0010] A data acquisition module, used to acquire a first target detection result and at least one set of second target detection results of a target area in a current frame image; the first target detection result is determined based on the current frame image, and the current frame image is collected by a visual sensor; the second target detection result is determined based on radar data collected by a radar sensor; the target area is determined based on first detection error distribution information of the visual sensor and second detection error distribution information of the radar sensor;

[0011] The data fusion module is used to fuse the first target detection result and the second target detection result to obtain the target detection result of the target area.

[0012] A third aspect of the present application proposes a target detection system, comprising: a visual sensor, a radar sensor and a processor, wherein the processor is used to execute the target detection method as described above.

[0013] In a fourth aspect, the present application provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the target detection method as described above is implemented.

[0014] A fifth aspect of the present application provides an electronic device, comprising:

[0015] Memory and processor;

[0016] The memory is connected to the processor and is used to store programs;

[0017] The processor is used to implement the above-mentioned target detection method by running the program in the memory.

[0018] In a sixth aspect, the present application proposes a computer program product, comprising computer program instructions, which, when executed by a processor, enable the processor to execute the target detection method as described above.

[0019] The target detection method proposed in the present application obtains a first target detection result and at least one group of second target detection results of the target area in the current frame image, the first target detection result is determined based on the current frame image, the current frame image is collected by a visual sensor, the second target detection result is determined based on radar data collected by a radar sensor, and the target area is determined based on first detection error distribution information of the visual sensor and second detection error distribution information of the radar sensor, so that according to the detection errors of the visual sensor and the radar sensor, the area where the target detection results of the radar sensor need to be fused can be accurately identified, and then by fusing the first target detection result and the second target detection result of the target area, the target detection accuracy of the target area can be effectively improved, thereby ensuring the accuracy and robustness of environmental perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of the structure of a target detection system provided in an embodiment of the present application.

[0022] Figure 2 A flowchart of a target detection method provided in an embodiment of the present application.

[0023] Figure 3 A schematic diagram of a first detection error distribution of a visual sensor provided in an embodiment of the present application.

[0024] Figure 4 A schematic diagram of a first candidate area determined based on the half-power beam width of a radar sensor provided in an embodiment of the present application.

[0025] Figure 5 A schematic diagram of the structure of a target detection device provided in an embodiment of the present application.

[0026] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solutions of the embodiments of the present application are applicable to application scenarios such as automatic vehicle driving and automatic robot control. By adopting the technical solutions of the embodiments of the present application, targets in the driving environment of the vehicle or robot can be accurately detected, thereby achieving accurate perception of the driving environment of the vehicle or robot.

[0028] At present, target detection is usually achieved through sensor fusion solutions. Sensor fusion solutions usually include early fusion and post-fusion. Early fusion directly fuses information at the level of raw data collected by different sensors, while post-fusion fuses at the feature or decision level. In the post-fusion process, traditional fusion solutions usually use Kalman filtering to fuse target detection results from different sensors. Tracking and matching are required during the fusion process, and a rule-based matching cost (matching cost) needs to be set. The selection of matching cost is difficult to cover all factors affecting fusion accuracy, so it is impossible to guarantee the fusion accuracy of target detection results from different sensors, and thus it is impossible to guarantee the detection accuracy of targets in the driving environment of vehicles or robots, and it is impossible to achieve accurate perception of the driving environment by vehicles or robots.

[0029] Based on the above technical status, the inventors of the present application have proposed a new target detection scheme after research. The target detection scheme can accurately fuse the target detection results of different sensors and effectively improve the detection accuracy of targets in a driving environment.

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] Figure 1 An exemplary implementation environment of the technical solution proposed in the embodiment of the present application is shown, and the implementation environment is a target detection system. Figure 1 As shown, the system at least includes a visual sensor 101, a radar sensor 102 and a processor 103. The target detection system can be set on any mobile object, which can be a vehicle or a robot, etc., so as to provide the mobile object with a target detection function to assist the mobile object to automatically perceive the driving environment, thereby assisting the mobile object to achieve intelligent driving or intelligent control.

[0032] Among them, both the visual sensor 101 and the radar sensor 102 can be used to perceive the driving environment of the mobile object and collect environmental perception data. For the visual sensor 101, it can include one or more visual sensors 101 arranged around the mobile object and / or at any position of the mobile object, for example, it can include but not limited to optical cameras, infrared cameras, etc. For the radar sensor 102, it can include one or more radar sensors 102 arranged around the mobile object and / or at any position of the mobile object, for example, it can include but not limited to millimeter wave radar, laser radar, etc. Based on the visual sensor 101 and the radar sensor 102, the driving environment of the mobile object during driving can be detected and data recorded, for example, the road perception data, surrounding vehicle perception data, obstacle perception data, etc. in the driving environment of the mobile object can be collected, and the distance, speed, color, texture and other information of the target in the driving environment can be provided.

[0033] Among them, the visual sensor 101 may also include a first perception module, and the radar sensor 102 may also include a second perception module. The first perception module may be used to perform target detection on the data collected by the visual sensor 101 to obtain a first target detection result in the driving environment, and the second perception module may be used to perform target detection on the data collected by the radar sensor 102 to obtain a second target detection result in the driving environment. The first perception module may be set inside the visual sensor 101, or may be set separately from the visual sensor 101. At the same time, the second perception module may be set inside the radar sensor 102, or may be set separately from the radar sensor 102. The specific setting may be based on actual needs.

[0034] The first perception module and the second perception module can send the first target detection result and the second target detection result to the processor 103 respectively, and the processor 103 realizes the fusion processing of the first target detection result and the second target detection result by processing and calculating the first target detection result and the second target detection result to obtain the target detection result in the driving environment of the mobile object.

[0035] One or more services or applications capable of executing target detection processing are running on the processor 103. As an optional implementation, the processor 103 may be a device with data processing function, such as a computer, a processing chip, or a server.

[0036] The visual sensor 101 and the radar sensor 102 can be connected to the processor 103 through a wired or wireless communication link. When the two are connected through a wired communication link, they can be connected through any type of signal line such as a coaxial signal line, a digital signal line, an optical cable signal line, a balanced signal line, etc. When the two are connected through a wireless communication link, they can be connected to each other through a network. The network can be any type of network. For example, the network can be a network that can be subdivided into multiple subnetworks. The network or the multiple subnetworks contained in the network can be a local area network (LAN), Ethernet, token ring, wide area network (WAN), Internet, virtual network, virtual private network (VPN), intranet, extranet, public switched telephone network (PSTN), infrared network, wireless network (such as Bluetooth, WIFI), mobile network (such as 4G, 5G), Internet of Things and / or any combination of these and / or other networks.

[0037] Furthermore, the above-mentioned target detection system may also include a memory for storing data collected by the visual sensor 101 and the radar sensor 102, or storing the first target detection result and the second target detection result, or storing the target detection result processed by the processor 103, or storing the intermediate processing results of the processor 103, etc.

[0038] Combine the following Figure 1 The application scenario shown provides an exemplary introduction to the target detection solution proposed in the embodiment of the present application.

[0039] See also Figure 2 The embodiment of the present application first proposes a target detection method, which can be executed by a processor disposed on a target vehicle or a target robot. The method includes:

[0040] S201. Obtain a first target detection result and at least one set of second target detection results of a target area in a current frame image; the first target detection result is determined based on the current frame image, and the current frame image is collected by a visual sensor; the second target detection result is determined based on radar data collected by a radar sensor; the target area is determined based on first detection error distribution information of the visual sensor and second detection error distribution information of the radar sensor.

[0041] Specifically, the current frame image mentioned above refers to the environmental perception data collected by the visual sensor at the current moment. The target area refers to the area where the first target detection result corresponding to the visual sensor and the second target detection result corresponding to the radar sensor are fused.

[0042] The first target detection result of the target area refers to the target detection result of the target area determined based on the current frame image. In implementation, image recognition can be performed on the current frame image to obtain the target detection result corresponding to the current frame image, and the target detection result of the target area in the target detection result corresponding to the current frame image is used as the first target detection result. Among them, the number of the first target detection results of the target area can be one or more, for example, it can include the detection results of one or more first targets to be measured at the corresponding data collection time in the target area, for any first target to be measured, the detection result of the first target to be measured can include the parameter values ​​of each first parameter of the first target to be measured, and the first parameter can include one or more of the size information (such as length, width, height), center position (such as the physical coordinates of the center position), heading angle and speed (such as speed in the X direction, speed in the Y direction).

[0043] The second target detection results of the target area may include one or more groups, and the multiple groups of second target detection results may correspond one-to-one to multiple radar sensors, or one-to-one to multiple data collection moments of one radar sensor, or one-to-one to multiple data collection moments of multiple radar sensors. That is, different second target detection results may be determined by radar data collected by different radar sensors, or by radar data collected by the same radar sensor at different data collection moments, or by radar data collected by multiple radar sensors at different data collection moments. Among them, the multiple data collection moments of the radar sensor may be determined based on the data collection moment of the current frame image, for example, may be multiple data collection moments within a time interval including the data collection moment of the current frame image.

[0044] In implementation, for any radar sensor, target detection can be performed on the radar data collected by the radar sensor at the corresponding data collection time to obtain the target detection result corresponding to the radar sensor at the data collection time, and the target detection result of the target area in the target detection result corresponding to the radar sensor at the data collection time is used as the second target detection result. Wherein, for any set of second target detection results, the number of second target detection results can be one or more, for example, it can include the detection results of one or more second targets to be measured at the corresponding data collection time in the target area. It can be understood that the second target to be measured can be the same as the first target to be measured, can be partially the same as the first target to be measured, and can also be different from the first target to be measured. For any second target to be measured, the detection result of the second target to be measured can include the parameter values ​​of each second parameter of the second target to be measured, and the second parameter can include one or more of size information (such as length, width, height), center position (such as the physical coordinates of the center position), heading angle and speed (such as speed in the X direction, speed in the Y direction). It can be understood that the second parameter can be the same as the first parameter, and can also be partially the same as the first parameter.

[0045] The first detection error distribution information of the visual sensor may be the corresponding relationship between the detection error and the position of the visual sensor in a predetermined area. The predetermined area may be an area of ​​a predetermined size around the target vehicle or the target robot, for example, an area of ​​a predetermined size centered on the rear axle of the target vehicle. The predetermined size may be a pre-configured size, and may also be updated according to the current driving environment of the target vehicle or the target robot. For example, the first detection error distribution information of the visual sensor may be as follows: Figure 3 As shown by Figure 3 It can be seen that the closer to the center area of ​​the predetermined area, the smaller the detection error of the visual sensor, and the farther away from the center area of ​​the predetermined area, the larger the detection error of the visual sensor.

[0046] The second detection error distribution information of the radar sensor may be a corresponding relationship between the detection error of the radar sensor and the position in the predetermined area. It is understandable that the predetermined area needs to satisfy both the data collection conditions of the visual sensor and the data collection conditions of the radar sensor.

[0047] The first detection error distribution information of the visual sensor and the second detection error distribution information of the radar sensor may be determined by target detection results of the visual sensor and the radar sensor under different conditions. The different conditions may include a plurality of different environmental conditions and / or weather conditions.

[0048] In implementation, the target area in the predetermined area can be determined based on the first detection error distribution information of the visual sensor and the second detection error distribution information of the radar sensor, and the first target detection result and the second target detection result of the target area can be fused. The number of target areas can be one or more. For example, the target area can be determined based on the comparison result of the detection error of the visual sensor and the detection error of the radar sensor, so that the target area can be determined quickly and effectively according to the first detection error distribution information of the visual sensor and the second detection error distribution information of the radar sensor.

[0049] S202: Fusing the first target detection result and the second target detection result to obtain a target detection result of the target area.

[0050] Specifically, the above-mentioned method of fusing the first target detection result and the second target detection result can be set according to actual needs. For example, for any first target detection result, a second target detection result that matches the first target detection result can be determined from the second target detection results, and the first target detection result and the second target detection result that matches the first target detection result can be fused to obtain a fused target detection result corresponding to the first target detection result.

[0051] Optionally, in the process of fusing the first target detection result and the second target detection result that matches the first target detection result, the first target detection result and the second target detection result that matches the first target detection result can be weightedly summed based on the target weight coefficient to obtain a fused target detection result corresponding to the first target detection result, and the median of the first target detection result and the second target detection result that matches the first target detection result can also be used as the fused target detection result corresponding to the first target detection result.

[0052] Among them, the target weight coefficient can be a pre-configured fixed value, and the target weight coefficient can also be adaptively updated according to the first detection error distribution information of the visual sensor and the second detection error distribution information of the radar sensor. For example, the size of the target weight coefficient can be determined based on the first detection error of the visual sensor and the second detection error of the radar sensor in the target area. It can be understood that the larger the detection error, the smaller the weight of the target detection result corresponding to the sensor in the weighted summation process, so that the first target detection result of the target area and the second target detection result matching the first target detection result can be accurately fused.

[0053] Thus, the fused target detection result corresponding to each first target detection result in the target area can be obtained. Among them, each first target detection result in the target area can be replaced with its corresponding fused target detection result to obtain the target detection result of the target area, thereby effectively improving the target detection accuracy of the target area.

[0054] It can be understood that, in the current frame image, the target detection results of each area outside the target area may be the first target detection results of the corresponding area.

[0055] From the above introduction, it can be seen that the target detection method proposed in the embodiment of the present application obtains a first target detection result and at least one group of second target detection results of the target area in the current frame image, the first target detection result is determined based on the current frame image, the current frame image is collected by a visual sensor, the second target detection result is determined based on radar data collected by the radar sensor, and the target area is determined based on the first detection error distribution information of the visual sensor and the second detection error distribution information of the radar sensor, so that according to the detection errors of the visual sensor and the radar sensor, the area where the target detection results of the radar sensor need to be fused can be accurately identified, and then by fusing the first target detection result and the second target detection result of the target area, the target detection accuracy of the target area can be effectively improved, thereby ensuring the accuracy and robustness of environmental perception.

[0056] In some embodiments, the first detection error distribution information includes detection errors of visual sensors in each sub-region within the predetermined region; the second detection error includes detection errors of radar sensors in each sub-region;

[0057] Determining a target area based on first detection error distribution information of a visual sensor and second detection error distribution information of a radar sensor includes:

[0058] Based on the magnitude relationship between the detection error of the visual sensor and the detection error of the radar sensor in the sub-region, it is determined whether to use the sub-region as a target region.

[0059] Specifically, in the process of determining the first detection error distribution information of the visual sensor and the second detection error distribution information of the radar sensor, the predetermined area can be divided into a plurality of sub-areas. For example, the predetermined area can be divided into a plurality of sub-areas by means of grid division, and the detection error of the visual sensor and the detection error of the radar sensor in each sub-area can be determined respectively. It is understandable that the sizes of different sub-areas can be the same or different. The finer the grid division, the higher the accuracy of the first detection error distribution information and the second detection error distribution information, and the more accurate the target area determined according to the first detection error distribution information and the second detection error distribution information, thereby effectively improving the target detection accuracy of the target area.

[0060] For any sub-area in the predetermined area, in the process of determining the detection error of the visual sensor in the sub-area, the target detection results of the visual sensor under multiple different predetermined conditions (such as environmental conditions and / or weather conditions) can be obtained respectively, and the detection error of the visual sensor in the sub-area can be determined according to the target detection results of the visual sensor in the sub-area and the actual values ​​of the first parameters of the target to be detected in the sub-area. For example, the detection errors corresponding to the target detection results of the visual sensor in the sub-area can be averaged to obtain the detection error of the visual sensor in the sub-area.

[0061] In addition, in the process of determining the detection error of the radar sensor in the sub-area, the target detection results of the radar sensor under multiple different predetermined conditions (such as environmental conditions and / or weather conditions) can be obtained respectively, and the detection error of the radar sensor in the sub-area can be determined according to the target detection results of the radar sensor in the sub-area and the actual values ​​of the second parameters of the target to be detected in the sub-area. For example, the detection errors corresponding to the target detection results of the radar sensor in the sub-area can be averaged to obtain the detection error of the radar sensor in the sub-area.

[0062] In implementation, the magnitude relationship between the detection error of the visual sensor and the detection error of the radar sensor in each sub-region in the predetermined region can be obtained based on the first detection error distribution information of the visual sensor and the second detection error distribution information of the radar sensor. For any sub-region, whether to use the sub-region as a target region can be determined based on the magnitude relationship between the detection error of the visual sensor and the detection error of the radar sensor in the sub-region.

[0063] Optionally, determining whether to use the sub-region as a target region according to a magnitude relationship between a detection error of a visual sensor and a detection error of a radar sensor in the sub-region includes:

[0064] If the detection error of the visual sensor in the sub-area is greater than the detection error of the radar sensor, the sub-area is regarded as the target area;

[0065] If the detection error of the visual sensor in the sub-region is less than or equal to the detection error of the radar sensor, the sub-region is regarded as a non-target region.

[0066] Therefore, the first detection error distribution information of the visual sensor and the second detection error distribution information of the radar sensor can be fully considered to determine the target area, and then the target detection results of the visual sensor and the radar sensor can be fused according to the performance of the visual sensor and the radar sensor in different areas, thereby greatly improving the fusion accuracy of the target detection results.

[0067] In some embodiments, the predetermined area is determined based on physical radiation characteristics of the radar sensor.

[0068] Specifically, the radar sensor can detect the target by emitting electromagnetic waves and receiving reflected signals. The physical radiation characteristics of the radar sensor can be used to characterize the distribution information of the intensity of the electromagnetic waves emitted by the radar sensor within the radiation range of the radar sensor. The physical radiation characteristics used to determine the predetermined area can be selected based on the degree of influence of different physical radiation characteristics of the radar sensor on the ranging and speed measurement accuracy of the radar sensor. For example, the physical radiation characteristics of the radar sensor can include the power of the radar wave emission.

[0069] Optionally, the predetermined area may be determined based on a half power beam width (HPBW) of the radar sensor. The half power beam width of the radar sensor may be measured under different predetermined conditions, or may be determined by parameters of the radar sensor itself.

[0070] In addition, in the process of determining the predetermined area based on the physical radiation characteristics of the radar sensor, the first candidate area can be determined based on the physical radiation characteristics of the radar sensor, and the overlapping area of ​​the first candidate area and the second candidate area is used as the predetermined area. Among them, the second candidate area can be determined according to the target detection requirements, for example, it can be an area of ​​a predetermined size centered on the rear axle of the target vehicle. The predetermined size can be a fixed value, and can also be updated according to the driving scene of the target vehicle or target robot. The driving scene of the target vehicle or target robot can include, for example, high-speed driving, turning driving, etc.

[0071] It is understandable that the radar sensor has different physical radiation characteristics in different working modes (e.g., long-range mode, short-range mode), and thus the first candidate area determined based on the physical radiation characteristics of the radar sensor is also different. In implementation, the first candidate area corresponding to the radar sensor in different working modes can be pre-configured. For example, the first candidate area determined based on the half-power beam width of the radar sensor can be as follows: Figure 4 shown.

[0072] During implementation, the predetermined areas corresponding to different operating modes of the radar sensor under different driving scenarios can be pre-configured. In the process of environmental perception through visual sensors and radar sensors, the predetermined areas can be determined based on the driving scenarios and the operating modes of the radar sensors, and environmental perception can be performed based on the first target detection results and the second target detection results under the predetermined areas.

[0073] Therefore, determining the predetermined area based on the physical radiation characteristics of the radar sensor can further improve the fusion accuracy of the first target detection result of the visual sensor and the second target detection result of the radar sensor while meeting the target detection requirements in the environment.

[0074] In some embodiments, the first target detection result and the second target detection result are fused to obtain the target detection result of the target area, including:

[0075] Based on the first feature of each first target detection result and the second feature of each second target detection result, respectively determine the second target detection results that match each first target detection result;

[0076] Fusing each first target detection result and a second target detection result matching the first target detection result to obtain a fusion result corresponding to each first target detection result;

[0077] The fusion results corresponding to the first target detection results are used as the target detection results of the target area.

[0078] Specifically, the first feature of the first target detection result can be used to characterize the predetermined attribute feature of the target to be detected corresponding to the first target detection result, and can also be used to characterize the data detection feature of the visual sensor, and the second feature of the second target detection result can be used to characterize the predetermined attribute feature of the target to be detected corresponding to the second target detection result, and can also be used to characterize the data detection feature of the radar sensor. For example, the predetermined attribute feature may include posture features, time features, etc., and the data detection feature may include detection range, installation location, etc.

[0079] In implementation, for any first target detection result, a second target detection result matching the first target detection result can be determined based on the first feature of the first target detection result and the second feature of each second target detection result. It can be understood that the second target detection result matching the first target detection result can include a second target detection result consistent with the first target detection result, such as the target to be detected corresponding to the first target detection result is consistent with the target to be detected corresponding to the second target detection result, and the time corresponding to the target to be detected is consistent.

[0080] Optionally, the degree of match between the first feature and the second feature may be determined by a machine learning model, and the second target detection result that matches the first target detection result may be determined based on the degree of match. For example, the second target detection result that has the highest degree of match with the first target detection result may be used as the second target detection result that matches the first target detection result. In addition, the similarity between the first feature and the second feature may be calculated. For example, the similarity between the first feature and the second feature may be calculated by methods such as Euclidean distance and cosine similarity, and the second target detection result that matches the first target detection result may be determined based on the similarity. For example, the second target detection result corresponding to the second feature that has the highest similarity with the first feature may be used as the second target detection result that matches the first target detection result.

[0081] Therefore, the method can obtain a second target detection result that matches each first target detection result.

[0082] In implementation, for any first target detection result, the first target detection result and a second target detection result matching the first target detection result may be fused to obtain a fusion result corresponding to the first target detection result.

[0083] Among them, in the process of fusing the first target detection result and the second target detection result matching the first target detection result, the parameter value of the target parameter in the first target detection result and the parameter value of the target parameter in the second target detection result matching the first target detection result can be fused. The target parameter may include all parameters of the same category in the first parameter and the second parameter, and may also include pre-configured parameters. For example, according to the target detection characteristics of the radar sensor, the parameters with higher detection accuracy of the radar sensor may be pre-configured as target parameters. For example, in the second parameter, the physical coordinates of the center position in the X direction and the speed in the X direction may be used as target parameters.

[0084] It is understandable that when there are multiple target parameters, each target parameter can be fused separately to obtain the fusion result corresponding to each target parameter, and the fusion result corresponding to the first target detection result can be determined according to the corresponding fusion result of each target parameter. For example, the parameter value of the target parameter in the first target detection result can be replaced with the fusion result corresponding to the corresponding target parameter, and the parameter value of each first parameter other than the target parameter remains unchanged, thereby effectively improving the fusion accuracy of the first target detection result and the second target detection result.

[0085] In addition, after obtaining the fusion result corresponding to each first target detection result, each first target detection result in the target area can be replaced with its corresponding fusion result to obtain the target detection result of the target area, thereby effectively improving the target detection accuracy of the target area.

[0086] It can be understood that when a second target detection result that matches the first target detection result is not determined, for example, the matching degree or similarity between each second target detection result and the first target detection result is lower than a predetermined threshold, the first target detection result may not be fused and may be eliminated. The specific setting can be made according to actual needs.

[0087] In some embodiments, the first feature includes a first target detection result and a timestamp corresponding to the first target detection result, and the second feature includes a second target detection result and a timestamp corresponding to the second target detection result.

[0088] Specifically, for any first target detection result, the first feature of the first target detection result may include the first target detection result, and for any second target detection result, the second feature of the second target detection result may include the second target detection result, wherein the first feature of the first target detection result may include the parameter values ​​of part or all of the first parameters in the first target detection result, and the second feature of the second target detection result may include the parameter values ​​of part or all of the second parameters in the second target detection result.

[0089] As an optional implementation manner, the first feature and the second feature may both include the physical coordinates of the center position in the X direction and the Y direction and the speed in the X direction and the Y direction.

[0090] Therefore, based on the first target detection result in the first feature and the second target detection result in the second feature, the matching degree or similarity between the detection information of the target to be measured represented by the first target detection result and the detection information of the target to be measured represented by the second target detection result can be effectively determined, and then the second target detection result that matches the first target detection result can be quickly and accurately determined.

[0091] In addition, for any first target detection result, the first feature of the first target detection result may also include a timestamp corresponding to the first target detection result. The timestamp corresponding to the first target detection result may be used to characterize the data collection time of the image corresponding to the first target detection result. For any second target detection result, the second feature of the second target detection result may also include a timestamp corresponding to the second target detection result. The timestamp corresponding to the second target detection result may be used to characterize the data collection time of the radar data corresponding to the second target detection result. Thus, according to the timestamp corresponding to the first target detection result in the first feature and the timestamp corresponding to the second target detection result in the second feature, the difference between the detection time corresponding to the first target detection result and the detection time corresponding to the second target detection result can be effectively determined, and then the second target detection result matching the first target detection result can be accurately determined.

[0092] At present, in the process of fusing the target detection results corresponding to the visual sensor and the target detection results corresponding to the radar sensor, it is usually necessary to synchronize the time of the two, and the synchronization process involves speed extrapolation, so the fusion accuracy of the two cannot be guaranteed. From the above introduction, it can be seen that the target detection method proposed in the embodiment of the present application fuses the target detection results corresponding to the visual sensor and the target detection results corresponding to the radar sensor by combining the target detection results and timestamps. There is no need to synchronize the time of the target detection results corresponding to the visual sensor and the target detection results corresponding to the radar sensor, thereby avoiding the influence of the time synchronization process on the fusion accuracy, and further improving the fusion accuracy of the target detection results corresponding to the visual sensor and the target detection results corresponding to the radar sensor.

[0093] In some embodiments, based on the first feature of each first target detection result and the second feature of each second target detection result, respectively determining a second target detection result that matches each first target detection result includes:

[0094] Inputting multiple feature pairs corresponding to the first target detection result into a pre-trained matching model respectively to obtain matching information corresponding to each feature pair; the feature pair includes a first feature of the first target detection result and a second feature of the second target detection result;

[0095] Based on the matching information corresponding to each feature pair, a second target detection result matching the first target detection result is determined.

[0096] Specifically, for any first target detection result, in the process of determining a second target detection result that matches the first target detection result, multiple feature pairs corresponding to the first target detection result can be determined respectively. For any feature pair, the feature pair may include a superposition result of a first feature of the first target detection result and a second feature of a second target detection result.

[0097] During implementation, multiple feature pairs corresponding to the first target detection result can be respectively input into a pre-trained matching model to output matching information corresponding to each feature pair through the matching model. The matching information can characterize the matching degree between the first target detection result and the second target detection result corresponding to the feature pair.

[0098] Optionally, the matching model may be a machine learning model such as a deep neural network, for example, an MLP (Multilayer Perceptron) model may be used.

[0099] In addition, the matching model can be trained by a sample set, which may include a plurality of first target detection samples corresponding to the visual sensor, a plurality of second target detection samples corresponding to the radar sensor, and calibration values ​​of matching information between each first target detection sample and each second target detection sample. The first target detection sample may include a first feature of a target detection result determined based on an image collected by the visual sensor, and the second target detection sample may include a second feature of a target detection result determined based on radar data collected by the radar sensor.

[0100] In implementation, each first target detection sample can be superimposed with all second target detection samples to obtain multiple feature pairs corresponding to each first target detection sample. For example, taking Nc first target detection samples and Nr second target detection samples as an example, a two-dimensional feature pair set of Nc*Nr can be formed, and the two-dimensional feature pair set is input into the initial matching model to train the initial matching model.

[0101] During the training process, the loss function value of the matching model can be determined based on the prediction results of the matching information of each feature pair in the two-dimensional feature pair set and the calibration value of the corresponding matching information in the sample set, and the matching model can be iteratively trained according to the loss function value to obtain the final trained matching model.

[0102] Therefore, through the matching model, in the process of fusing the first target detection result and the second target detection result, the matching information corresponding to each feature pair can be determined quickly and accurately.

[0103] After obtaining the matching information corresponding to each feature pair, a second target detection result that matches the first target detection result may be determined based on the matching information corresponding to each feature pair.

[0104] In some embodiments, determining a second target detection result that matches the first target detection result based on the matching information corresponding to each feature pair includes:

[0105] Determine a target feature pair in each feature pair based on a comparison result of matching information corresponding to each feature pair;

[0106] A second target detection result corresponding to the second feature in the target feature pair is determined as the second target detection result that matches the first target detection result.

[0107] Specifically, for any first target detection result, in the process of determining the second target detection result that matches the first target detection result based on the matching information of each feature pair corresponding to the first target detection result, the target feature pair can be determined from each feature pair based on the comparison result of the matching information of each feature pair corresponding to the first target detection result. For example, the feature pair with the highest matching degree represented by the matching information can be used as the target feature pair. It is understandable that the feature pair with the second highest or the second highest matching degree can also be used as the target feature pair according to actual needs, and no specific limitation is made here.

[0108] During implementation, the second target detection result corresponding to the second feature in the target feature pair can be used as the second target detection result matching the first target detection result, and the first target detection result and the second target detection result matching the first target detection result can be fused, so that the second target detection result matching each first target detection result can be determined quickly and accurately, thereby improving the fusion efficiency and accuracy of the first target detection result and the second target detection result.

[0109] It is understandable that the second target detection results that match different first target detection results may all be different. Wherein, in the case where the second target detection results that match multiple first target detection results are the same second target detection result, the second target detection result and the first target detection result with the highest matching degree may be fused.

[0110] Optionally, in the process of determining the second target detection result that matches the first target detection result based on the matching information corresponding to each feature pair, it can be achieved through a classification network (such as a softmax network). For example, the classification network can be connected to the output end of the matching model, and the classification network can automatically identify the second target detection result that matches the first target detection result based on the matching information of each feature pair output by the matching network.

[0111] Therefore, after each feature pair is input into the matching model, the second target detection result that matches the first target detection result can be automatically output through the classification network, further improving the fusion efficiency and fusion accuracy of the first target detection result and the second target detection result.

[0112] In some embodiments, a first target detection result and a second target detection result matching the first target detection result are fused to obtain a fusion result corresponding to the first target detection result, including:

[0113] Based on the weight coefficient corresponding to the target area, a weighted sum is performed on the first target detection result and the second target detection result matching the first target detection result to obtain a fusion result corresponding to the first target detection result.

[0114] Specifically, the weight coefficient corresponding to the target area may include a first weight coefficient corresponding to the first target detection result and a second weight coefficient corresponding to the second target detection result, and the sum of the first weight coefficient and the second weight coefficient is 1. The weight coefficient may be a pre-configured fixed value, and may also be determined based on the first detection error distribution information of the visual sensor and the second detection error distribution information of the radar sensor.

[0115] Optionally, the first weight coefficient and the second weight coefficient can be determined based on the difference between the detection error of the visual sensor in the target area and the detection error of the radar sensor in the target area, wherein the difference may be greater than 0, and the larger the difference is, the smaller the first weight coefficient is and the larger the second weight coefficient is; the smaller the difference is, the larger the first weight coefficient is and the smaller the second weight coefficient is. It can be understood that when the difference is less than or equal to 0, the first target detection result and the second target detection result may not be fused.

[0116] It can be understood that when the target area includes multiple sub-areas, the first weight coefficient and the second weight coefficient can be determined based on the difference between the detection error of the visual sensor in the target area and the detection error of the radar sensor in the target area in the sub-area where the first target detection object is located.

[0117] During implementation, the first target detection result and the second target detection result matching the first target detection result can be weighted and summed based on the weight coefficient corresponding to the target area to obtain a fusion result corresponding to the first target detection result, thereby enabling the first target detection result and the second target detection result to be fused quickly and accurately.

[0118] In some embodiments, a method for determining a weight coefficient corresponding to a target area includes:

[0119] determining a first detection error of the vision sensor within the target area based on the first detection error distribution information, and determining a second detection error of the radar sensor within the target area based on the second detection error distribution information;

[0120] Based on a first detection error of the visual sensor and a second detection error of the radar sensor in the target area, a weight coefficient corresponding to the target area is determined.

[0121] Specifically, the first detection error of the visual sensor in the target area can be determined based on the first detection error distribution information. Wherein, in the case where the target area includes only one sub-area, the first detection error of the visual sensor in the target area may include the detection error corresponding to the sub-area in the first detection error distribution information. In the case where the target area includes multiple sub-areas, the first detection error of the visual sensor in the target area may include the detection error corresponding to each sub-area in the target area in the first detection error distribution information, and may also include the average of the detection errors corresponding to the sub-areas in the target area.

[0122] At the same time, the second detection error of the radar sensor in the target area can be determined based on the second detection error distribution information. Wherein, in the case where the target area includes only one sub-area, the second detection error of the radar sensor in the target area may include the detection error corresponding to the sub-area in the second detection error distribution information. In the case where the target area includes multiple sub-areas, the second detection error of the radar sensor in the target area may include the detection error corresponding to each sub-area in the target area in the second detection error distribution information, and may also include the average of the detection errors corresponding to the sub-areas in the target area.

[0123] In implementation, the weight coefficient corresponding to the target area can be determined based on the first detection error of the visual sensor and the second detection error of the radar sensor in the target area. For example, the first fusion parameter α can be determined based on the first detection error, and the second fusion parameter β can be determined based on the second detection error, the first fusion parameter α is negatively correlated with the first detection error, and the second fusion parameter β is negatively correlated with the second detection error, and the first fusion parameter α and the second fusion parameter β are normalized to obtain the first weight coefficient corresponding to the first target detection result and the second weight coefficient corresponding to the second target detection result. For example, the first weight coefficient α N =α / (α+β), the second weight coefficient β N =β / (α+β).

[0124] It can be understood that, in the case where the target region includes first detection errors and second detection errors corresponding to multiple sub-regions, the first weight coefficient and the second weight coefficient corresponding to each sub-region in the target region can be determined respectively.

[0125] In the process of fusing the first target detection result and the second target detection result based on the first weight coefficient and the second weight coefficient, each target parameter may be fused based on the first weight coefficient and the second weight coefficient. For example, when the target parameters include the physical coordinates of the center position in the X direction and the speed in the X direction, the fusion result X of the physical coordinates of the center position in the X direction is last It can be shown as formula (1):

[0126] X last =α N X C +β N X R (1)

[0127] In the formula, X C is the physical coordinate of the center position in the X direction in the first target detection result; R is the physical coordinate of the center position in the X direction in the second target detection result.

[0128] The fusion result of the velocity in the X direction V Xlast It can be shown as formula (2):

[0129] V Xlast =α N V XC +β N V XR (2)

[0130] Where V XC V is the speed in the X direction in the first target detection result; XRis the speed in the X direction in the second target detection result.

[0131] Therefore, the weight coefficient in the fusion processing of the first target detection result and the second target detection result is determined according to the first detection error distribution information of the visual sensor and the second detection error distribution information of the radar sensor, which can further improve the fusion accuracy of the first target detection result and the second target detection result.

[0132] Corresponding to the above-mentioned target detection method, the present application embodiment also provides a target detection device, see Figure 5 As shown, the device comprises:

[0133] The data acquisition module 501 is used to acquire a first target detection result and at least one set of second target detection results of a target area in a current frame image; the first target detection result is determined based on the current frame image, and the current frame image is collected by a visual sensor; the second target detection result is determined based on radar data collected by a radar sensor; the target area is determined based on first detection error distribution information of the visual sensor and second detection error distribution information of the radar sensor;

[0134] The data fusion module 502 is used to fuse the first target detection result and the second target detection result to obtain the target detection result of the target area.

[0135] In a possible implementation manner, the first detection error distribution information includes the detection error of the visual sensor in each sub-area in the predetermined area; the second detection error includes the detection error of the radar sensor in each sub-area;

[0136] The data acquisition module 501 is also used for:

[0137] Whether to use the sub-region as the target region is determined based on a magnitude relationship between a detection error of the visual sensor and a detection error of the radar sensor in the sub-region.

[0138] In a possible implementation manner, the predetermined area is determined based on physical radiation characteristics of the radar sensor.

[0139] In a possible implementation, the data fusion module 502 may be specifically used for:

[0140] Based on the first features of each of the first target detection results and the second features of each of the second target detection results, respectively determine second target detection results that match each of the first target detection results;

[0141] fusing each of the first target detection results and a second target detection result matching the first target detection result to obtain a fusion result corresponding to each of the first target detection results;

[0142] The fusion results corresponding to the first target detection results are used as the target detection results of the target area.

[0143] In a possible implementation, the first feature includes the first target detection result and a timestamp corresponding to the first target detection result, and the second feature includes the second target detection result and a timestamp corresponding to the second target detection result.

[0144] In a possible implementation, the data fusion module 502 may be specifically used for:

[0145] Inputting a plurality of feature pairs corresponding to the first target detection result into a pre-trained matching model respectively to obtain matching information corresponding to each of the feature pairs; the feature pairs include a first feature of the first target detection result and a second feature of the second target detection result;

[0146] Based on the matching information corresponding to each of the feature pairs, a second target detection result matching the first target detection result is determined.

[0147] In a possible implementation, the data fusion module 502 may be specifically used for:

[0148] Determining a target feature pair among the feature pairs based on a comparison result of the matching information corresponding to the feature pairs;

[0149] A second target detection result corresponding to the second feature in the target feature pair is determined as a second target detection result that matches the first target detection result.

[0150] In a possible implementation, the data fusion module 502 may be specifically used for:

[0151] Based on the weight coefficient corresponding to the target area, a weighted sum is performed on the first target detection result and the second target detection result matching the first target detection result to obtain a fusion result corresponding to the first target detection result.

[0152] In a possible implementation, the data fusion module 502 may also be used to:

[0153] determining a first detection error of the visual sensor within the target area based on the first detection error distribution information, and determining a second detection error of the radar sensor within the target area based on the second detection error distribution information;

[0154] A weight coefficient corresponding to the target area is determined based on a first detection error of the visual sensor and a second detection error of the radar sensor in the target area.

[0155] The target detection device provided in this embodiment belongs to the same application concept as the target detection method provided in the above embodiments of this application, can execute the target detection method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the target detection method. For technical details not fully described in this embodiment, please refer to the specific processing content of the target detection method provided in the above embodiments of this application, which will not be repeated here.

[0156] An embodiment of the present application also provides a target detection system, which includes at least a visual sensor, a radar sensor and a processor, and the processor can be used to execute the target detection method described in any of the above embodiments.

[0157] The above-mentioned target detection system can be set on any vehicle or robot to assist the vehicle or robot in automatically perceiving the driving environment, thereby assisting the vehicle or robot in realizing intelligent driving or intelligent control.

[0158] In a possible implementation, the processor is further configured to generate the first target detection result based on the image captured by the visual sensor.

[0159] Specifically, the processor can be used to simultaneously generate a first target detection result based on the image collected by the visual sensor, receive a second target detection result corresponding to the radar sensor, and fuse the first target detection result and the second target detection result.

[0160] As an optional implementation, the processor may adopt a perception module of a visual sensor, that is, the first target detection result and the second target detection result are fused inside the perception module of the visual sensor, which is lightweight and easy to deploy.

[0161] Another embodiment of the present application also provides an electronic device, see Figure 6 As shown, the electronic device includes:

[0162] Memory 200 and processor 210;

[0163] The memory 200 is connected to the processor 210 and is used to store programs;

[0164] The processor 210 is used to implement the target detection method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0165] Specifically, the electronic device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .

[0166] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected to each other via a bus.

[0167] A bus may include a pathway that transfers information between components of a computer system.

[0168] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0169] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0170] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0171] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0172] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0173] The communication interface 220 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0174] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement each step of any target detection method provided in the above embodiments of the present application.

[0175] The method in the present application can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions, and when the computer programs or instructions are loaded and executed on a computer, the process or function described in the present application is executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, a core network device, an OAM or other programmable device.

[0176] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0177] The computer program or instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it may also be an optical medium, such as a digital video disk; it may also be a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0178] In addition, the embodiment of the present application may also be a storage medium on which a computer program is stored. The computer program is executed by a processor to execute the steps of the target detection method described in any of the above embodiments of this specification, and specifically the following steps may be implemented:

[0179] S201. Obtain a first target detection result and at least one set of second target detection results of a target area in a current frame image; the first target detection result is determined based on the current frame image, and the current frame image is collected by a visual sensor; the second target detection result is determined based on radar data collected by a radar sensor; the target area is determined based on first detection error distribution information of the visual sensor and second detection error distribution information of the radar sensor.

[0180] S202: Fusing the first target detection result and the second target detection result to obtain a target detection result of the target area.

[0181] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0182] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0183] The steps in the methods of the embodiments of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in the embodiments can be replaced or combined. The modules and submodules in the devices and terminals in the embodiments of the present application can be combined, divided and deleted according to actual needs.

[0184] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0185] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0186] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0187] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0188] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0189] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A target detection method, characterized in that: include: Obtaining a first target detection result and at least one set of second target detection results for a target area in a current frame image; The first target detection result is determined based on the current frame image, and the current frame image is collected by a visual sensor; The second target detection result is determined based on radar data collected by the radar sensor; the target area is determined based on first detection error distribution information of the visual sensor and second detection error distribution information of the radar sensor; The first target detection result and the second target detection result are fused to obtain a target detection result of the target area.

2. The method according to claim 1, characterized in that The first detection error distribution information includes the detection error of the visual sensor in each sub-area in the predetermined area; the second detection error includes the detection error of the radar sensor in each sub-area; Determining the target area based on the first detection error distribution information of the visual sensor and the second detection error distribution information of the radar sensor includes: Whether to use the sub-region as the target region is determined based on a magnitude relationship between a detection error of the visual sensor and a detection error of the radar sensor in the sub-region.

3. The method according to claim 2, characterized in that The predetermined area is determined based on physical radiation characteristics of the radar sensor.

4. The method according to any one of claims 1 to 3, characterized in that: The fusing the first target detection result and the second target detection result to obtain the target detection result of the target area includes: Based on the first features of each of the first target detection results and the second features of each of the second target detection results, respectively determine second target detection results that match each of the first target detection results; fusing each of the first target detection results and a second target detection result matching the first target detection result to obtain a fusion result corresponding to each of the first target detection results; The fusion results corresponding to the first target detection results are used as the target detection results of the target area.

5. The method according to claim 4, characterized in that The first feature includes the first target detection result and a timestamp corresponding to the first target detection result, and the second feature includes the second target detection result and a timestamp corresponding to the second target detection result.

6. The method according to claim 4, characterized in that The determining, based on the first features of the first target detection results and the second features of the second target detection results, respectively, second target detection results matching the first target detection results includes: Inputting a plurality of feature pairs corresponding to the first target detection result into a pre-trained matching model respectively to obtain matching information corresponding to each of the feature pairs; the feature pairs include a first feature of the first target detection result and a second feature of the second target detection result; Based on the matching information corresponding to each of the feature pairs, a second target detection result matching the first target detection result is determined.

7. The method according to claim 6, characterized in that The determining, based on the matching information corresponding to each of the feature pairs, a second target detection result matching the first target detection result comprises: Determining a target feature pair among the feature pairs based on a comparison result of the matching information corresponding to the feature pairs; A second target detection result corresponding to the second feature in the target feature pair is determined as a second target detection result that matches the first target detection result.

8. The method according to claim 4, characterized in that The first target detection result and the second target detection result matching the first target detection result are fused to obtain a fusion result corresponding to the first target detection result, including: Based on the weight coefficient corresponding to the target area, a first target detection result and a second target detection result matching the first target detection result are weightedly summed to obtain a fusion result corresponding to the first target detection result.

9. The method according to claim 8, characterized in that The method for determining the weight coefficient corresponding to the target area includes: determining a first detection error of the visual sensor within the target area based on the first detection error distribution information, and determining a second detection error of the radar sensor within the target area based on the second detection error distribution information; A weight coefficient corresponding to the target area is determined based on a first detection error of the visual sensor and a second detection error of the radar sensor in the target area.

10. A target detection device, characterized in that: include: A data acquisition module, used to acquire a first target detection result and at least one set of second target detection results of a target area in a current frame image; The first target detection result is determined based on the current frame image, and the current frame image is collected by a visual sensor; The second target detection result is determined based on radar data collected by the radar sensor; the target area is determined based on first detection error distribution information of the visual sensor and second detection error distribution information of the radar sensor; The data fusion module is used to fuse the first target detection result and the second target detection result to obtain the target detection result of the target area.

11. A target detection system, characterized in that: include: A visual sensor, a radar sensor and a processor, wherein the processor is used to execute the target detection method as described in any one of claims 1 to 9.

12. The system according to claim 11, characterized in that The processor is also used to generate the first target detection result based on the image captured by the visual sensor.

13. An electronic device, characterized in that: include: Memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the target detection method according to any one of claims 1 to 9 by running the program in the memory.

14. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, enable the processor to perform the target detection method according to any one of claims 1 to 9.

Images