Overlapping area determination method, target detection method, device, equipment and medium

Abstract

The present application discloses a method, device, computer equipment and storage medium for determining overlapping areas, and relates to the field of road detection technology. The method for determining overlapping areas includes obtaining a first point cloud image obtained by a millimeter-wave radar detecting in a first coverage area of ​​the millimeter-wave radar, and determining the coordinates of the first endpoint of the lane line in the first point cloud image; obtaining a first monitoring video stream obtained by a camera detecting in a second coverage area of ​​the camera, and determining the coordinates of the second endpoint of the driving track of a specific vehicle in the first monitoring video stream based on the first monitoring video stream; based on the coordinates of the first endpoint and the coordinates of the second endpoint, determining the overlapping area of ​​the first coverage area and the second coverage area, and using the overlapping area as the overlapping area in the object recognition processing process. The overlapping area determined by this method is more accurate.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of road detection technology, and in particular to an overlapping area determination method, target detection method, device, equipment and medium. Background Art

[0002] With the increasing number of vehicles in cities, it is becoming increasingly difficult to supervise the vehicle driving process. In practical applications, it is often necessary to detect targets on the road and obtain road condition information.

[0003] In traditional technology, target detection on roads usually involves acquiring road data, processing the road data using a preset algorithm, and obtaining target detection results. However, due to the large amount of road data and the limited computing power of existing computing devices, the data processing efficiency is low. Traffic detection has high latency requirements, so a traffic data processing method with high data processing efficiency is urgently needed. Summary of the invention

[0004] Based on this, it is necessary to provide a method for determining an overlapping area, a target detection method, an apparatus, a device and a medium to address the problems existing in the above methods.

[0005] A method for determining an overlapping area, the method comprising:

[0006] Obtain the millimeter-wave point cloud image of the millimeter-wave radar detection area and the video stream of the camera detection area;

[0007] Determine the lateral edge and the longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving trajectory of the dynamic target in the video stream; the lane line and the dynamic target are in the same coordinate system;

[0008] The overlapping area between the millimeter wave radar detection area and the camera detection area is determined based on the horizontal edge and the vertical edge.

[0009] In one embodiment of the present application, the method further includes:

[0010] At least one lane line in the millimeter wave point cloud image is mapped to the pixel coordinate system of the video stream using calibration parameters so that the lane line and the dynamic target are in the same coordinate system.

[0011] In one embodiment of the present application, determining the lateral edge and the longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving trajectory of the dynamic target in the video stream includes:

[0012] Get the proximal endpoint of each lane line in the pixel coordinate system; the proximal endpoint is the endpoint that is spatially close to the setting position of the millimeter wave radar;

[0013] Obtain the far end point of the driving trajectory; the far end point is the end point that is far away from the camera setting position in space;

[0014] The transverse edge and the longitudinal edge are obtained according to the driving track, each proximal end point and the distal end point.

[0015] In one embodiment of the present application, the transverse edge includes a first transverse edge and a second transverse edge, and obtaining the transverse edge and the longitudinal edge according to the driving track, each proximal endpoint and the distal endpoint includes:

[0016] Fitting each proximal end point using a preset fitting algorithm to obtain a first transverse edge;

[0017] The second lateral edge is determined based on a perpendicular line to the lane line passing through the distal end point or based on a parallel line to the first lateral edge passing through the distal end point.

[0018] In one embodiment of the present application, there are multiple dynamic targets, the longitudinal edges include a first longitudinal edge and a second longitudinal edge, and the lateral edge and the longitudinal edge are obtained according to the driving trajectory, each proximal endpoint and the distal endpoint, including:

[0019] Obtain the two outermost driving trajectories as candidate longitudinal edges;

[0020] The candidate longitudinal edges are fitted using the least square method to obtain the first longitudinal edge and the second longitudinal edge.

[0021] In one embodiment of the present application, the method further includes:

[0022] Perform frame processing on the video stream to obtain multiple frame images;

[0023] Image detection algorithm and image tracking algorithm are used to process multiple frames of images to obtain dynamic targets in the video stream and the driving trajectory of each dynamic target.

[0024] In one embodiment of the present application, obtaining the two outermost driving tracks as candidate longitudinal edges includes:

[0025] An outermost driving trajectory among the driving trajectories that are closer to a straight line among the multiple driving trajectories is obtained as a candidate longitudinal edge.

[0026] A target detection method, the method comprising:

[0027] Acquire millimeter-wave radar point cloud images and camera images at the same time and in the same scene;

[0028] Determining overlapping point cloud images and non-overlapping point cloud images from the point cloud images according to the overlapping areas, and determining overlapping camera images and non-overlapping camera images from the camera images according to the overlapping areas; wherein the overlapping areas are obtained according to the method for determining overlapping areas described in the above embodiment;

[0029] The overlapping camera images and the overlapping point cloud images are fused to obtain a fused image, and a fusion detection result is obtained according to the fused image;

[0030] Obtain point cloud detection results based on non-overlapping point cloud images, and obtain camera image detection results based on non-overlapping camera images;

[0031] The fusion detection results, point cloud detection results and camera image detection results are processed according to the position information corresponding to the point cloud image, the fusion image and the camera image to obtain the target detection result.

[0032] A device for determining an overlapping area, the device comprising:

[0033] An acquisition module is used to acquire the millimeter-wave point cloud image of the millimeter-wave radar detection area and the video stream of the camera detection area;

[0034] An edge determination module, used to determine the lateral edge and the longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving trajectory of the dynamic target in the video stream; the lane line and the dynamic target are in the same coordinate system;

[0035] The overlapping area determination module is used to determine the overlapping area between the millimeter wave radar detection area and the camera detection area according to the horizontal edge and the vertical edge.

[0036] A target detection device, comprising:

[0037] An acquisition module is used to acquire millimeter-wave radar point cloud images and camera images at the same time and in the same scene;

[0038] a determination module, configured to determine overlapping point cloud images and non-overlapping point cloud images from point cloud images according to the overlapping area, and to determine overlapping camera images and non-overlapping camera images from camera images according to the overlapping area; wherein the overlapping area is obtained according to the method for determining the overlapping area described in the above embodiment;

[0039] The first fusion module is used to fuse the overlapping camera images and the overlapping point cloud images to obtain a fused image, and obtain a fusion detection result according to the fused image;

[0040] A second fusion module is used to obtain a point cloud detection result based on the non-overlapping point cloud image and obtain a camera image detection result based on the non-overlapping camera image;

[0041] The processing module is used to process the fusion detection results, the point cloud detection results and the camera image detection results according to the position information corresponding to the point cloud image, the fusion image and the camera image to obtain the target detection result.

[0042] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the following steps are implemented:

[0043] Obtain the millimeter-wave point cloud image of the millimeter-wave radar detection area and the video stream of the camera detection area;

[0044] Determine the lateral edge and the longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving trajectory of the dynamic target in the video stream; the lane line and the dynamic target are in the same coordinate system;

[0045] The overlapping area between the millimeter wave radar detection area and the camera detection area is determined based on the horizontal edge and the vertical edge.

[0046] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the following steps are implemented:

[0047] Acquire millimeter-wave radar point cloud images and camera images at the same time and in the same scene;

[0048] Determining overlapping point cloud images and non-overlapping point cloud images from the point cloud images according to the overlapping areas, and determining overlapping camera images and non-overlapping camera images from the camera images according to the overlapping areas; wherein the overlapping areas are obtained according to the method for determining overlapping areas described in the above embodiment;

[0049] The overlapping camera images and the overlapping point cloud images are fused to obtain a fused image, and a fusion detection result is obtained according to the fused image;

[0050] Obtain point cloud detection results based on non-overlapping point cloud images, and obtain camera image detection results based on non-overlapping camera images;

[0051] The fusion detection results, point cloud detection results and camera image detection results are processed according to the position information corresponding to the point cloud image, the fusion image and the camera image to obtain the target detection result.

[0052] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0053] Obtain the millimeter-wave point cloud image of the millimeter-wave radar detection area and the video stream of the camera detection area;

[0054] Determine the lateral edge and the longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving trajectory of the dynamic target in the video stream; the lane line and the dynamic target are in the same coordinate system;

[0055] The overlapping area between the millimeter wave radar detection area and the camera detection area is determined based on the horizontal edge and the vertical edge.

[0056] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0057] Acquire millimeter-wave radar point cloud images and camera images at the same time and in the same scene;

[0058] Determining overlapping point cloud images and non-overlapping point cloud images from the point cloud images according to the overlapping areas, and determining overlapping camera images and non-overlapping camera images from the camera images according to the overlapping areas; wherein the overlapping areas are obtained according to the method for determining overlapping areas described in the above embodiment;

[0059] The overlapping camera images and the overlapping point cloud images are fused to obtain a fused image, and a fusion detection result is obtained according to the fused image;

[0060] Obtain point cloud detection results based on non-overlapping point cloud images, and obtain camera image detection results based on non-overlapping camera images;

[0061] The fusion detection results, point cloud detection results and camera image detection results are processed according to the position information corresponding to the point cloud image, the fusion image and the camera image to obtain the target detection result.

[0062] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0063] The above-mentioned overlapping area determination method, target detection method, device, equipment and medium can improve the accuracy of the overlapping area. The overlapping area determination method includes obtaining a millimeter wave point cloud image of the millimeter wave radar detection area and a video stream of the camera detection area; determining the lateral edge and longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving trajectory of the dynamic target in the video stream; the lane line and the dynamic target are in the same coordinate system; determining the overlapping area between the millimeter wave radar detection area and the camera detection area according to the lateral edge and the longitudinal edge. The method of this implementation determines the overlapping area more accurately, so the data used for multi-sensor data fusion and target detection based on the overlapping area is more accurate, so the multi-sensor data fusion result obtained is more refined, and the data processing efficiency for target detection is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic diagram of an implementation environment of a method for determining an overlapping area provided in an embodiment of the present application;

[0065] Figure 2 A schematic diagram of image distortion provided in an embodiment of the present application;

[0066] Figure 3 A flowchart of a method for determining an overlapping area provided in an embodiment of the present application;

[0067] Figure 4 A flowchart of another method for determining an overlapping area provided in an embodiment of the present application;

[0068] Figure 5 A schematic diagram of an overlapping area provided in an embodiment of the present application;

[0069] Figure 6 A schematic diagram of multiple lane lines provided in an embodiment of the present application;

[0070] Figure 7 A schematic diagram of an overlapping area provided in an embodiment of the present application;

[0071] Figure 8 A flow chart of a target detection method provided in an embodiment of the present application;

[0072] Fig. 9 A block diagram of a device for determining an overlapping area provided in an embodiment of the present application;

[0073] Fig.10 A block diagram of a target detection device provided in an embodiment of the present application;

[0074] Fig.11 An internal structure diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0076] In one embodiment, a method for determining an overlapping area is provided, the method comprising obtaining a millimeter wave point cloud image of a millimeter wave radar detection area and a video stream of a camera detection area; determining the lateral edge and longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving trajectory of a dynamic target in the video stream; the lane line and the dynamic target are in the same coordinate system; and determining the overlapping area between the millimeter wave radar detection area and the camera detection area according to the lateral edge and the longitudinal edge. The method of this implementation determines the overlapping area more accurately, so the data used for multi-sensor data fusion and target detection based on the overlapping area is more accurate, so the multi-sensor data fusion result obtained is more refined, and the data processing efficiency for target detection is higher.

[0077] Below, the implementation environment involved in the method for determining the overlapping area provided in the embodiment of the present application will be briefly described.

[0078] Figure 1 A roadside perception system to which the above overlapping area determination method is applicable, such as Figure 1 As shown, the roadside perception system may include a millimeter-wave radar 101, a camera 102 and a roadside unit (English: Road Side Unit, abbreviated as: RSU) 103. The dotted line represents the lane line. Generally speaking, the height of the millimeter-wave radar 101 and the camera 102 from the ground is 4 to 6 meters. The coverage of the millimeter-wave radar 101 and the camera 102 can be expanded by a higher height. There is an overlap in the detection area of ​​the millimeter-wave radar 101 and the camera 102, and the overlapping range is called the overlapping area. The millimeter-wave radar has the advantages of long-range measurement and accurate speed measurement, but the millimeter-wave radar has a weak ability to classify targets, and sometimes even identifies one target as two; the video stream collected by the camera can be used to identify the target type, but the ability to identify the distance and speed of the target is weak.

[0079] Generally speaking, the detection range of a millimeter-wave radar is approximately 30 to 300 meters from the location of the millimeter-wave radar, and the detection range of a camera is approximately 4 to 120 meters from the location of the camera.

[0080] The roadside perception system can fuse the acquired millimeter wave point cloud data with the camera image or video stream. The following operations are required for fusion processing:

[0081] 1. System installation initialization: After the millimeter wave radar and camera are installed, the sampling frequencies of the millimeter wave radar and camera need to be adjusted so that they are the same or approximately the same. For example, the difference between the sampling frequencies of the millimeter wave radar and camera can be adjusted to be less than the frequency threshold. This allows the millimeter wave radar and camera to collect data in time synchronization.

[0082] 2. Data time synchronization: After adjusting the sampling frequency of the millimeter-wave radar and the camera, the process of realizing time synchronization processing of the millimeter-wave radar and the camera includes: obtaining the millimeter-wave radar and the camera's timestamps t1 and t2 accurate to milliseconds in real time (if they cannot be obtained directly, they can be converted into timestamps under the same time axis through time axis matching, the reference time axis is t, and the difference Δt is converted according to the time axis, and the conversion time t'=t-Δt); then calculate the absolute value of the timestamp difference |tx-ty| and determine whether it is less than a certain set fixed value δ, for example, it can be set to δ=10ms; if the absolute value is less than δ, it is considered that the two frames of data are obtained at the same time; if the absolute value is greater than δ, then the next frame is searched for time matching according to a certain frame rate. It should be noted that this embodiment only provides a solution for data time synchronization, and other methods can also be used for time synchronization processing in actual data processing, and this application does not limit this.

[0083] 3. System calibration: In one embodiment, a system calibration method is proposed to obtain the calibration parameters of the roadside perception system. Obtain the camera's intrinsic parameters. Definition of camera intrinsic parameters: The camera's intrinsic parameters are the projection of the three-dimensional coordinate points in the camera coordinate system onto the imaging plane, mainly including the intrinsic parameter matrix and the distortion coefficient.

[0084] Among them, the internal parameter matrix can be expressed as:

[0085]

[0086] Among them, the values ​​in the intrinsic parameter matrix are only related to the internal parameters of the camera and do not change with the position of the object. Among them, f represents the focal length, dx and dy represent pixels / mm, and u0 and v0 represent the horizontal and vertical pixel numbers between the pixel coordinates of the center of the image coordinate system and the pixel coordinates of the image origin.

[0087] Among them, the distortion coefficient can eliminate the distortion effect of the camera convex lens, mainly including radial distortion and tangential distortion. Radial distortion is caused by the manufacturing process of the lens shape, including barrel distortion and pincushion distortion, such as Figure 2 shown.

[0088] The radial distortion coefficient can be expressed as follows:

[0089]

[0090] The tangential distortion coefficient can be expressed as follows:

[0091]

[0092] Among them, k1, k2, k3, p1 and p2 are distortion parameters.

[0093] According to the intrinsic parameter matrix of the camera and the initial joint calibration parameters of the millimeter-wave radar relative to the camera, the point cloud corresponding to the millimeter-wave point cloud data can be mapped to the image to obtain the mapped point cloud; then the overlapping area between the point cloud target frame of the calibration object on the image and the image recognition target frame is calculated, the calibration object is at least one target in the overlapping detection area of ​​the camera and the millimeter-wave radar, the point cloud target frame is a target frame drawn on the image based on the calibration object in the mapped point cloud, and the image recognition target frame is a target frame of the calibration object obtained by image recognition of the image; based on the overlapping area corresponding to each calibration object, the initial joint calibration parameters are adjusted until the overlapping area meets the preset threshold, and the adjusted joint calibration parameters are output as the target joint calibration parameters of the millimeter-wave radar relative to the camera. The target joint calibration parameters are used to spatially synchronize the millimeter-wave point cloud data with the camera data.

[0094] Please refer to Figure 3 , which shows a flowchart of a method for determining an overlapping area provided in an embodiment of the present application. The method for determining an overlapping area can be applied to Figure 1 In the roadside unit in the roadside perception system shown, the method for determining the overlapping area includes:

[0095] Step 301: The roadside unit obtains the millimeter wave point cloud image of the millimeter wave radar detection area and the video stream of the camera detection area.

[0096] In the embodiment of the present application, the millimeter wave radar can emit electromagnetic waves to scan the entire detection area of ​​the millimeter wave radar. The radar receives the echo signals reflected by all targets in the detection area of ​​the millimeter wave radar. The millimeter wave radar receives the echo signals, processes the echo signals, obtains the millimeter wave point cloud image, and sends the millimeter wave point cloud image to the roadside unit. In the embodiment of the present application, the camera installed on the side of the road can obtain the video stream of the camera detection area.

[0097] Step 302: The roadside unit determines the lateral edge and the longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving trajectory of the dynamic target in the video stream.

[0098] Among them, the lane lines and dynamic targets are in the same coordinate system.

[0099] In an embodiment of the present application, the process of a roadside unit acquiring at least one lane line in a millimeter-wave point cloud image may include the following contents: there is at least one lane line in the millimeter-wave radar detection area, and the roadside unit may acquire the coordinates of each discrete point on the at least one lane line based on the millimeter-wave point cloud image, and obtain a continuous lane line by fitting each discrete point.

[0100] In an embodiment of the present application, the process of the roadside unit acquiring the driving trajectory of the dynamic target in the video stream may include the following contents: the dynamic target in the video stream may be, for example, a specific vehicle. The video stream includes image frames of the front of the dynamic target entering the camera detection area and image frames of the rear of the dynamic target leaving the camera detection area. Among them, the roadside unit may perform frame processing on the video stream to obtain multiple frames of images. Then, the multiple frames of images are processed using an image detection algorithm and an image tracking algorithm to obtain the dynamic targets in the video stream and the driving trajectory of each dynamic target. Optionally, in an embodiment of the present application, the roadside unit may use opencv to frame the video stream into multiple frames of images. Optionally, in an embodiment of the present application, the image tracking algorithm may be, for example, a pedestrian re-identification algorithm, and the image detection algorithm may be, for example, a Yolo target detection algorithm.

[0101] In an embodiment of the present application, the roadside unit can determine the lateral edge of the overlapping area based on at least one lane line in the millimeter wave point cloud image, and determine the longitudinal edge of the overlapping area based on the driving trajectory of the dynamic target in the video stream.

[0102] Furthermore, since the three-dimensional coordinates of the lane lines in the millimeter wave point cloud image are in the millimeter radar coordinate system, and the two-dimensional coordinates of the dynamic targets in the video stream are in the pixel coordinate system, in order to determine the overlapping area, the lane lines and the dynamic targets need to be unified in the same coordinate system. In the embodiment of the present application, the roadside unit can use the calibration parameters to map at least one lane line in the millimeter wave point cloud image to the pixel coordinate system of the video stream, so that the lane lines and the dynamic targets are in the same coordinate system.

[0103] It should be noted that in the following embodiments, the coordinates of the default lane lines and dynamic targets are in the same pixel coordinate system.

[0104] Step 303: The roadside unit determines the overlapping area between the millimeter wave radar detection area and the camera detection area according to the lateral edge and the longitudinal edge.

[0105] In the embodiment of the present application, the area enclosed by the horizontal edge and the vertical edge is the overlapping area between the millimeter wave radar detection area and the camera detection area.

[0106] The method for determining the overlapping area provided in the embodiment of the present application includes obtaining a millimeter wave point cloud image of the millimeter wave radar detection area and a video stream of the camera detection area; determining the lateral edge and longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving trajectory of the dynamic target in the video stream; the lane line and the dynamic target are in the same coordinate system; and determining the overlapping area between the millimeter wave radar detection area and the camera detection area according to the lateral edge and the longitudinal edge. The method of this implementation determines the overlapping area more accurately, so the data used for multi-sensor data fusion and target detection based on the overlapping area is more accurate, so the multi-sensor data fusion result obtained is more refined, and the data processing efficiency for target detection is higher.

[0107] In an optional implementation, Figure 4 As shown, the embodiment of the present application provides another method for determining an overlapping area, which may include the following contents:

[0108] Step 401: The roadside unit obtains the millimeter wave point cloud image of the millimeter wave radar detection area and the video stream of the camera detection area.

[0109] Step 402: The roadside unit obtains the proximal end point of each lane line in the pixel coordinate system.

[0110] Among them, the proximal endpoint is an endpoint that is spatially close to the setting position of the millimeter wave radar.

[0111] In the embodiment of the present application, the lane line is composed of discrete points. The point on the lane line closest to the millimeter-wave radar setting position can be determined by comparing the coordinates of each discrete point on the lane line. The closest point is the proximal endpoint of the lane line.

[0112] Step 403: The roadside unit obtains the far end point of the driving trajectory.

[0113] The far end point is an end point that is far away from the camera setting position in space.

[0114] Optionally, in an embodiment of the present application, the far endpoint is the point on the driving trajectory that is farthest from the camera setting position, and the far endpoint is the point at the farthest position that can be detected by the camera.

[0115] Optionally, the dynamic target may have multiple driving trajectories, wherein the dynamic target may be controlled to travel in a straight line at the edge positions on both sides of the camera detection area for multiple times, thereby obtaining multiple driving trajectories. For each driving trajectory, the roadside unit may detect the point on the driving trajectory that is farthest from the camera setting position, thereby obtaining multiple far-end endpoints.

[0116] Step 404: The roadside unit obtains the lateral edge and the longitudinal edge according to the driving trajectory, each proximal endpoint and the distal endpoint.

[0117] In the embodiment of the present application, the near-end endpoint corresponds to the starting position of the millimeter-wave radar detection area, and the far-end endpoint corresponds to the ending position of the camera detection area. Figure 5 As shown, the millimeter wave radar detection area is represented by the AB line segment, and the camera detection area is represented by the CD line segment, wherein C represents the near end point and B represents the far end point. The area between the near end point and the far end point is the overlapping area.

[0118] In an optional implementation, the roadside unit may determine the lateral edge based on each proximal endpoint and distal endpoint. The process is as follows:

[0119] Optionally, in an embodiment of the present application, the lateral edge may include a first lateral edge and a second lateral edge. The roadside unit may use a preset fitting algorithm to fit each proximal endpoint to obtain the first lateral edge, and then determine the second lateral edge based on a perpendicular line of the lane line passing through the distal endpoint or based on a parallel line to the first lateral edge passing through the distal endpoint.

[0120] Among them, Figure 6As shown, five lane lines are shown by way of example, which are represented by dotted lines, respectively, wherein A1, A2, A3, A4 and A5 represent the proximal end points of each lane line. The roadside unit can connect two adjacent proximal end points to obtain a candidate straight line, wherein the candidate straight line can be as follows Figure 6 The roadside unit can sharpen the candidate straight line to obtain the first lateral edge, which can be as shown in the dashed line in FIG. Figure 5 As shown by the solid line in .

[0121] Optionally, based on obtaining the first lateral edge, the roadside unit may generate a vertical line passing through the far end point and perpendicular to the lane line, and determine the vertical line as the second lateral edge.

[0122] Optionally, when there are multiple distal endpoints, the roadside unit may connect adjacent distal endpoints to obtain a candidate straight line, and then sharpen the candidate straight line to obtain a second lateral edge.

[0123] In an optional implementation, the dynamic target's driving trajectories include at least two, and the at least two driving trajectories are respectively located at the edge positions on both sides of the camera detection area. The roadside unit can select two from the at least two driving trajectories as the longitudinal edges of the overlapping area.

[0124] In an optional implementation, there are multiple dynamic targets, and the longitudinal edges include a first longitudinal edge and a second longitudinal edge. The process of the roadside unit determining the lateral edge according to the driving trajectory may be: the multiple dynamic targets may be controlled to repeatedly drive along the edge positions on both sides of the camera detection area in a straight line, thereby obtaining multiple driving trajectories. The roadside unit may select two of the at least two driving trajectories as the first longitudinal edge and the second longitudinal edge, respectively.

[0125] Optionally, the roadside unit may obtain two outermost driving trajectories as candidate longitudinal edges, and then use the least squares method to fit the candidate longitudinal edges to obtain the first longitudinal edge and the second longitudinal edge.

[0126] Optionally, the process of the roadside unit obtaining the two outermost driving tracks as candidate longitudinal edges may include the following: obtaining the outermost driving track among the driving tracks that are closer to a straight line among the multiple driving tracks as the candidate longitudinal edges. The roadside unit may detect the straightness of the multiple driving tracks, the higher the straightness, the closer the driving track is to a straight line, and the driving tracks with straightness greater than a threshold are taken as candidate driving tracks, and then the outermost driving tracks are obtained from the candidate driving tracks as candidate longitudinal edges.

[0127] Step 405: The roadside unit forms an overlapping area according to the area enclosed by the first lateral edge, the second lateral edge, the first longitudinal edge, and the second longitudinal edge.

[0128] like Figure 7 As shown, Figure 7 A first longitudinal edge L1 and a second longitudinal edge L2 are shown, as well as a first transverse edge L3 and a second transverse edge L4.

[0129] The roadside unit can obtain the coordinates of the two first target intersection points of the first lateral edge L3 and the first longitudinal edge L1 and the second longitudinal edge L2 in the pixel coordinate system, and obtain the coordinates of the two second target intersection points of the second lateral edge L4 and the first longitudinal edge L1 and the second longitudinal edge L2 in the pixel coordinate system, and determine the range of the overlapping area according to the coordinates of the two first target intersection points in the pixel coordinate system and the coordinates of the two second target intersection points in the pixel coordinate system.

[0130] In the embodiment of the present application, the first lateral edge is determined by the proximal endpoint, the second lateral edge is determined by the distal endpoint, and then the first longitudinal edge and the second longitudinal edge are determined by the outermost driving track, and then the overlapping area is determined by the area surrounded by the first lateral edge, the second lateral edge, the first longitudinal edge and the second longitudinal edge. On the one hand, the overlapping area is determined by combining the actual detection data of the millimeter wave radar and the camera, so that the overlapping area is more accurate. On the other hand, for the target in the overlapping area, the target information can be directly obtained by fusing the point cloud detection results and the image detection results, which improves the data processing efficiency and accuracy of obtaining the target information.

[0131] Please refer to Figure 8 , which shows a flow chart of a target detection method provided by an embodiment of the present application. The target detection method can be applied to Figure 1 In the roadside unit shown, the method for determining the overlapping area includes:

[0132] Step 801: The roadside unit obtains the millimeter-wave radar point cloud image and the camera image at the same time and in the same scene.

[0133] In step 802, the roadside unit determines overlapping point cloud images and non-overlapping point cloud images from the point cloud images according to the overlapping area, and determines overlapping camera images and non-overlapping camera images from the camera images according to the overlapping area; wherein the overlapping area is obtained according to the method for determining the overlapping area described in the above embodiment.

[0134] In this embodiment of the present application, the roadside unit can determine the portion of the point cloud image that is within the overlapping area, that is, obtain the overlapping point cloud image.

[0135] The roadside unit can determine the portion within the overlapping area from the camera images, that is, obtain the overlapping camera images.

[0136] Step 803: The roadside unit fuses the overlapping camera images and the overlapping point cloud images to obtain a fused image, and acquires a fused detection result based on the fused image.

[0137] In the embodiment of the present application, the roadside unit performs fusion processing on the overlapping camera images and the overlapping point cloud images.

[0138] In the embodiment of the present application, the fusion detection result includes target information of each target in the overlapping area.

[0139] Step 804: The roadside unit obtains point cloud detection results based on the non-overlapping point cloud images, and obtains camera image detection results based on the non-overlapping camera images.

[0140] In an embodiment of the present application, the roadside unit can obtain target information of targets outside the overlapping area and within the millimeter-wave radar detection area based on the non-overlapping point cloud image, wherein the target information of targets outside the overlapping area and within the millimeter-wave radar detection area is determined based on the point cloud target information.

[0141] The roadside unit may obtain target information of a target outside the overlapping area and within the camera detection area based on the non-overlapping camera images, wherein the target information of the target outside the overlapping area and within the camera detection area is determined based on the camera target information.

[0142] Step 805: The roadside unit processes the fusion detection result, the point cloud detection result and the camera image detection result according to the point cloud image, the fusion image and the position information corresponding to the camera image to obtain the target detection result.

[0143] In an embodiment of the present application, the roadside unit can combine the target information of the target located in the non-overlapping point cloud image, the target information of the target located in the overlapping area, and the target information of the target located in the non-overlapping camera image to obtain the target detection results in the millimeter-wave radar detection area and the camera detection area.

[0144] The target detection method provided in the embodiment of the present application divides the system perception area, uses the point cloud of the millimeter-wave radar for perception at the far end, uses the camera image for perception at the near end, and performs fusion perception in the overlapping area of ​​the two sensors. Therefore, the target detection method of this embodiment has a wider perception range (in addition to the fusion segment, it also includes the detection results of the non-fusion segment), and the target detection method of this implementation applies the method in the above-mentioned embodiment to determine the overlapping area, and the determined overlapping area is more accurate. Therefore, multi-sensor data fusion and target detection are performed based on the overlapping area, and the data used is more accurate. Therefore, the obtained multi-sensor data fusion result is more refined, and the data processing efficiency for target detection is higher.

[0145] It should be understood that although Figure 2-Figure 8 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-Figure 8 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0146] Please refer to Fig. 9 , which shows a block diagram of a device for determining an overlapping area provided in an embodiment of the present application. The device for determining an overlapping area can be configured in Figure 1 In the roadside unit in the implementation environment shown, Figure 8 As shown, the overlapping area determination device may include an acquisition module 901, an edge determination module 902 and an overlapping area determination module 903, wherein:

[0147] An acquisition module 901 is used to acquire a millimeter-wave point cloud image of a millimeter-wave radar detection area and a video stream of a camera detection area;

[0148] The edge determination module 902 is used to determine the lateral edge and the longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving track of the dynamic target in the video stream; the lane line and the dynamic target are in the same coordinate system;

[0149] The overlapping area determination module 903 is used to determine the overlapping area between the millimeter wave radar detection area and the camera detection area according to the horizontal edge and the vertical edge.

[0150] In one embodiment of the present application, the edge determination module 902 is further configured to:

[0151] At least one lane line in the millimeter wave point cloud image is mapped to the pixel coordinate system of the video stream using calibration parameters so that the lane line and the dynamic target are in the same coordinate system.

[0152] In one embodiment of the present application, the edge determination module 902 is further configured to:

[0153] Get the proximal endpoint of each lane line in the pixel coordinate system; the proximal endpoint is the endpoint that is spatially close to the setting position of the millimeter wave radar;

[0154] Obtain the far end point of the driving trajectory; the far end point is the end point that is far away from the camera setting position in space;

[0155] The transverse edge and the longitudinal edge are obtained according to the driving track, each proximal end point and the distal end point.

[0156] In one embodiment of the present application, the horizontal edge includes a first horizontal edge and a second horizontal edge, and the edge determination module 902 is further configured to:

[0157] Fitting each proximal end point using a preset fitting algorithm to obtain a first transverse edge;

[0158] The second lateral edge is determined based on a perpendicular line to the lane line passing through the distal end point or based on a parallel line to the first lateral edge passing through the distal end point.

[0159] In one embodiment of the present application, there are multiple dynamic targets, and the longitudinal edges include a first longitudinal edge and a second longitudinal edge. The edge determination module 902 is further used to:

[0160] Obtain the two outermost driving trajectories as candidate longitudinal edges;

[0161] The candidate longitudinal edges are fitted using the least square method to obtain the first longitudinal edge and the second longitudinal edge.

[0162] In one embodiment of the present application, the edge determination module 902 is further configured to:

[0163] Perform frame processing on the video stream to obtain multiple frame images;

[0164] Image detection algorithm and image tracking algorithm are used to process multiple frames of images to obtain dynamic targets in the video stream and the driving trajectory of each dynamic target.

[0165] In one embodiment of the present application, the edge determination module 902 is further configured to:

[0166] An outermost driving trajectory among the driving trajectories that are closer to a straight line among the multiple driving trajectories is obtained as a candidate longitudinal edge.

[0167] For the specific definition of the overlapping area determination device, please refer to the definition of the overlapping area determination method above, which will not be repeated here. Each module in the above overlapping area determination device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0168] Please refer to Fig.10, which shows a block diagram of a target detection device provided by an embodiment of the present application, the target detection device can be configured in Figure 1 In the roadside unit in the implementation environment shown, Fig.10 As shown, the target detection device may include an acquisition module 1001, a determination module 1002, a first fusion module 1003, a second fusion module 1004 and a processing module 1005, wherein:

[0169] An acquisition module 1001 is used to acquire a millimeter-wave radar point cloud image and a camera image at the same time and in the same scene;

[0170] A determination module 1002 is used to determine overlapping point cloud images and non-overlapping point cloud images from point cloud images according to the overlapping area, and to determine overlapping camera images and non-overlapping camera images from camera images according to the overlapping area; wherein the overlapping area is obtained according to the method for determining the overlapping area described in the above embodiment;

[0171] The first fusion module 1003 is used to fuse the overlapping camera images and the overlapping point cloud images to obtain a fused image, and obtain a fusion detection result according to the fused image;

[0172] The second fusion module 1004 is used to obtain a point cloud detection result according to the non-overlapping point cloud image, and obtain a camera image detection result according to the non-overlapping camera image;

[0173] The processing module 1005 is used to process the fusion detection result, the point cloud detection result and the camera image detection result according to the position information corresponding to the point cloud image, the fusion image and the camera image to obtain the target detection result.

[0174] For the specific definition of the target detection device, please refer to the definition of the target detection method above, which will not be repeated here. Each module in the above target detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0175] In one embodiment of the present application, a computer device is provided. The computer device may be a roadside unit, and its internal structure diagram may be as follows: Fig.11As shown. The computer device includes a processor, a memory and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database can be used to store a preset first coordinate transformation matrix and a second coordinate transformation matrix, and the computer program is executed by the processor to implement a method for determining an overlapping area. Alternatively, the computer program is executed by the processor to implement a target detection method.

[0176] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0177] In one embodiment of the present application, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0178] Obtain the millimeter-wave point cloud image of the millimeter-wave radar detection area and the video stream of the camera detection area;

[0179] Determine the lateral edge and the longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving trajectory of the dynamic target in the video stream; the lane line and the dynamic target are in the same coordinate system;

[0180] The overlapping area between the millimeter wave radar detection area and the camera detection area is determined based on the horizontal edge and the vertical edge.

[0181] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0182] At least one lane line in the millimeter wave point cloud image is mapped to the pixel coordinate system of the video stream using calibration parameters so that the lane line and the dynamic target are in the same coordinate system.

[0183] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0184] Get the proximal endpoint of each lane line in the pixel coordinate system; the proximal endpoint is the endpoint that is spatially close to the setting position of the millimeter wave radar;

[0185] Obtain the far end point of the driving trajectory; the far end point is the end point that is far away from the camera setting position in space;

[0186] The transverse edge and the longitudinal edge are obtained according to the driving track, each proximal end point and the distal end point.

[0187] In one embodiment of the present application, the lateral edge includes a first lateral edge and a second lateral edge, and the processor further implements the following steps when executing the computer program:

[0188] Fitting each proximal end point using a preset fitting algorithm to obtain a first transverse edge;

[0189] The second lateral edge is determined based on a perpendicular line to the lane line passing through the distal end point or based on a parallel line to the first lateral edge passing through the distal end point.

[0190] In one embodiment of the present application, there are multiple dynamic targets, and the longitudinal edges include a first longitudinal edge and a second longitudinal edge. When the processor executes the computer program, the following steps are also implemented:

[0191] Obtain the two outermost driving trajectories as candidate longitudinal edges;

[0192] The candidate longitudinal edges are fitted using the least square method to obtain the first longitudinal edge and the second longitudinal edge.

[0193] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0194] Perform frame processing on the video stream to obtain multiple frame images;

[0195] Image detection algorithm and image tracking algorithm are used to process multiple frames of images to obtain dynamic targets in the video stream and the driving trajectory of each dynamic target.

[0196] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0197] An outermost driving trajectory among the driving trajectories that are closer to a straight line among the multiple driving trajectories is obtained as a candidate longitudinal edge.

[0198] The computer device provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.

[0199] In one embodiment of the present application, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0200] Acquire millimeter-wave radar point cloud images and camera images at the same time and in the same scene;

[0201] Determining overlapping point cloud images and non-overlapping point cloud images from the point cloud images according to the overlapping areas, and determining overlapping camera images and non-overlapping camera images from the camera images according to the overlapping areas; wherein the overlapping areas are obtained according to the method for determining overlapping areas described in the above embodiment;

[0202] The overlapping camera images and the overlapping point cloud images are fused to obtain a fused image, and a fusion detection result is obtained according to the fused image;

[0203] Obtain point cloud detection results based on non-overlapping point cloud images, and obtain camera image detection results based on non-overlapping camera images;

[0204] The fusion detection results, point cloud detection results and camera image detection results are processed according to the position information corresponding to the point cloud image, the fusion image and the camera image to obtain the target detection result.

[0205] The computer device provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.

[0206] In one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0207] Obtain the millimeter-wave point cloud image of the millimeter-wave radar detection area and the video stream of the camera detection area;

[0208] Determine the lateral edge and the longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving trajectory of the dynamic target in the video stream; the lane line and the dynamic target are in the same coordinate system;

[0209] The overlapping area between the millimeter wave radar detection area and the camera detection area is determined based on the horizontal edge and the vertical edge.

[0210] In one embodiment of the present application, when the computer program is executed by a processor, the following steps can also be implemented: at least one lane line in the millimeter wave point cloud image is mapped to the pixel coordinate system of the video stream using calibration parameters so that the lane line and the dynamic target are in the same coordinate system.

[0211] In one embodiment of the present application, when the computer program is executed by the processor, the following steps may also be implemented: obtaining the proximal endpoint of each lane line in the pixel coordinate system; the proximal endpoint is an endpoint that is spatially close to the setting position of the millimeter wave radar;

[0212] Obtain the far end point of the driving trajectory; the far end point is the end point that is far away from the camera setting position in space;

[0213] The transverse edge and the longitudinal edge are obtained according to the driving track, each proximal end point and the distal end point.

[0214] In one embodiment of the present application, the transverse edge includes a first transverse edge and a second transverse edge, and the computer program can also implement the following steps when executed by the processor: fitting each proximal end point using a preset fitting algorithm to obtain a first transverse edge;

[0215] The second lateral edge is determined based on a perpendicular line to the lane line passing through the distal end point or based on a parallel line to the first lateral edge passing through the distal end point.

[0216] In one embodiment of the present application, there are multiple dynamic targets, and the longitudinal edges include a first longitudinal edge and a second longitudinal edge. When the computer program is executed by the processor, the following steps may also be implemented:

[0217] Obtain the two outermost driving trajectories as candidate longitudinal edges;

[0218] The candidate longitudinal edges are fitted using the least square method to obtain the first longitudinal edge and the second longitudinal edge.

[0219] In one embodiment of the present application, when the computer program is executed by the processor, the following steps may also be implemented: performing frame processing on the video stream to obtain multiple frames of images;

[0220] Image detection algorithm and image tracking algorithm are used to process multiple frames of images to obtain dynamic targets in the video stream and the driving trajectory of each dynamic target.

[0221] In one embodiment of the present application, when the computer program is executed by the processor, the following steps may also be implemented: obtaining the outermost driving track among the driving tracks that are closer to a straight line among the multiple driving tracks as a candidate longitudinal edge.

[0222] The computer-readable storage medium provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.

[0223] In one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0224] Acquire millimeter-wave radar point cloud images and camera images at the same time and in the same scene;

[0225] Determining overlapping point cloud images and non-overlapping point cloud images from the point cloud images according to the overlapping areas, and determining overlapping camera images and non-overlapping camera images from the camera images according to the overlapping areas; wherein the overlapping areas are obtained according to the method for determining overlapping areas described in the above embodiment;

[0226] The overlapping camera images and the overlapping point cloud images are fused to obtain a fused image, and a fusion detection result is obtained according to the fused image;

[0227] Obtain point cloud detection results based on non-overlapping point cloud images, and obtain camera image detection results based on non-overlapping camera images;

[0228] The fusion detection results, point cloud detection results and camera image detection results are processed according to the position information corresponding to the point cloud image, the fusion image and the camera image to obtain the target detection result.

[0229] The computer-readable storage medium provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.

[0230] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0231] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0232] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A method for determining an overlapping area, characterized in that: The method comprises: Obtain the millimeter-wave point cloud image of the millimeter-wave radar detection area and the video stream of the camera detection area; Determine the lateral edge and the longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving track of the dynamic target in the video stream; the lane line and the dynamic target are in the same coordinate system; An overlapping area between the millimeter-wave radar detection area and the camera detection area is determined according to the lateral edge and the longitudinal edge.

2. The method according to claim 1, characterized in that The method further comprises: At least one lane line in the millimeter wave point cloud image is mapped to a pixel coordinate system of the video stream using calibration parameters so that the lane line and the dynamic target are in the same coordinate system.

3. The method according to claim 2, characterized in that The determining of the lateral edge and the longitudinal edge of the overlapping area according to at least one lane line in the millimeter wave point cloud image and the driving track of the dynamic target in the video stream includes: Obtaining the proximal endpoint of each lane line in a pixel coordinate system; the proximal endpoint is an endpoint spatially close to the setting position of the millimeter wave radar; Acquire the distal endpoint of the driving trajectory; the distal endpoint is an endpoint that is spatially far away from the camera setting position; The transverse edge and the longitudinal edge are acquired according to the driving track, each of the proximal end points and the distal end point.

4. The method according to claim 3, characterized in that The transverse edge includes a first transverse edge and a second transverse edge, and the step of acquiring the transverse edge and the longitudinal edge according to the driving track, each of the proximal endpoints and the distal endpoint includes: Fitting each of the proximal endpoints using a preset fitting algorithm to obtain the first transverse edge; The second lateral edge is determined according to a perpendicular line to the lane line passing through the distal end point or according to a parallel line to the first lateral edge passing through the distal end point.

5. The method according to claim 3 or 4, characterized in that: There are multiple dynamic targets, the longitudinal edges include a first longitudinal edge and a second longitudinal edge, and obtaining the transverse edge and the longitudinal edge according to the driving trajectory, each of the proximal endpoints and the distal endpoint includes: Obtain the two outermost driving trajectories as candidate longitudinal edges; The candidate longitudinal edges are fitted using a least square method to obtain the first longitudinal edge and the second longitudinal edge.

6. The method according to claim 5, characterized in that The method further comprises: Performing frame processing on the video stream to obtain multiple frames of images; The multiple frames of images are processed using an image detection algorithm and an image tracking algorithm to obtain dynamic targets in the video stream and the driving tracks of each dynamic target.

7. The method according to claim 5, characterized in that The step of obtaining the two outermost driving tracks as candidate longitudinal edges includes: The outermost driving track among the driving tracks that are closer to a straight line among the multiple driving tracks is obtained as the candidate longitudinal edge.

8. A target detection method, characterized in that: The method comprises: Acquire millimeter-wave radar point cloud images and camera images at the same time and in the same scene; Determining overlapping point cloud images and non-overlapping point cloud images from the point cloud images according to the overlapping area, and determining overlapping camera images and non-overlapping camera images from the camera images according to the overlapping area; wherein the overlapping area is obtained according to the method for determining the overlapping area according to any one of claims 1 to 7; Performing fusion processing on the overlapping camera images and the overlapping point cloud images to obtain a fused image, and obtaining a fusion detection result according to the fused image; Acquire a point cloud detection result according to the non-overlapping point cloud image, and acquire a camera image detection result according to the non-overlapping camera image; The fusion detection result, the point cloud detection result and the camera image detection result are processed according to the position information corresponding to the point cloud image, the fusion image and the camera image to obtain a target detection result.

9. A device for determining an overlapping area, characterized in that: The device comprises: An acquisition module is used to acquire the millimeter-wave point cloud image of the millimeter-wave radar detection area and the video stream of the camera detection area; An edge determination module, configured to determine a lateral edge and a longitudinal edge of an overlapping area according to at least one lane line in the millimeter wave point cloud image and a driving track of a dynamic target in the video stream; the lane line and the dynamic target are in the same coordinate system; An overlapping area determination module is used to determine the overlapping area between the millimeter wave radar detection area and the camera detection area according to the horizontal edge and the vertical edge.

10. A target detection device, characterized in that: The device comprises: An acquisition module is used to acquire millimeter-wave radar point cloud images and camera images at the same time and in the same scene; a determination module, configured to determine overlapping point cloud images and non-overlapping point cloud images from the point cloud images according to the overlapping area, and to determine overlapping camera images and non-overlapping camera images from the camera images according to the overlapping area; wherein the overlapping area is obtained according to the method for determining the overlapping area according to any one of claims 1 to 7; A first fusion module is used to fuse the overlapping camera images and the overlapping point cloud images to obtain a fused image, and obtain a fusion detection result according to the fused image; A second fusion module is used to obtain a point cloud detection result according to the non-overlapping point cloud image, and to obtain a camera image detection result according to the non-overlapping camera image; A processing module is used to process the fusion detection result, the point cloud detection result and the camera image detection result according to the position information corresponding to the point cloud image, the fusion image and the camera image to obtain a target detection result.

11. A roadside unit, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented, or when the processor executes the computer program, the steps of the method according to claim 8 are implemented.

12. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented; or when the computer program is executed by a processor, the steps of the method according to claim 8 are implemented.

Images