Target detection method, device, roadside base station and storage medium

Abstract

The present application relates to a target detection method, device, roadside base station and storage medium. The method comprises: receiving two-dimensional target detection data of the area to be detected; the two-dimensional target detection data comprises the position of each two-dimensional target frame in the area to be detected and the confidence of each two-dimensional target frame; the confidence is used to characterize the probability that the target in the two-dimensional target frame belongs to the corresponding type; dividing the point cloud data of the area to be detected according to the preset three-dimensional anchor frame to obtain multiple three-dimensional anchor frames including point cloud data; wherein the initial confidence corresponding to each target in each three-dimensional anchor frame is the same; determining the target detection result according to the position of each two-dimensional target frame in the area to be detected, the position of the multiple three-dimensional anchor frames, the confidence of each two-dimensional target frame and the initial confidence corresponding to each target in the multiple three-dimensional anchor frames. The use of this method can improve the accuracy of target detection using point cloud data.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a target detection method, device, roadside base station and storage medium. Background Art

[0002] Object detection based on computer vision technology is increasingly being used in many fields such as intelligent transportation, sea and land security, etc.

[0003] In the field of intelligent transportation, the traditional target detection method is that road traffic auxiliary equipment performs target detection on the original three-dimensional point cloud data collected by the three-dimensional laser sensor to obtain the three-dimensional detection result of the target.

[0004] LiDAR itself is a ranging sensor that can measure the distance to targets in a scene. However, due to the sparse and low-resolution characteristics of LiDAR point cloud data, the accuracy of target detection results obtained by relying on LiDAR's ranging data is low. Summary of the invention

[0005] Based on this, it is necessary to provide a target detection method, device, roadside base station and storage medium to address the above technical problems.

[0006] A target detection method, comprising:

[0007] Receive two-dimensional target detection data of the area to be detected; wherein the two-dimensional target detection data includes the position of each two-dimensional target frame in the area to be detected and the confidence of each two-dimensional target frame; the confidence is used to characterize the probability that the target in the two-dimensional target frame belongs to the corresponding type;

[0008] Dividing the point cloud data of the area to be measured according to a preset three-dimensional anchor point frame to obtain a plurality of three-dimensional anchor point frames including the point cloud data; wherein the initial confidence corresponding to each target in each three-dimensional anchor point frame is the same;

[0009] The target detection result is determined according to the position of each two-dimensional target frame in the test area, the position of multiple three-dimensional anchor frames, the confidence of each two-dimensional target frame and the initial confidence corresponding to each target in multiple three-dimensional anchor frames.

[0010] In one embodiment, the target detection result is determined according to the position of each two-dimensional target frame in the detection area, the position of multiple three-dimensional anchor frames, the confidence of each two-dimensional target frame, and the initial confidence corresponding to each target in the multiple three-dimensional anchor frames, including:

[0011] The multiple 3D anchor frames are screened according to the positions of each 2D target frame and the positions of the multiple 3D anchor frames in the test area to obtain multiple candidate 3D anchor frames;

[0012] The target detection result is determined based on the confidence of each 2D target box and the initial confidence of multiple candidate 3D anchor boxes.

[0013] In one of the embodiments, multiple 3D anchor frames are screened according to the positions of each 2D target frame in the area to be detected and the positions of multiple 3D anchor frames to obtain multiple candidate 3D anchor frames, including:

[0014] Based on each two-dimensional target frame, obtain the two-dimensional position coordinates of the target in the two-dimensional target frame;

[0015] According to the preset coordinate transformation matrix, the two-dimensional position coordinates are transformed into the three-dimensional coordinate system to obtain the transformed position coordinates of the target;

[0016] According to the transformed position coordinates of the target and the positions of multiple 3D anchor boxes, multiple 3D anchor boxes are screened to obtain candidate 3D anchor boxes corresponding to the 2D target box.

[0017] In one of the embodiments, multiple three-dimensional anchor boxes are screened according to the transformed position coordinates of the target and the positions of multiple three-dimensional anchor boxes to obtain candidate three-dimensional anchor boxes corresponding to the two-dimensional target box, including:

[0018] Get the 3D position coordinates of the target in each 3D anchor point frame;

[0019] Calculate the position distance between the transformed position coordinates of each target and the three-dimensional position coordinates of each target;

[0020] For each target's transformed position coordinates, the 3D anchor point frame with the smallest position distance is obtained from multiple 3D anchor point frames as the candidate 3D anchor point frame corresponding to the 2D target frame.

[0021] In one embodiment, determining the target detection result according to the confidence of each two-dimensional target box and the initial confidence of multiple candidate three-dimensional anchor boxes includes:

[0022] Get the initial confidence of the candidate 3D anchor box corresponding to each 2D target box;

[0023] According to a preset confidence floating strategy, the initial confidence of the candidate 3D anchor point frame corresponding to each 2D target frame is increased to obtain the floating confidence of the candidate 3D anchor point frame corresponding to each 2D target frame;

[0024] The target detection result is determined according to the confidence of each two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame.

[0025] In one embodiment, determining the target detection result according to the confidence of each two-dimensional target frame and the floating confidence of multiple candidate three-dimensional anchor point frames corresponding to each two-dimensional target frame includes:

[0026] For each two-dimensional target frame, the confidence of each two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame are weighted summed to obtain the floating weighted confidence of the candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame;

[0027] The target detection result is determined according to the floating weighted confidence of the candidate anchor box corresponding to each two-dimensional target box and the initial confidence of other non-candidate three-dimensional anchor boxes in multiple three-dimensional anchor boxes.

[0028] In one of the embodiments, the confidence of the two-dimensional target frame includes confidences of multiple different candidate types; the floating confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame includes floating confidences of multiple different candidate types, and the confidence of the two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame include the same candidate type;

[0029] For each two-dimensional target frame, the confidence of each two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame are weighted summed to obtain the floating weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame, including:

[0030] For each two-dimensional target frame, calculate the product of the confidence of each candidate type and the first weight to obtain the first weighted confidence corresponding to each candidate type;

[0031] Calculate the product of the floating confidence of each candidate type and the second weight to obtain the second weighted confidence corresponding to each candidate type; wherein the sum of the first weight and the second weight is 1, and the first weight is greater than the second weight;

[0032] Calculate the sum of the first weighted re-confidence corresponding to each candidate type and the second weighted re-confidence corresponding to the candidate type to obtain the upward weighted confidence corresponding to each candidate type;

[0033] Based on the floating weighted confidence corresponding to each candidate type, the floating weighted confidence of the candidate 3D anchor box corresponding to the 2D target box is determined.

[0034] In one embodiment, determining the target detection result according to the confidence of each two-dimensional target box and the initial confidence of multiple candidate three-dimensional anchor boxes includes:

[0035] For each two-dimensional target frame, the confidence of each two-dimensional target frame and the initial confidence of multiple candidate three-dimensional anchor frames corresponding to each two-dimensional target frame are weighted summed to obtain the initial weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame;

[0036] The target detection result is determined according to the initial weighted confidence of the candidate 3D anchor box corresponding to each 2D target box and the initial confidence of other non-candidate 3D anchor boxes in multiple 3D anchor boxes.

[0037] In one of the embodiments, the confidence of the two-dimensional target frame includes confidences of multiple different candidate types; the initial confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame includes confidences of multiple different candidate types, and the confidence of the two-dimensional target frame and the initial confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame include the same candidate type;

[0038] For each two-dimensional target frame, the confidence of each two-dimensional target frame and the initial confidence of multiple candidate three-dimensional anchor frames corresponding to each two-dimensional target frame are weighted summed to obtain the initial weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame, including:

[0039] For each two-dimensional target frame, calculate the product of the confidence of each candidate type and the third weight to obtain the third-weighted confidence corresponding to each candidate type;

[0040] Calculate the product of the initial confidence of each candidate type and the fourth weight to obtain the fourth weight reset confidence corresponding to each candidate type; wherein the sum of the third weight and the fourth weight is 1, and the third weight is greater than the fourth weight;

[0041] Calculate the sum of the third weighted reset confidence corresponding to each candidate type and the fourth weighted reset confidence corresponding to the candidate type to obtain the initial weighted confidence corresponding to each candidate type;

[0042] Based on the initial weighted confidence corresponding to each candidate type, the initial weighted confidence of the candidate anchor box corresponding to the two-dimensional target box is determined.

[0043] A target detection device, comprising:

[0044] A data receiving module is used to receive two-dimensional target detection data of the area to be detected; wherein the two-dimensional target detection data includes the position of each two-dimensional target frame in the area to be detected and the confidence of each two-dimensional target frame; the confidence is used to characterize the probability that the target in the two-dimensional target frame belongs to the corresponding type;

[0045] An anchor point generation module, used for dividing the point cloud data of the area to be measured according to a preset three-dimensional anchor point frame to obtain a plurality of three-dimensional anchor point frames including the point cloud data; wherein the initial confidence corresponding to each target in each three-dimensional anchor point frame is the same;

[0046] The target determination module is used to determine the target detection result based on the position of each two-dimensional target frame in the test area, the position of multiple three-dimensional anchor frames, the confidence of each two-dimensional target frame, and the initial confidence corresponding to each target in multiple three-dimensional anchor frames.

[0047] A roadside base station includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0048] Receive two-dimensional target detection data of the area to be detected; wherein the two-dimensional target detection data includes the position of each two-dimensional target frame in the area to be detected and the confidence of each two-dimensional target frame; the confidence is used to characterize the probability that the target in the two-dimensional target frame belongs to the corresponding type;

[0049] Dividing the point cloud data of the area to be measured according to a preset three-dimensional anchor point frame to obtain a plurality of three-dimensional anchor point frames including the point cloud data; wherein the initial confidence corresponding to each target in each three-dimensional anchor point frame is the same;

[0050] The target detection result is determined according to the position of each two-dimensional target frame in the test area, the position of multiple three-dimensional anchor frames, the confidence of each two-dimensional target frame and the initial confidence corresponding to each target in multiple three-dimensional anchor frames.

[0051] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0052] Receive two-dimensional target detection data of the area to be detected; wherein the two-dimensional target detection data includes the position of each two-dimensional target frame in the area to be detected and the confidence of each two-dimensional target frame; the confidence is used to characterize the probability that the target in the two-dimensional target frame belongs to the corresponding type;

[0053] Dividing the point cloud data of the area to be measured according to a preset three-dimensional anchor point frame to obtain a plurality of three-dimensional anchor point frames including the point cloud data; wherein the initial confidence corresponding to each target in each three-dimensional anchor point frame is the same;

[0054] The target detection result is determined according to the position of each two-dimensional target frame in the test area, the position of multiple three-dimensional anchor frames, the confidence of each two-dimensional target frame and the initial confidence corresponding to each target in multiple three-dimensional anchor frames.

[0055] The above-mentioned target detection method, device, roadside base station and storage medium divide the point cloud data of the area to be tested by a preset three-dimensional anchor point frame, obtain multiple three-dimensional anchor point frames with the same initial confidence, including point cloud data, and perform subsequent target detection on the point cloud data in each three-dimensional anchor point frame in combination with the initial confidence and position of the three-dimensional anchor point frame and the received two-dimensional target detection data of the area to be tested, to obtain the final target detection result. The two-dimensional image can accurately and intuitively reflect the image features of the target in the area to be tested. The two-dimensional target detection data obtained based on the two-dimensional image detection has high accuracy. In the process of target detection based on the point cloud data, the two-dimensional target detection data is combined to guide the target detection process of the point cloud data, so as to improve the accuracy of target detection using point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of an application scenario of a target detection method in an embodiment;

[0057] Figure 2 Schematic diagram of a target detection method in one embodiment;

[0058] Figure 3 is a plan view of three-dimensional anchor point boxes of different preset sizes in one embodiment;

[0059] Figure 4 A schematic diagram of a process for determining a target detection result in one embodiment;

[0060] Figure 5 A schematic diagram of a process of screening and obtaining candidate three-dimensional anchor point boxes in one embodiment;

[0061] Figure 6 A schematic diagram of a process of obtaining a candidate three-dimensional anchor point frame by position distance screening in one embodiment;

[0062] Figure 7 A schematic diagram of a process for determining a target detection result in another embodiment;

[0063] Figure 8 A schematic diagram of a flow chart of obtaining a floating weighted confidence by weighted summation in an embodiment;

[0064] Fig. 9 A schematic diagram of a process for determining a target detection result in another embodiment;

[0065] Fig.10 A schematic diagram of a process of obtaining an initial weighted confidence by weighted summation in one embodiment;

[0066] Fig.11 is a structural block diagram of a target detection device in one embodiment;

[0067] Fig.12 FIG. 4 is a diagram showing the internal structure of a roadside base station in an embodiment. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0069] The target detection method provided in this application can be applied to Figure 1 In the system composed of the roadside base station and each target object in the road scene shown, a laser radar and an image acquisition device are integrated on the roadside base station. The image acquisition device collects a two-dimensional image of the area to be measured in the road scene through an image sensor, and performs target detection on the two-dimensional image to obtain two-dimensional target detection data, and sends the two-dimensional target detection data to the data processing center of the roadside base station. The data processing center of the roadside base station divides the point cloud data of the area to be measured collected by the laser radar according to the preset three-dimensional anchor point frame to obtain multiple three-dimensional anchor point frames including point cloud data. The data processing center of the roadside base station then determines the target detection result based on the position of the two-dimensional target frame in the two-dimensional target detection data and the confidence of the two-dimensional target frame, the position of multiple three-dimensional anchor point frames, and the initial confidence of multiple three-dimensional anchor point frames. Among them, the image sensor for collecting two-dimensional images and the laser radar sensor for collecting point cloud data can be integrated into the roadside base station in the form of a single sensor or a sensor cluster.

[0070] In one embodiment, Figure 2 As shown, a target detection method is provided, which is applied to Figure 1 The roadside base station in FIG. 1 is taken as an example to illustrate the method, which includes the following steps:

[0071] S210: Receive two-dimensional target detection data of the area to be detected.

[0072] The two-dimensional target detection data includes the position of each two-dimensional target frame in the detection area and the confidence of each two-dimensional target frame. The confidence is used to characterize the probability that the target in the two-dimensional target frame belongs to the corresponding type. For example, a two-dimensional target frame includes a target, and the confidence p of the target may include p 人 =20%, p 车 =30%, p 非机 =50%, p 人 = 20% indicates that the probability that the target belongs to a human is 20%, p 车 =30% means the probability that the target belongs to the car is 30%, p 非机 =50% indicates that the probability that the target is a non-motor vehicle is 20%.

[0073] Specifically, the image acquisition device integrated in the roadside base station acquires two-dimensional images of the area to be measured, and performs target detection based on the acquired two-dimensional images to obtain two-dimensional target detection data, and sends the obtained two-dimensional target detection data to the roadside base station. The roadside base station receives the two-dimensional target detection data obtained by the image acquisition device, such as the position of each two-dimensional target frame in the area to be measured and the confidence of each two-dimensional target frame.

[0074] S220 , dividing the point cloud data of the area to be measured according to a preset three-dimensional anchor point frame to obtain a plurality of three-dimensional anchor point frames including the point cloud data.

[0075] Among them, the initial confidence corresponding to each target in each 3D anchor box is the same.

[0076] Specifically, the roadside base station obtains the point cloud data of the area to be measured, and uses a three-dimensional anchor frame of a preset size to divide the obtained point cloud data in a sliding window form, until all the point cloud data of the entire area to be measured are divided into the three-dimensional anchor frame, and multiple anchor frames including point cloud data are obtained. For example, any position in the area to be measured is used as the center point of a three-dimensional anchor frame of a preset size to generate a three-dimensional anchor frame of a preset size, and the entire three-dimensional anchor frame is slid along a certain direction, such as a horizontal direction, by a preset distance to obtain a new three-dimensional anchor frame, and so on, to obtain multiple three-dimensional anchor frames including point cloud data. Among them, the types of targets in three-dimensional anchor frames of different sizes are different. When the obtained point cloud data is divided by three-dimensional anchor frames of multiple preset sizes, the same center point is used to generate three-dimensional anchor frames of multiple preset sizes. Figure 3 As shown, in the point cloud image composed of point cloud data, any position O is used as the center point of the 3D anchor box of preset size, a 2*2*2 sized 3D anchor box A is generated, a 4*3*4 sized 3D anchor box B is generated, and a 3*5*3 sized 3D anchor box C is generated, and all 3D anchor boxes are given the same confidence as the initial confidence. For example, the initial confidence can be 10%, then the initial confidence of the 3D anchor box A is used to characterize the probability that the target in the 3D anchor box A belongs to a person is 10%, the initial confidence of the 3D anchor box B is used to characterize the probability that the target in the 3D anchor box B belongs to a car is 10%, and the initial confidence of the 3D anchor box C is used to characterize the probability that the target in the 3D anchor box C belongs to a non-motor vehicle is 10%.

[0077] S230, determining the target detection result according to the position of each two-dimensional target frame in the detected area, the positions of multiple three-dimensional anchor frames, the confidence of each two-dimensional target frame, and the initial confidence corresponding to each target in the multiple three-dimensional anchor frames.

[0078] Specifically, the roadside base station determines the candidate three-dimensional anchor point frame in the multiple three-dimensional anchor point frames according to the positions of each two-dimensional target frame and the positions of the multiple three-dimensional anchor point frames. Among them, the candidate three-dimensional anchor point frame and the two-dimensional target frame have a one-to-one correspondence relationship in position. For example, the roadside base station converts the position of the three-dimensional anchor point frame using the three-dimensional coordinate system to the two-dimensional coordinate system used by the position of the two-dimensional target frame according to the preset coordinate conversion matrix, and obtains the position of the three-dimensional anchor point frame in the two-dimensional coordinate system. The candidate three-dimensional anchor point frame is determined according to the degree of matching between the position of the three-dimensional anchor point frame in the two-dimensional coordinate system and the position of the two-dimensional target frame, and the three-dimensional anchor point frame with a matching degree greater than the preset degree can be used as the candidate three-dimensional anchor point frame. The roadside base station can sum the confidence of the two-dimensional target frame and the initial confidence of the candidate three-dimensional anchor point frame to determine the confidence of the candidate three-dimensional anchor point frame, and then based on the confidence of the candidate three-dimensional anchor point frame and the initial confidence of other non-candidate three-dimensional anchor point frames, that is, based on the confidence of all three-dimensional anchor point frames, further feature extraction is performed on the point cloud data in each three-dimensional anchor point frame, and subsequent target detection is performed to obtain the target detection result.

[0079] Optionally, the above-mentioned processes of S220 to S230 can be implemented using a point cloud detection model. The point cloud detection model is a deep learning network model for three-dimensional target detection. In this embodiment, when the roadside base station performs target detection on the point cloud data of the area to be tested, it uses a preset three-dimensional anchor point frame to divide the point cloud data of the area to be tested, obtains multiple three-dimensional anchor point frames with the same initial confidence, and combines the initial confidence and position of the three-dimensional anchor point frame and the received two-dimensional target detection data of the area to be tested to perform subsequent target detection on the point cloud data in each three-dimensional anchor point frame to obtain the final target detection result. The two-dimensional image can accurately and intuitively reflect the image features of the target in the area to be tested. The two-dimensional target detection data obtained based on the two-dimensional image detection has high accuracy. In the process of performing target detection based on the point cloud data, the two-dimensional target detection data is combined to guide the target detection process of the point cloud data to improve the accuracy of target detection using point cloud data.

[0080] In one embodiment, in order to improve the efficiency of target detection using point cloud data, Figure 4 As shown, S230 includes:

[0081] S410 , screening multiple 3D anchor frames according to the positions of each 2D target frame in the area to be measured and the positions of multiple 3D anchor frames to obtain multiple candidate 3D anchor frames.

[0082] Among them, when the roadside base station screens the candidate 3D anchor point frames, it is necessary to first align the coordinate systems of the 2D target frame and the 3D anchor point frame.

[0083] Specifically, the roadside base station can convert the position of the two-dimensional target frame into a three-dimensional coordinate system to unify the coordinate system, and then screen multiple three-dimensional anchor frames based on the converted positions of each two-dimensional target frame in the test area and the positions of multiple three-dimensional anchor frames to obtain multiple candidate three-dimensional anchor frames.

[0084] S420: Determine the target detection result according to the confidence of each two-dimensional target frame and the initial confidence of multiple candidate three-dimensional anchor point frames.

[0085] Specifically, the roadside base station corrects the initial confidence of the candidate 3D anchor point frame corresponding to the location of the target according to the confidence of the 2D target frame, such as replacing the initial confidence of the 3D anchor point frame that is smaller than the confidence of the 2D target frame with the confidence of the 2D target frame, and performs subsequent target detection on the point cloud data in the candidate 3D anchor point frame based on the corrected initial confidence of the candidate 3D anchor point frame to determine the target detection result. For example, the confidence of the 2D target frame includes p 人 =5%, p 车 =25%, p 非机 =58%, the initial confidence p' of the corresponding candidate 3D anchor box 人 =10%, p' 车 =10%, p' 非机 =10%, then p 车 =25%, p 非机 = 58% replace p' 车 =10%, p' 非机 = 10%, and the initial confidence of the corrected candidate 3D anchor box includes p" 人 =10%, p" 车 =25%, p" 非机 =58%.

[0086] In this embodiment, the roadside base station corrects the initial confidence of the screened candidate three-dimensional anchor point frame through the confidence of the two-dimensional target frame, and performs subsequent target detection on the point cloud data in the candidate three-dimensional anchor point frame based on the corrected initial confidence of the candidate three-dimensional anchor point frame. The target detection process of the point cloud data is guided based on the correction of the confidence, which improves the accuracy of target detection using point cloud data and the efficiency of target detection.

[0087] In one embodiment, the position of the two-dimensional target frame can be converted to a three-dimensional coordinate system, and a candidate three-dimensional anchor point frame can be obtained by screening from multiple three-dimensional anchor point frames based on the position distance between the converted position coordinates and the three-dimensional position coordinates of the target in the three-dimensional anchor point frame. Figure 5 As shown, S410 includes:

[0088] S510: Based on each two-dimensional target frame, obtain the two-dimensional position coordinates of the target in the two-dimensional target frame.

[0089] Specifically, the roadside base station obtains the geometric center point position of each two-dimensional target frame as the two-dimensional position coordinates of the target.

[0090] S520 . According to a preset coordinate conversion matrix, the two-dimensional position coordinates are converted into a three-dimensional coordinate system to obtain the converted position coordinates of the target.

[0091] The coordinate conversion matrix is ​​a spatial conversion relationship between the two coordinate systems obtained by pre-calibrating the two-dimensional coordinate system used by the image acquisition device and the three-dimensional coordinate system used by the roadside base station.

[0092] Specifically, the roadside base station converts the two-dimensional position coordinates of the target to the three-dimensional coordinate system used by the roadside base station according to a preset coordinate conversion matrix, and obtains the converted position coordinates of the corresponding target in the three-dimensional coordinate system.

[0093] S530 , screening multiple three-dimensional anchor frames according to the converted position coordinates of the target and the positions of the multiple three-dimensional anchor frames to obtain candidate three-dimensional anchor frames corresponding to the two-dimensional target frame.

[0094] Specifically, the roadside base station may screen multiple three-dimensional anchor frames according to the position distance between the converted position coordinates of the target and the positions of the multiple anchor frames, and obtain candidate three-dimensional anchor frames corresponding to the two-dimensional target frame.

[0095] Among them, Figure 6 As shown, the process of filtering based on location distance includes:

[0096] S610: Obtain the three-dimensional position coordinates of the target in each three-dimensional anchor point frame.

[0097] Specifically, the roadside base station obtains the geometric center point position of each three-dimensional anchor point frame as the three-dimensional position coordinates of the target.

[0098] S620: Calculate the position distance between the converted position coordinates of each target and the three-dimensional position coordinates of each target.

[0099] S630 . For each target's converted position coordinates, obtain a three-dimensional anchor point frame with the smallest position distance from multiple three-dimensional anchor point frames as a candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame.

[0100] Specifically, the roadside base station calculates the position distance between the converted position coordinates of each target and the three-dimensional position coordinates of each target, and selects the three-dimensional anchor point frame with the smallest position distance for the converted position coordinates of each target as the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame. The candidate three-dimensional anchor point frame may include multiple three-dimensional anchor point frames with the same center point.

[0101] Optionally, the roadside base station may also use the intersection over union (IOU) in target detection to screen multiple 3D anchor boxes to obtain candidate 3D anchor boxes corresponding to the 2D target box. The specific process is as follows:

[0102] The roadside base station converts the obtained two-dimensional target frame to the three-dimensional coordinate system used by the roadside base station according to the preset coordinate conversion matrix, generates a converted three-dimensional target frame, and obtains the conversion position coordinates of each converted three-dimensional target frame in the three-dimensional coordinate system. For each converted three-dimensional target frame, obtain the overlapping area volume V of the three-dimensional anchor point frame with overlapping area and the converted three-dimensional target frame 重叠 , and the difference V between the sum of the volumes of the 3D anchor box and the transformed 3D target box with the overlapping area and the volume of the overlapping area 差 , calculate V 重叠 / V 差 , obtain the intersection-and-union ratio of the 3D anchor box with overlapping area and the converted 3D target box. Select the 3D anchor box with the largest intersection-and-union ratio as the candidate 3D anchor box corresponding to the 2D target box.

[0103] Optionally, the converted 3D target frame and the 3D anchor frame can be projected into the same plane, and the intersection and union ratio can be calculated by area to reduce the amount of calculation and improve efficiency. Where, the intersection and union ratio = the overlapping area S of the projected 3D anchor frame and the converted 3D target frame 重叠 / The difference S between the sum of the areas of the projected 3D anchor box and the transformed 3D target box and the area of ​​the overlapping area 差 In this embodiment, the roadside base station uses a preset coordinate conversion matrix to convert the two-dimensional position coordinates of the target to the three-dimensional coordinate system used by the roadside base station, and obtains the converted position coordinates of the corresponding target in the three-dimensional coordinate system. The candidate three-dimensional anchor point frame is determined according to the position distance between the converted position coordinates of each target and the three-dimensional position coordinates of the target, and the three-dimensional anchor point frame with the smallest position distance for each target is used as the candidate three-dimensional anchor point frame, so as to realize the screening of the three-dimensional anchor point frame to ensure that the point cloud data is not lost, thereby improving the accuracy of target detection using point cloud data.

[0104] In one embodiment, in order to improve the influence of two-dimensional target detection data on the final target detection result, Figure 7 As shown, S420 includes:

[0105] S710: Obtain the initial confidence of the candidate 3D anchor point box corresponding to each 2D target box.

[0106] S720. According to a preset confidence increasing strategy, increase the initial confidence of the candidate 3D anchor point frame corresponding to each 2D target frame to obtain the increased confidence of the candidate 3D anchor point frame corresponding to each 2D target frame.

[0107] The preset confidence increasing strategy may be to increase the confidence by a percentage of the confidence itself, or to increase the confidence by a preset confidence value, or to directly increase the confidence value to a preset confidence value.

[0108] Specifically, the roadside base station increases the initial confidence of the candidate 3D anchor point frame corresponding to each 2D target frame according to a preset confidence floating strategy, such as increasing its own confidence by 50%, to obtain the floating confidence of the candidate 3D anchor point frame corresponding to each 2D target frame. For example, the initial confidence of the candidate 3D anchor point frame corresponding to each 2D target frame is 20%, and the roadside base station increases the initial confidence of the candidate 3D anchor point frame corresponding to each 2D target frame by 10% according to the preset confidence floating strategy of increasing its own confidence by 50%, and obtains the floating confidence of the candidate 3D anchor point frame corresponding to each 2D target frame, that is, 30%.

[0109] S730 , determining a target detection result according to the confidence of each two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame.

[0110] Specifically, for each two-dimensional target frame, the roadside base station performs weighted summation of the confidence of each two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame to obtain the floating weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame. The roadside base station then performs subsequent target detection based on the floating weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame and the initial confidence of other non-candidate three-dimensional anchor frames in multiple three-dimensional anchor frames to determine the final target detection result.

[0111] Optionally, the roadside base station may also adopt a preset confidence reduction strategy to reduce the initial confidence of other non-candidate three-dimensional anchor point frames in multiple three-dimensional anchor point frames, and obtain the reduced confidence of the candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame. Among them, the preset confidence reduction strategy can be to reduce the confidence of its own confidence percentage, or to reduce the preset confidence value, or directly reduce to the preset confidence value. For each two-dimensional target frame, the roadside base station performs weighted summation of the confidence of each two-dimensional target frame and the initial confidence of the candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame to obtain the initial weighted confidence of the candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame. The roadside base station then performs subsequent target detection based on the initial weighted confidence of the candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame and the reduced confidence of other non-candidate three-dimensional anchor point frames in multiple three-dimensional anchor point frames to determine the final target detection result.

[0112] The confidence of the two-dimensional target frame includes confidences of multiple different candidate types; the floating confidence of the candidate three-dimensional anchor frame corresponding to the two-dimensional target frame includes floating confidences of multiple different candidate types, and the confidence of the two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor frame corresponding to the two-dimensional target frame include the same candidate types. Figure 8 As shown, the process of obtaining the floating weighted confidence by the weighted summation includes:

[0113] S810: For each two-dimensional target frame, calculate the product of the confidence of each candidate type and the first weight to obtain the first weighted confidence corresponding to each candidate type.

[0114] S820: Calculate the product of the floating confidence of each candidate type and the second weight to obtain the second weighted confidence corresponding to each candidate type.

[0115] The sum of the first weight and the second weight is 1, and the first weight is greater than the second weight.

[0116] Specifically, the roadside base station calculates the product of the confidence of each candidate type and the first weight for each two-dimensional target frame, and obtains the first weighted confidence corresponding to each candidate type. For example, the candidate types corresponding to each two-dimensional target frame are people, cars, and non-motor vehicles, and the obtained confidences are P and P, respectively. 人 =80%, P 车 =45%, P 非机 =67%. The first weight T1 = 2 / 3, then the first weighted confidence C1 includes C 人1 ,C 车1 and C 非机1 Among them, C 人1 =P 人 *T1=80%*2 / 3=53.33%, C 车1 =P车 *T1=45%*2 / 3=30%,C 非机1 =P 非机 *T1=67%*2 / 3=44.67%. The candidate types corresponding to the floating confidence are also people, cars and non-motor vehicles. The floating confidences Q are Q 人 =30%, Q 车 =30%, Q 非机 =30%. The second weight T2 = 1 / 3, then the second weighted confidence C2 includes C 人2 ,C 车2 and C 非机2 Among them, C 人2 =Q 人 *T2=30%*1 / 3=10%,C 车2 =Q 车 *T2=30%*1 / 3=10%,C 非机1 =Q 非机 *T1=30%*1 / 3=10%.

[0117] S830: Calculate the sum of the first weighted re-confidence corresponding to each candidate type and the second weighted re-confidence corresponding to the candidate type to obtain an upward weighted confidence corresponding to each candidate type.

[0118] S840: Determine the floating weighted confidence of the candidate 3D anchor point box corresponding to the 2D target box based on the floating weighted confidence corresponding to each candidate type.

[0119] Specifically, the roadside base station calculates the sum of the first weighted confidence corresponding to each candidate type and the second weighted confidence corresponding to the candidate type to obtain the floating weighted confidence corresponding to each candidate type. For example, the floating weighted confidence C of the candidate type being human is 人 =C 人1 +C 人2 =53.33%+10%=63.33%, the floating weighted confidence C of the candidate type being car 车 =C 车1 +C 车2 =30%+10%=40%, the floating weighted confidence C of the candidate type being a non-motor vehicle 非机 =C 非机1 +C 非机2 =44.67%+10%=54.67%. The roadside base station uses the floating weighted confidence corresponding to each candidate type as the floating weighted confidence of the candidate 3D anchor point frame corresponding to the 2D target frame, that is, C 人 , C 车 and C 非机 As the floating weighted confidence of the candidate 3D anchor box corresponding to the 2D target box.

[0120] In this embodiment, the roadside base station increases the initial confidence of the candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame according to a preset confidence floating strategy to obtain the floating confidence of the candidate three-dimensional anchor point frame to improve the influence of the two-dimensional target detection data on the final target detection result, assign a larger weight to the confidence of the two-dimensional target frame, and assign a smaller weight to the floating confidence of the candidate three-dimensional anchor point frame, and then perform weighted summation of the same candidate type to further improve the influence of the two-dimensional target detection data on the final target detection result. The two-dimensional target detection data obtained based on two-dimensional image detection has high accuracy, which improves the influence of the two-dimensional target detection data on the final target detection result, thereby improving the accuracy of the final target detection result obtained by using point cloud data for target detection.

[0121] In one embodiment, the confidence of the 2D target frame of the same candidate type and the initial confidence of the candidate 3D anchor point frame corresponding to the 2D target can be directly weighted and summed to increase the influence of the 2D target detection data on the final target detection result. Fig. 9 As shown, S420 includes:

[0122] S910. For each two-dimensional target frame, perform weighted summation on the confidence of each two-dimensional target frame and the initial confidences of multiple candidate three-dimensional anchor frames corresponding to each two-dimensional target frame to obtain an initial weighted confidence of the candidate anchor frames corresponding to each two-dimensional target frame.

[0123] S920: Determine a target detection result according to an initial weighted confidence of a candidate 3D anchor point frame corresponding to each 2D target frame and initial confidences of other non-candidate 3D anchor point frames in the multiple 3D anchor point frames.

[0124] Specifically, for each 2D target frame, the roadside base station performs weighted summation of the confidence of each 2D target frame and the initial confidence of the candidate 3D anchor frame corresponding to each 2D target frame to obtain the initial weighted confidence of the candidate 3D anchor frame corresponding to each 2D target frame. The roadside base station then performs subsequent target detection based on the initial weighted confidence of the candidate 3D anchor frame corresponding to each 2D target frame and the initial confidence of other non-candidate 3D anchor frames in multiple 3D anchor frames to determine the final target detection result.

[0125] The confidence of the two-dimensional target frame includes confidences of multiple different candidate types; the initial confidence of the candidate three-dimensional anchor frame corresponding to the two-dimensional target frame includes confidences of multiple different candidate types, and the confidence of the two-dimensional target frame and the initial confidence of the candidate three-dimensional anchor frame corresponding to the two-dimensional target frame contain the same candidate types. Fig.10 As shown, the process of obtaining the initial weighted confidence by the weighted summation includes:

[0126] S1010. For each two-dimensional target frame, calculate the product of the confidence of each candidate type and the third weight to obtain the third-weighted confidence corresponding to each candidate type.

[0127] S1020. Calculate the product of the initial confidence of each candidate type and the fourth weight to obtain the fourth weight reset confidence corresponding to each candidate type.

[0128] The sum of the third weight and the fourth weight is 1, and the third weight is greater than the fourth weight.

[0129] S1030. Calculate the sum of the third weighted reset confidence corresponding to each candidate type and the fourth weighted reset confidence corresponding to the candidate type to obtain an initial weighted confidence corresponding to each candidate type.

[0130] S1040: Determine an initial weighted confidence of a candidate 3D anchor box corresponding to the 2D target box based on the initial weighted confidence corresponding to each candidate type.

[0131] The specific process is similar to the above S810 to S840, except that the floating confidence is replaced by the initial confidence, and the floating weighted confidence is replaced by the initial weighted confidence, which will not be described in detail here.

[0132] In this embodiment, the roadside base station assigns a larger weight to the confidence of the two-dimensional target box and a smaller weight to the initial confidence of the candidate three-dimensional anchor point box, and then performs weighted summation on the same candidate type to increase the influence of the two-dimensional target detection data on the final target detection result. The two-dimensional target detection data obtained based on two-dimensional image detection has high accuracy, which increases the influence of the two-dimensional target detection data on the final target detection result, thereby improving the accuracy of the final target detection result obtained by using point cloud data for target detection.

[0133] It should be understood that although Figure 1-10 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-10 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0134] In one embodiment, Fig.11As shown, a target detection device is provided, including: a data receiving module 1101, an anchor point generating module 1102 and a target determining module 1103, wherein:

[0135] The data receiving module 1101 is used to receive two-dimensional target detection data of the area to be detected; wherein the two-dimensional target detection data includes the position of each two-dimensional target frame in the area to be detected and the confidence of each two-dimensional target frame; the confidence is used to characterize the probability that the target in the two-dimensional target frame belongs to the corresponding type;

[0136] The anchor point generation module 1102 is used to divide the point cloud data of the area to be measured according to the preset three-dimensional anchor point frame to obtain multiple three-dimensional anchor point frames including the point cloud data; wherein the initial confidence corresponding to each target in each three-dimensional anchor point frame is the same;

[0137] The target determination module 1103 is used to determine the target detection result according to the position of each two-dimensional target frame in the detection area, the position of multiple three-dimensional anchor frames, the confidence of each two-dimensional target frame and the initial confidence corresponding to each target in multiple three-dimensional anchor frames.

[0138] In one embodiment, the target determination module 1103 is specifically used to:

[0139] According to the positions of each two-dimensional target frame and the positions of multiple three-dimensional anchor frames in the test area, multiple three-dimensional anchor frames are screened to obtain multiple candidate three-dimensional anchor frames; the target detection result is determined according to the confidence of each two-dimensional target frame and the initial confidence of multiple candidate three-dimensional anchor frames.

[0140] In one embodiment, the target determination module 1103 is specifically used to:

[0141] Based on each two-dimensional target frame, the two-dimensional position coordinates of the target in the two-dimensional target frame are obtained; according to a preset coordinate transformation matrix, the two-dimensional position coordinates are transformed into a three-dimensional coordinate system to obtain the transformed position coordinates of the target; according to the transformed position coordinates of the target and the positions of multiple three-dimensional anchor frames, multiple three-dimensional anchor frames are screened to obtain candidate three-dimensional anchor frames corresponding to the two-dimensional target frame.

[0142] In one embodiment, the target determination module 1103 is specifically used to:

[0143] Obtain the three-dimensional position coordinates of the target in each three-dimensional anchor point frame; calculate the position distance between the converted position coordinates of each target and the three-dimensional position coordinates of each target; for the converted position coordinates of each target, obtain the three-dimensional anchor point frame with the smallest position distance from multiple three-dimensional anchor point frames as the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame.

[0144] In one embodiment, the target determination module 1103 is specifically used to:

[0145] Obtain the initial confidence of the candidate 3D anchor point frame corresponding to each 2D target frame; increase the initial confidence of the candidate 3D anchor point frame corresponding to each 2D target frame according to the preset confidence floating strategy to obtain the floating confidence of the candidate 3D anchor point frame corresponding to each 2D target frame; determine the target detection result according to the confidence of each 2D target frame and the floating confidence of the candidate 3D anchor point frame corresponding to each 2D target frame.

[0146] In one embodiment, the target determination module 1103 is specifically used to:

[0147] For each two-dimensional target frame, the confidence of each two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame are weighted and summed to obtain the floating weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame; the target detection result is determined according to the floating weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame and the initial confidence of other non-candidate three-dimensional anchor frames in multiple three-dimensional anchor frames.

[0148] In one embodiment, the confidence of the two-dimensional target frame includes confidences of multiple different candidate types; the floating confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame includes floating confidences of multiple different candidate types, and the confidence of the two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame include the same candidate type; the target determination module 1103 is specifically used to:

[0149] For each two-dimensional target frame, calculate the product of the confidence of each candidate type and the first weight to obtain the first-weighted reset confidence corresponding to each candidate type; calculate the product of the floating confidence of each candidate type and the second weight to obtain the second-weighted reset confidence corresponding to each candidate type; wherein the sum of the first weight and the second weight is 1, and the first weight is greater than the second weight; calculate the sum of the first-weighted reset confidence corresponding to each candidate type and the second-weighted reset confidence corresponding to the candidate type to obtain the floating weighted confidence corresponding to each candidate type; based on the floating weighted confidence corresponding to each candidate type, determine the floating weighted confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame.

[0150] In one embodiment, the target determination module 1103 is specifically used to:

[0151] For each two-dimensional target frame, the confidence of each two-dimensional target frame and the initial confidence of multiple candidate three-dimensional anchor frames corresponding to each two-dimensional target frame are weighted and summed to obtain the initial weighted confidence of the candidate three-dimensional anchor frames corresponding to each two-dimensional target frame; the target detection result is determined based on the initial weighted confidence of the candidate three-dimensional anchor frames corresponding to each two-dimensional target frame and the initial confidence of other non-candidate three-dimensional anchor frames in the multiple three-dimensional anchor frames.

[0152] In one embodiment, the confidence of the two-dimensional target frame includes confidences of multiple different candidate types; the initial confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame includes confidences of multiple different candidate types, and the confidence of the two-dimensional target frame and the initial confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame include the same candidate type; the target determination module 1103 is specifically used to:

[0153] For each two-dimensional target frame, calculate the product of the confidence of each candidate type and the third weight to obtain the third-weighted reset confidence corresponding to each candidate type; calculate the product of the initial confidence of each candidate type and the fourth weight to obtain the fourth-weighted reset confidence corresponding to each candidate type; wherein the sum of the third weight and the fourth weight is 1, and the third weight is greater than the fourth weight; calculate the sum of the third-weighted reset confidence corresponding to each candidate type and the fourth-weighted reset confidence corresponding to the candidate type to obtain the initial weighted confidence corresponding to each candidate type; based on the initial weighted confidence corresponding to each candidate type, determine the initial weighted confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame.

[0154] For the specific definition of the target detection device, please refer to the definition of the target detection method above, which will not be repeated here. Each module in the above target detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0155] In one embodiment, a roadside base station is provided, and its internal structure diagram can be as follows: Fig.12As shown. The roadside base station includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the roadside base station is used to provide computing and control capabilities. The memory of the roadside base station includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the roadside base station is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a target detection method is implemented. The display screen of the roadside base station can be a liquid crystal display screen or an electronic ink display screen, and the input device of the roadside base station can be a touch layer covering the display screen, or a button, trackball or touchpad set on the housing of the roadside base station, or an external keyboard, touchpad or mouse, etc.

[0156] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the roadside base station to which the scheme of the present application is applied. The specific roadside base station may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0157] In one embodiment, a roadside base station is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0158] Receive two-dimensional target detection data of the area to be measured; wherein the two-dimensional target detection data includes the position of each two-dimensional target frame in the area to be measured and the confidence of each two-dimensional target frame; the confidence is used to characterize the probability that the target in the two-dimensional target frame belongs to the corresponding type; divide the point cloud data of the area to be measured according to the preset three-dimensional anchor frame to obtain multiple three-dimensional anchor frames including point cloud data; wherein the initial confidence corresponding to each target in each three-dimensional anchor frame is the same; determine the target detection result according to the position of each two-dimensional target frame in the area to be measured, the position of multiple three-dimensional anchor frames, the confidence of each two-dimensional target frame and the initial confidence corresponding to each target in multiple three-dimensional anchor frames.

[0159] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0160] According to the positions of each two-dimensional target frame and the positions of multiple three-dimensional anchor frames in the test area, multiple three-dimensional anchor frames are screened to obtain multiple candidate three-dimensional anchor frames; the target detection result is determined according to the confidence of each two-dimensional target frame and the initial confidence of multiple candidate three-dimensional anchor frames.

[0161] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0162] Based on each two-dimensional target frame, the two-dimensional position coordinates of the target in the two-dimensional target frame are obtained; according to a preset coordinate transformation matrix, the two-dimensional position coordinates are transformed into a three-dimensional coordinate system to obtain the transformed position coordinates of the target; according to the transformed position coordinates of the target and the positions of multiple three-dimensional anchor frames, multiple three-dimensional anchor frames are screened to obtain candidate three-dimensional anchor frames corresponding to the two-dimensional target frame.

[0163] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0164] Obtain the three-dimensional position coordinates of the target in each three-dimensional anchor point frame; calculate the position distance between the converted position coordinates of each target and the three-dimensional position coordinates of each target; for the converted position coordinates of each target, obtain the three-dimensional anchor point frame with the smallest position distance from multiple three-dimensional anchor point frames as the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame.

[0165] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0166] Obtain the initial confidence of the candidate 3D anchor point frame corresponding to each 2D target frame; increase the initial confidence of the candidate 3D anchor point frame corresponding to each 2D target frame according to the preset confidence floating strategy to obtain the floating confidence of the candidate 3D anchor point frame corresponding to each 2D target frame; determine the target detection result according to the confidence of each 2D target frame and the floating confidence of the candidate 3D anchor point frame corresponding to each 2D target frame.

[0167] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0168] For each two-dimensional target frame, the confidence of each two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame are weighted and summed to obtain the floating weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame; the target detection result is determined according to the floating weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame and the initial confidence of other non-candidate three-dimensional anchor frames in multiple three-dimensional anchor frames.

[0169] In one embodiment, the confidence of the two-dimensional target frame includes confidences of multiple different candidate types; the floating confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame includes floating confidences of multiple different candidate types, and the confidence of the two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame include the same candidate type; when the processor executes the computer program, the following steps are further implemented:

[0170] For each two-dimensional target frame, calculate the product of the confidence of each candidate type and the first weight to obtain the first-weighted reset confidence corresponding to each candidate type; calculate the product of the floating confidence of each candidate type and the second weight to obtain the second-weighted reset confidence corresponding to each candidate type; wherein the sum of the first weight and the second weight is 1, and the first weight is greater than the second weight; calculate the sum of the first-weighted reset confidence corresponding to each candidate type and the second-weighted reset confidence corresponding to the candidate type to obtain the floating weighted confidence corresponding to each candidate type; based on the floating weighted confidence corresponding to each candidate type, determine the floating weighted confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame.

[0171] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0172] For each two-dimensional target frame, the confidence of each two-dimensional target frame and the initial confidence of multiple candidate three-dimensional anchor frames corresponding to each two-dimensional target frame are weighted and summed to obtain the initial weighted confidence of the candidate anchor frame corresponding to each two-dimensional target frame; the target detection result is determined based on the initial weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame and the initial confidence of other non-candidate three-dimensional anchor frames in the multiple three-dimensional anchor frames.

[0173] In one embodiment, the confidence of the two-dimensional target frame includes confidences of multiple different candidate types; the initial confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame includes confidences of multiple different candidate types, and the confidence of the two-dimensional target frame and the initial confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame include the same candidate type; when the processor executes the computer program, the following steps are further implemented:

[0174] For each two-dimensional target frame, calculate the product of the confidence of each candidate type and the third weight to obtain the third-weighted reset confidence corresponding to each candidate type; calculate the product of the initial confidence of each candidate type and the fourth weight to obtain the fourth-weighted reset confidence corresponding to each candidate type; wherein the sum of the third weight and the fourth weight is 1, and the third weight is greater than the fourth weight; calculate the sum of the third-weighted reset confidence corresponding to each candidate type and the fourth-weighted reset confidence corresponding to the candidate type to obtain the initial weighted confidence corresponding to each candidate type; based on the initial weighted confidence corresponding to each candidate type, determine the initial weighted confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame.

[0175] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0176] Receive two-dimensional target detection data of the area to be measured; wherein the two-dimensional target detection data includes the position of each two-dimensional target frame in the area to be measured and the confidence of each two-dimensional target frame; the confidence is used to characterize the probability that the target in the two-dimensional target frame belongs to the corresponding type; divide the point cloud data of the area to be measured according to the preset three-dimensional anchor frame to obtain multiple three-dimensional anchor frames including point cloud data; wherein the initial confidence corresponding to each target in each three-dimensional anchor frame is the same; determine the target detection result according to the position of each two-dimensional target frame in the area to be measured, the position of multiple three-dimensional anchor frames, the confidence of each two-dimensional target frame and the initial confidence corresponding to each target in multiple three-dimensional anchor frames.

[0177] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0178] According to the positions of each two-dimensional target frame and the positions of multiple three-dimensional anchor frames in the test area, multiple three-dimensional anchor frames are screened to obtain multiple candidate three-dimensional anchor frames; the target detection result is determined according to the confidence of each two-dimensional target frame and the initial confidence of multiple candidate three-dimensional anchor frames.

[0179] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0180] Based on each two-dimensional target frame, the two-dimensional position coordinates of the target in the two-dimensional target frame are obtained; according to a preset coordinate transformation matrix, the two-dimensional position coordinates are transformed into a three-dimensional coordinate system to obtain the transformed position coordinates of the target; according to the transformed position coordinates of the target and the positions of multiple three-dimensional anchor frames, multiple three-dimensional anchor frames are screened to obtain candidate three-dimensional anchor frames corresponding to the two-dimensional target frame.

[0181] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0182] Obtain the three-dimensional position coordinates of the target in each three-dimensional anchor point frame; calculate the position distance between the converted position coordinates of each target and the three-dimensional position coordinates of each target; for the converted position coordinates of each target, obtain the three-dimensional anchor point frame with the smallest position distance from multiple three-dimensional anchor point frames as the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame.

[0183] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0184] Obtain the initial confidence of the candidate anchor point frame corresponding to each two-dimensional target frame; according to the preset confidence floating strategy, increase the initial confidence of the candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame to obtain the floating confidence of the candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame; determine the target detection result according to the confidence of each two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame.

[0185] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0186] For each two-dimensional target frame, the confidence of each two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame are weighted and summed to obtain the floating weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame; the target detection result is determined according to the floating weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame and the initial confidence of other non-candidate three-dimensional anchor frames in multiple three-dimensional anchor frames.

[0187] In one embodiment, the confidence of the two-dimensional target frame includes confidences of multiple different candidate types; the floating confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame includes floating confidences of multiple different candidate types, and the confidence of the two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame include the same candidate type; when the computer program is executed by the processor, the following steps are also implemented:

[0188] For each two-dimensional target frame, calculate the product of the confidence of each candidate type and the first weight to obtain the first-weighted reset confidence corresponding to each candidate type; calculate the product of the floating confidence of each candidate type and the second weight to obtain the second-weighted reset confidence corresponding to each candidate type; wherein the sum of the first weight and the second weight is 1, and the first weight is greater than the second weight; calculate the sum of the first-weighted reset confidence corresponding to each candidate type and the second-weighted reset confidence corresponding to the candidate type to obtain the floating weighted confidence corresponding to each candidate type; based on the floating weighted confidence corresponding to each candidate type, determine the floating weighted confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame.

[0189] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0190] For each two-dimensional target frame, the confidence of each two-dimensional target frame and the initial confidence of multiple candidate three-dimensional anchor frames corresponding to each two-dimensional target frame are weighted summed to obtain the initial weighted confidence of the candidate three-dimensional anchor frame corresponding to each two-dimensional target frame;

[0191] The target detection result is determined according to the initial weighted confidence of the candidate 3D anchor box corresponding to each 2D target box and the initial confidence of other non-candidate 3D anchor boxes in multiple 3D anchor boxes.

[0192] In one embodiment, the confidence of the two-dimensional target frame includes confidences of multiple different candidate types; the initial confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame includes confidences of multiple different candidate types, and the confidence of the two-dimensional target frame and the initial confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame include the same candidate type; when the computer program is executed by the processor, the following steps are also implemented:

[0193] For each two-dimensional target frame, calculate the product of the confidence of each candidate type and the third weight to obtain the third-weighted reset confidence corresponding to each candidate type; calculate the product of the initial confidence of each candidate type and the fourth weight to obtain the fourth-weighted reset confidence corresponding to each candidate type; wherein the sum of the third weight and the fourth weight is 1, and the third weight is greater than the fourth weight; calculate the sum of the third-weighted reset confidence corresponding to each candidate type and the fourth-weighted reset confidence corresponding to the candidate type to obtain the initial weighted confidence corresponding to each candidate type; based on the initial weighted confidence corresponding to each candidate type, determine the initial weighted confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame.

[0194] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0195] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0196] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A target detection method, characterized in that: The method comprises: Receive two-dimensional target detection data of the area to be detected; wherein the two-dimensional target detection data includes the position of each two-dimensional target frame in the area to be detected and the confidence of each two-dimensional target frame; the confidence is used to characterize the probability that the target in the two-dimensional target frame belongs to the corresponding type; Dividing the point cloud data of the area to be measured according to a preset three-dimensional anchor point frame to obtain a plurality of three-dimensional anchor point frames including the point cloud data; wherein the initial confidence corresponding to each target in each three-dimensional anchor point frame is the same; The multiple three-dimensional anchor point frames are screened according to the positions of each two-dimensional target frame in the test area and the positions of the multiple three-dimensional anchor point frames to obtain a candidate three-dimensional anchor point frame corresponding to each two-dimensional target frame; Determine a reference confidence of each candidate 3D anchor point frame corresponding to the 2D target frame according to a preset confidence adjustment strategy and an initial confidence of each candidate 3D anchor point frame corresponding to the 2D target frame; the reference confidence is any one of a downward floating confidence, an upward floating confidence and an initial weighted confidence; The target detection result is determined according to the confidence of each of the two-dimensional target frames and the reference confidence of the candidate three-dimensional anchor point frames corresponding to each of the two-dimensional target frames.

2. The method according to claim 1, characterized in that The screening of the multiple 3D anchor frames according to the positions of each 2D target frame in the area to be measured and the positions of the multiple 3D anchor frames to obtain a candidate 3D anchor frame corresponding to each 2D target frame comprises: Based on each of the two-dimensional target frames, obtaining the two-dimensional position coordinates of the target in the two-dimensional target frame; According to a preset coordinate conversion matrix, the two-dimensional position coordinates are converted into a three-dimensional coordinate system to obtain the converted position coordinates of the target; The multiple anchor frames are screened according to the converted position coordinates of the target and the positions of the multiple three-dimensional anchor frames to obtain candidate three-dimensional anchor frames corresponding to the two-dimensional target frame.

3. The method according to claim 2, characterized in that The step of screening the multiple three-dimensional anchor frames according to the converted position coordinates of the target and the positions of the multiple three-dimensional anchor frames to obtain candidate three-dimensional anchor frames corresponding to the two-dimensional target frame includes: Obtain the three-dimensional position coordinates of the target in each of the three-dimensional anchor point frames; Calculating the position distance between the transformed position coordinates of each of the targets and the three-dimensional position coordinates of each of the targets; For each of the converted position coordinates of the target, a three-dimensional anchor point frame with the smallest position distance is obtained from the multiple three-dimensional anchor point frames as a candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame.

4. The method according to claim 1, characterized in that Determining the reference confidence of each candidate 3D anchor point frame corresponding to the 2D target frame according to a preset confidence adjustment strategy and the initial confidence of each candidate 3D anchor point frame corresponding to the 2D target frame includes: According to the confidence floating strategy, the initial confidence of the candidate 3D anchor point frame corresponding to each of the 2D target frames is increased to obtain the floating confidence of the candidate 3D anchor point frame corresponding to each of the 2D target frames as the reference confidence.

5. The method according to claim 4, characterized in that Determining the target detection result according to the confidence of each of the two-dimensional target frames and the reference confidence of the candidate three-dimensional anchor point frames corresponding to each of the two-dimensional target frames includes: For each of the two-dimensional target frames, weighted summing the confidence of each of the two-dimensional target frames and the floating confidence of the candidate three-dimensional anchor point frames corresponding to each of the two-dimensional target frames is performed to obtain the floating weighted confidence of the candidate three-dimensional anchor point frames corresponding to each of the two-dimensional target frames; The target detection result is determined according to the floating weighted confidence of the candidate three-dimensional anchor point frame corresponding to each of the two-dimensional target frames and the initial confidence of other non-candidate three-dimensional anchor point frames in the multiple three-dimensional anchor point frames.

6. The method according to claim 5, characterized in that The confidence of the two-dimensional target frame includes confidences of multiple different candidate types; the floating confidence of the candidate three-dimensional anchor frame corresponding to the two-dimensional target frame includes floating confidences of multiple different candidate types, and the confidence of the two-dimensional target frame and the floating confidence of the candidate three-dimensional anchor frame corresponding to the two-dimensional target frame include the same candidate type; For each of the two-dimensional target frames, weighted summing the confidence of each of the two-dimensional target frames and the floating confidence of the candidate three-dimensional anchor frame corresponding to each of the two-dimensional target frames to obtain the floating weighted confidence of the candidate three-dimensional anchor frame corresponding to each of the two-dimensional target frames includes: For each of the two-dimensional target frames, calculate the product of the confidence of each candidate type and the first weight to obtain the first weighted confidence corresponding to each of the candidate types; Calculate the product of the floating confidence of each candidate type and the second weight to obtain the second weighted confidence corresponding to each candidate type; wherein the sum of the first weight and the second weight is 1, and the first weight is greater than the second weight; Calculate the sum of the first weighted re-confidence corresponding to each candidate type and the second weighted re-confidence corresponding to the candidate type to obtain an upward weighted confidence corresponding to each candidate type; Based on the floating weighted confidence corresponding to each of the candidate types, the floating weighted confidence of the candidate three-dimensional anchor point box corresponding to the two-dimensional target box is determined.

7. The method according to claim 1, characterized in that Determining the reference confidence of each candidate 3D anchor point frame corresponding to the 2D target frame according to a preset confidence adjustment strategy and the initial confidence of each candidate 3D anchor point frame corresponding to the 2D target frame includes: According to the confidence weighting strategy, for each of the two-dimensional target frames, the confidence of each of the two-dimensional target frames and the initial confidences of multiple candidate three-dimensional anchor frames corresponding to each of the two-dimensional target frames are weighted and summed to obtain the initial weighted confidence of the candidate three-dimensional anchor frames corresponding to each of the two-dimensional target frames as the reference confidence.

8. The method according to claim 7, characterized in that The confidence of the two-dimensional target frame includes confidences of multiple different candidate types; the initial confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame includes confidences of multiple different candidate types, and the confidence of the two-dimensional target frame and the initial confidence of the candidate three-dimensional anchor point frame corresponding to the two-dimensional target frame include the same candidate type; For each of the two-dimensional target frames, weighted summing the confidence of each of the two-dimensional target frames and the initial confidences of multiple candidate three-dimensional anchor frames corresponding to each of the two-dimensional target frames to obtain the initial weighted confidence of the candidate three-dimensional anchor frames corresponding to each of the two-dimensional target frames includes: For each of the two-dimensional target frames, calculate the product of the confidence of each candidate type and the third weight to obtain the third-weighted confidence corresponding to each of the candidate types; Calculate the product of the initial confidence of each candidate type and the fourth weight to obtain the fourth weight reset confidence corresponding to each candidate type; wherein the sum of the third weight and the fourth weight is 1, and the third weight is greater than the fourth weight; Calculate the sum of the third weighted re-confidence corresponding to each candidate type and the fourth weighted re-confidence corresponding to the candidate type to obtain an initial weighted confidence corresponding to each candidate type; Based on the initial weighted confidence corresponding to each of the candidate types, an initial weighted confidence of the candidate three-dimensional anchor point box corresponding to the two-dimensional target box is determined.

9. A target detection device, characterized in that: The device comprises: A data receiving module, used for receiving two-dimensional target detection data of the area to be detected; wherein the two-dimensional target detection data includes the position of each two-dimensional target frame in the area to be detected and the confidence of each two-dimensional target frame; the confidence is used to characterize the probability that the target in the two-dimensional target frame belongs to the corresponding type; An anchor point generation module, used for dividing the point cloud data of the area to be measured according to a preset three-dimensional anchor point frame to obtain a plurality of three-dimensional anchor point frames including the point cloud data; wherein the initial confidence corresponding to each target in each three-dimensional anchor point frame is the same; A target determination module is used to screen the multiple three-dimensional anchor frames according to the positions of each two-dimensional target frame in the test area and the positions of the multiple three-dimensional anchor frames to obtain a candidate three-dimensional anchor frame corresponding to each of the two-dimensional target frames; determine the reference confidence of the candidate three-dimensional anchor frame corresponding to each of the two-dimensional target frames according to a preset confidence adjustment strategy and the initial confidence of the candidate three-dimensional anchor frame corresponding to each of the two-dimensional target frames; determine the target detection result according to the confidence of each of the two-dimensional target frames and the reference confidence of the candidate three-dimensional anchor frame corresponding to each of the two-dimensional target frames; the reference confidence is any one of a downward floating confidence, an upward floating confidence and an initial weighted confidence.

10. A roadside base station, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images