Three-dimensional target detection method, device, apparatus and storage medium

Abstract

The present disclosure provides a three-dimensional target detection method, device and equipment and storage medium, relates to the technical field of artificial intelligence, in particular to the technical field of automatic driving, image processing and the like. The three-dimensional target detection method comprises: processing an image to determine two-dimensional perception information of a target in the image; based on the two-dimensional perception information, obtaining a plurality of groups of candidate three-dimensional detection results of the target; and performing fusion processing on the plurality of groups of candidate three-dimensional detection results to determine a final three-dimensional detection result of the target. The present disclosure can conveniently and efficiently obtain the three-dimensional detection result of the target.

Description

Three-dimensional target detection methods, devices, equipment and storage media Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to the fields of autonomous driving and image processing, and in particular to a three-dimensional target detection method, apparatus, device, and storage medium. Background Technology

[0002] Three-dimensional (3D) object detection is an important technology for autonomous driving perception systems and a prerequisite for functions such as path planning, motion prediction, and collision avoidance.

[0003] In related technologies, complex deep learning network models or high-precision 3D sensors (such as LiDAR) can be used for 3D target detection. Summary of the Invention

[0004] This disclosure provides a three-dimensional target detection method, apparatus, device, and storage medium.

[0005] According to one aspect of this disclosure, a three-dimensional target detection method is provided, comprising: processing an image to determine two-dimensional perception information of a target in the image; obtaining multiple sets of candidate three-dimensional detection results of the target based on the two-dimensional perception information; and performing fusion processing on the multiple sets of candidate three-dimensional detection results to determine the final three-dimensional detection result of the target.

[0006] According to another aspect of this disclosure, a three-dimensional target detection device is provided, comprising: a determining module for processing an image to determine two-dimensional perception information of a target in the image; an acquiring module for acquiring multiple sets of candidate three-dimensional detection results of the target based on the two-dimensional perception information; and a fusing module for fusing the multiple sets of candidate three-dimensional detection results to determine the final three-dimensional detection result of the target.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to said at least one processor; wherein the memory stores instructions executable by said at least one processor, said instructions being executed by said at least one processor to enable said at least one processor to perform the method as described in any of the foregoing aspects.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method according to any of the preceding aspects.

[0009] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method according to any of the preceding aspects.

[0010] According to the technical solution disclosed herein, the three-dimensional detection results of the target can be obtained simply and efficiently.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0013] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0014] Figure 2 is a schematic diagram of various two-dimensional sensing information provided according to embodiments of the present disclosure;

[0015] Figure 3 is a schematic diagram of an overall framework for 3D object detection provided according to an embodiment of the present disclosure;

[0016] Figure 4 is a schematic diagram of the orientation angle provided according to an embodiment of the present disclosure;

[0017] Figure 5 is a schematic diagram of an application scenario provided according to an embodiment of this disclosure;

[0018] Figure 6 is a schematic diagram of another overall framework for 3D object detection provided according to an embodiment of the present disclosure;

[0019] Figure 7 is a schematic diagram according to a second embodiment of the present disclosure;

[0020] Figure 8 is a schematic diagram of IPM for wheel contact points provided according to an embodiment of the present disclosure;

[0021] Figure 9 is a schematic diagram of determining 3D position information based on two wheel contact points in different directions according to an embodiment of the present disclosure;

[0022] Figure 10 is a schematic diagram of two rectangular boxes based on the fitting of boundary points of passable areas according to embodiments of the present disclosure;

[0023] Figure 11 is a schematic diagram according to a third embodiment of the present disclosure;

[0024] Figure 12 is a schematic diagram of an electronic device used to implement the three-dimensional target detection method of the present disclosure embodiments. Detailed Implementation

[0025] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0026] In related technologies, image-based methods require calculating the depth information of the target, which necessitates complex deep learning networks, making the computation quite complex. Alternatively, high-precision 3D sensors (such as LiDAR) can be used to obtain 3D detection results.

[0027] However, the above implementation method is too complicated.

[0028] To facilitate simple and efficient 3D target detection, the present disclosure provides the following embodiments.

[0029] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure, which provides a three-dimensional target detection method, the method comprising:

[0030] 101. Process the image to determine the two-dimensional perceptual information of the target in the image.

[0031] 102. Based on the two-dimensional perception information, obtain multiple sets of candidate three-dimensional detection results for the target.

[0032] 103. The multiple sets of candidate 3D detection results are fused to determine the final 3D detection result of the target.

[0033] Taking autonomous driving as an example, cameras can be installed on autonomous vehicles to capture images. The autonomous driving system can then process these images to obtain two-dimensional perception information of the targets within them. The camera can be a monocular camera, specifically a monocular fisheye camera.

[0034] In autonomous driving scenarios, the target can also be called an obstacle, which can be other vehicles, pedestrians, traffic facilities, etc.

[0035] Two-dimensional sensing information is obtained by processing images. Since the sensing information is two-dimensional, it does not require high-precision and complex equipment such as 3D sensors.

[0036] Taking other vehicles as an example, the two-dimensional perception information of the vehicle includes: category information (such as cars, buses, trucks, etc.), positioning data (such as the vehicle's 2D position and speed), two-dimensional (2D) bounding box, vehicle dividing lines, wheel contact points, and freespace boundary points.

[0037] A 2D bounding box refers to a 2D rectangular frame that surrounds a vehicle.

[0038] The vehicle body dividing line is a straight line that passes through the rear of the vehicle and is perpendicular to the ground.

[0039] The wheel contact point refers to the point where the vehicle's wheel intersects with the ground.

[0040] Passable area refers to the area through which vehicles can travel.

[0041] The traversable zone boundary refers to the boundary between the traversable and impassable zones, and this boundary is usually a curve surrounding the vehicle.

[0042] Passable area boundary point; point on the boundary of the passable area.

[0043] As shown in Figure 2, taking a vehicle as an example, by processing the image, we can obtain the vehicle's 2D bounding box 201, body dividing line 202, wheel contact point 203, and passable area boundary point 204.

[0044] After obtaining two-dimensional perception information, multiple sets of candidate 3D detection results can be obtained based on the two-dimensional perception information.

[0045] Multiple candidate 3D detection results can include: first candidate 3D detection result, second candidate 3D detection result, and third candidate 3D detection result.

[0046] As shown in Figure 3, the first candidate 3D detection result is mainly determined based on the vehicle segmentation line, the second candidate 3D detection result is mainly determined based on the wheel contact point, and the third candidate 3D detection result is mainly determined based on the boundary point of the passable area.

[0047] After obtaining the above three candidate 3D detection results, these three candidate 3D detection results can be fused to obtain the final 3D detection result.

[0048] Taking a vehicle as an example, the 3D detection results can specifically include: the vehicle's heading angle, 3D position information, and 3D size information.

[0049] As shown in Figure 4, a three-dimensional coordinate system can be pre-established. The orientation angle refers to the angle θ between the vehicle's front direction and the x-axis. The 3D position information is the three-dimensional position coordinates (x, y, z) of the vehicle's center point (such as the centroid or geometric center). The 3D dimension information is the vehicle's length, width, and height (L, W, H).

[0050] The final orientation angle can be obtained by weighted summation of candidate orientation angles from multiple candidate 3D detection results. The final 3D position information and final 3D size information can be obtained by averaging the candidate 3D position information and candidate 3D size information from multiple candidate 3D detection results.

[0051] In this embodiment, two-dimensional perception information is obtained by processing the image, and three-dimensional target detection is performed based on this information, eliminating the need for complex 3D sensor equipment. Furthermore, the final three-dimensional detection result is obtained by fusing multiple candidate three-dimensional detection results based on the two-dimensional perception information, without the need to calculate depth information, and therefore without the need for complex deep learning networks. Thus, the three-dimensional detection result of the target can be obtained simply and efficiently.

[0052] To better understand this disclosure, the application scenarios of the embodiments of this disclosure will be described.

[0053] Figure 5 is a schematic diagram of an application scenario provided by an embodiment of this disclosure. This embodiment takes an autonomous driving scenario as an example. As shown in Figure 5, the equipment involved in the autonomous driving scenario includes: an autonomous driving vehicle 501 and a server 502. The autonomous driving vehicle 501 and the server 502 can be connected through a communication network, which can be a wireless communication network, such as communicating through signals provided by a base station. In addition, in order to achieve accurate positioning of the autonomous driving vehicle, a satellite (not shown in the figure) may also be included in the autonomous driving scenario. The server 502 can be a local server of the automaker corresponding to the autonomous driving vehicle or a cloud server.

[0054] The autonomous driving function of autonomous vehicles can be achieved through an autonomous driving system (503). The levels of autonomous driving achieved by these systems are currently classified as L0 to L5. L0 represents no autonomous driving, i.e., traditional manual driving by the driver; L1, also known as driver assistance, includes basic functions such as cruise control, automatic parking, and lane keeping; L2, also known as semi-autonomous driving, includes functions such as automatic assisted driving and hazard prediction braking; L3, also known as conditional autonomous driving, compared to L2, can achieve fully automated driving on normal roads, but manual assistance is still required in emergency situations; L4 represents highly automated driving, where the vehicle's overall braking performance and responsiveness have reached a high level, the driver does not need to operate the vehicle, and the driving is relatively smooth; L5 represents unconditional fully autonomous driving technology, achieving full automation regardless of road conditions or weather.

[0055] Furthermore, as shown in Figure 5, the autonomous driving system 503 may include: a perception system 5031, a decision-making and planning system 5032, and an execution and control system 5033.

[0056] The various systems included in an autonomous driving system can also be referred to as subsystems, modules, components, units, etc. These subsystems can be software, hardware, or a combination of both. Some or all of these subsystems can be deployed on the autonomous vehicle and / or on a server.

[0057] The perception system 5031 is used to acquire perception data, which may include obstacle data, vehicle data, etc. Obstacle data includes, for example, the position and speed of obstacles, while vehicle data includes, for example, the position and speed of the vehicle itself.

[0058] The perception system 5032 may include sensors installed on the vehicle, or sensors and a computing platform. If the sensors have computing capabilities, they can calculate obstacle data, vehicle data, etc., based on the collected sensor data. If the sensors do not have computing capabilities, the computing platform can perform calculations on the sensor data collected by the sensors to obtain obstacle data and vehicle data.

[0059] Sensors can include: cameras, lidar, millimeter-wave radar, ultrasonic radar, Global Positioning System (GPS), inertial measurement unit (IMU), etc.

[0060] The computing platform can be an onboard computing chip, or the vehicle can send the collected sensor data to a server, where the computing platform performs calculations on the relevant information.

[0061] The decision-making and planning system 5032 is used to make decisions based on the perception data obtained by the perception system 5031, thereby obtaining decision data. Decisions may include path planning, behavior planning, trajectory planning, etc. Correspondingly, the decision data may include the resulting driving behavior data, path data, trajectory data, etc.

[0062] The execution and control system 5033 may specifically include the vehicle's chassis system, which is used to control the vehicle's driving behavior based on the decision data obtained by the decision system 5033, such as controlling the vehicle's steering device, throttle device, braking device, etc., so that the vehicle travels according to the decided driving behavior, path, trajectory, etc.

[0063] Specifically, in this embodiment of the disclosure, the perception system primarily uses a camera as its sensor. The camera acquires images containing targets. The perception system processes these images to obtain multiple sets of candidate 3D detection results for the targets. Based on these candidate 3D detection results, a final 3D detection result is obtained. The final 3D detection result includes: final orientation angle, final 3D position information, and final 3D size information. Subsequently, the decision-making and planning system can perform planning based on the final 3D detection result, and the execution and control system can execute corresponding operations based on the planning results.

[0064] The candidate 3D detection results are mainly calculated based on the vehicle body segmentation line, wheel contact point, and passable area boundary point. The above three types of perception information (vehicle body segmentation line, wheel contact point, and passable area boundary point) can be obtained through a pre-trained neural network model. For example, a first neural network model is used to process the input image to determine the vehicle body segmentation line; a second neural network model is used to process the input image to determine the wheel contact point; and a third neural network model is used to process the input image to determine the passable area boundary point.

[0065] During the driving process, autonomous vehicles can continuously acquire multiple images. For example, an image is acquired at time t, and another image is acquired at time t+1. The images corresponding to the above three types of perception information may be acquired at different times. For example, a first neural network is used to process the first image at the first time to determine the vehicle body dividing line and wheel contact point, and a third neural network model is used to process the second image at the second time to determine the boundary point of the passable area.

[0066] To ensure data accuracy, the three types of sensing information mentioned above can be processed for time synchronization and / or spatial synchronization.

[0067] In addition to obtaining vehicle information, the perception system can also obtain information about other obstacles (such as pedestrians). The present disclosure is mainly for obtaining 3D detection results of vehicles. Therefore, information about other obstacles can be filtered out, such as filtering out the passable boundary points of pedestrians.

[0068] In addition, the image can include information about multiple obstacles, and the three types of perception information mentioned above can be correlated to obtain two-dimensional perception information about the same obstacle.

[0069] Therefore, as shown in Figure 6, the overall architecture can include: obtaining two-dimensional perception information through a perception system, which may include: category information, vehicle body segmentation lines, wheel contact points, passable area boundary points, and positioning data; performing data preprocessing on the two-dimensional perception information, such as time synchronization, spatial synchronization, and noise filtering; performing data association processing on the preprocessed two-dimensional perception information, i.e., performing 2D image domain association. The above data preprocessing and data association are mainly performed on vehicle body segmentation lines, wheel contact points, and passable boundary points; obtaining candidate 3D detection results based on the associated two-dimensional perception information, mainly including obtaining a first candidate 3D detection result based on the vehicle body segmentation lines, a second candidate 3D detection result based on the wheel contact points, and a third candidate 3D detection result based on the passable area boundary points; and fusing the candidate 3D detection results to obtain the final 3D detection result.

[0070] In conjunction with the above application scenarios, this disclosure also provides a three-dimensional target detection method.

[0071] Figure 7 is a schematic diagram according to a second embodiment of the present disclosure, which provides a three-dimensional target detection method, the method comprising:

[0072] 701. Process the image to determine the two-dimensional perceptual information of the target in the image.

[0073] Images can be captured using cameras installed on autonomous vehicles.

[0074] Two-dimensional sensing information can include: vehicle body dividing lines, wheel contact points, and boundary points of passable areas.

[0075] The image can be processed using a first neural network model to obtain the vehicle body segmentation lines. A second neural network model can be used to process the image to obtain the wheel contact points. A third neural network model can be used to process the image to obtain the boundary points of the passable area. These three neural network models can be pre-trained.

[0076] 702. Perform data preprocessing on the two-dimensional sensing information to obtain preprocessed two-dimensional sensing information.

[0077] Data preprocessing may include at least one of the following: time synchronization processing, spatial synchronization processing, and noise filtering processing.

[0078] Specifically, assuming that the two-dimensional perception information includes: vehicle body dividing lines, wheel contact points, and passable area boundary points, the above three types of perception information may be obtained by processing images at different times. Therefore, it is necessary to perform time synchronization processing on the above three types of perception information at different times. Specifically, timestamps can be added to the perception information, and time synchronization processing can be performed based on the timestamps.

[0079] In addition, autonomous vehicles may move when acquiring images at different times. Therefore, spatial synchronization processing can be performed on the above three types of perception information. Specifically, spatial synchronization can be performed using the pose information and speed information of the autonomous vehicle with corresponding timestamps.

[0080] In addition, taking a vehicle as an example, the two-dimensional perception information may contain perception information of other targets (such as pedestrians). In this case, removing the perception information of other targets is to perform noise filtering. Specifically, lane lines can be used to remove the boundary points of the passable area of ​​other targets.

[0081] In this embodiment, accurate preprocessed two-dimensional perception information can be obtained through time synchronization processing, spatial synchronization processing, and noise filtering processing, thereby improving the accuracy of 3D target detection.

[0082] 703. Perform data association processing on the preprocessed two-dimensional sensing information to obtain the associated two-dimensional sensing information of the same target among the multiple targets.

[0083] The image may contain multiple vehicles, such as the first vehicle and the second vehicle. Therefore, the two-dimensional perception information may come from multiple vehicles. In order to ensure the accuracy of the results, data association processing is required to obtain the two-dimensional perception information of the same vehicle.

[0084] Specifically, the positional relationship between the preprocessed two-dimensional sensing information and the two-dimensional bounding box of the target is determined; based on the positional relationship, the associated two-dimensional sensing information of the same target is determined.

[0085] For example, taking two vehicles as an example, by processing the image, a first 2D bounding box and a second 2D bounding box can be obtained, as well as a first vehicle body segmentation line, a second vehicle body segmentation line, a first wheel contact point, a second wheel contact point, a first passable area boundary point, and a second passable area boundary point.

[0086] Assuming that the first vehicle body segmentation line, the first wheel contact point, and the first passable area boundary point are located within the first 2D bounding box, the associated 2D perception information of the first vehicle is determined to include: a vehicle body segmentation line, the first wheel contact point, and the first passable area boundary point. Then, based on these three types of perception information, multiple sets of candidate 3D detection results of the first vehicle are obtained, and then based on the multiple sets of candidate 3D detection results, the final 3D detection result of the first vehicle is obtained.

[0087] In this embodiment, based on the positional relationship between the two-dimensional sensing information and the two-dimensional bounding box, the two-dimensional sensing information of the same target can be obtained easily and efficiently.

[0088] In this embodiment, by performing data preprocessing on the two-dimensional perception information, more accurate data can be obtained, thereby improving the accuracy of three-dimensional target detection; by performing data association processing on the two-dimensional perception information, two-dimensional perception information of the same target can be obtained, thereby improving the accuracy of 3D target detection.

[0089] 704. Based on the associated two-dimensional perception information of the same target, obtain multiple sets of candidate three-dimensional detection results for the same target.

[0090] The target is a vehicle; the two-dimensional perception information includes at least two of the following: vehicle body segmentation lines, wheel contact points, and traversable area boundary points; the acquisition of multiple candidate three-dimensional detection results of the target based on the two-dimensional perception information includes at least two of the following: acquiring a first candidate three-dimensional detection result based on the vehicle body segmentation lines; acquiring a second candidate three-dimensional detection result based on the wheel contact points; and acquiring a third candidate three-dimensional detection result based on the traversable area boundary points.

[0091] In this embodiment, since the vehicle body dividing line, wheel contact point and passable area boundary point are closely related to the vehicle, the candidate 3D detection results of the vehicle can be obtained by acquiring the above three types of perception information, which can lead to more accurate candidate 3D detection results. Based on the candidate 3D detection results, a more accurate final 3D detection result can be obtained.

[0092] For obtaining the first candidate 3D detection result based on the vehicle body segmentation lines:

[0093] The first candidate 3D detection result includes: first candidate orientation angle, first candidate 3D position information, and first candidate 3D size information;

[0094] The two-dimensional sensing information also includes: current category information and two-dimensional location information;

[0095] The step of obtaining the first candidate 3D detection result based on the vehicle body segmentation line includes:

[0096] Determine the current ratio between the vehicle body dividing line and the two-dimensional bounding box of the target;

[0097] Based on the correspondence between the current ratio value and the preset ratio value and the direction angle, the first candidate direction angle is determined;

[0098] Based on the correspondence between the current category information and the preset category information and three-dimensional size information, the first candidate three-dimensional size information is determined;

[0099] The first candidate three-dimensional position information is determined based on the two-dimensional position information and the height information in the first candidate three-dimensional size information.

[0100] As shown in Figure 2, after determining the vehicle body dividing line 202 and the 2D bounding box 201, their ratio can be calculated. For example, the 2D bounding box is a matrix frame with side lengths L0 and height H0. After determining the body dividing line, the distance between the body dividing line and the edge in the height direction can be calculated, such as the distance between the body dividing line 202 and the left side of the 2D bounding box. Assuming this distance is L1, the ratio is L1 / L0. A table of empirical values ​​can be pre-prepared, recording the relationship between the ratio values ​​and the direction angle. Based on this correspondence, the first candidate direction angle can be obtained.

[0101] In addition, template information can be pre-configured, which records the correspondence between category information and 3D size information. For example, the 3D size information for a car is l1*w1*h1, and the 3D size information for a bus is l2*w2*h2. If the current category information is a car, then the first candidate 3D size information is l1*w1*h1.

[0102] In addition, the perception information may include positioning data, which includes 2D position information, such as (x1, y1). Assuming that the height information obtained based on the template information is h1, and the center point position is taken as the vehicle position, then the 3D position information of the vehicle is (x1, y1, h1 / 2).

[0103] In this embodiment, the first candidate 3D detection result can be obtained simply and efficiently based on the vehicle body segmentation line.

[0104] Regarding the second candidate 3D detection result obtained based on the wheel contact point:

[0105] The second candidate 3D detection result includes: second candidate orientation angle, second candidate 3D position information, and second candidate 3D size information;

[0106] The wheel has two contact points. The process of obtaining a second candidate 3D detection result based on these wheel contact points includes:

[0107] Perform inverse perspective mapping (IPM) on the two wheel contact points to obtain two transformation points, and determine the line connecting the two transformation points;

[0108] The second candidate direction angle is determined based on the direction angle of the connecting line, and the current wheelbase information of the target is determined based on the length of the connecting line;

[0109] Based on the current wheelbase information and the correspondence between the preset wheelbase information and the preset wheelbase information and the three-dimensional dimension information, the second candidate three-dimensional dimension information is determined;

[0110] Based on the position information of the two transformation points, the current axis distance information, and the height information in the second candidate three-dimensional dimension information, the second candidate three-dimensional position information is determined.

[0111] In images captured by a forward-facing camera, due to the perspective effect, objects that are originally parallel appear to intersect in the image. IPM can eliminate this perspective effect, hence the name inverse perspective transformation.

[0112] The wheel contact point determined from the image is subject to perspective effect. This perspective effect can be eliminated by using IPM to obtain the location information of the wheel contact point in the real world.

[0113] For example, as shown in Figure 8, based on the image, two wheel contact points 801 can be determined. After performing IPM on these two wheel contact points 801, two transformation points 802 are obtained, which are the two vehicle contact points that eliminate the perspective effect.

[0114] After obtaining two transformation points, the angle θ between the two transformation points can be calculated based on their position information. This angle can be used as the second candidate direction angle, or the second candidate direction angle can be obtained by performing a preset calculation on this angle.

[0115] After obtaining two transformation points, the length D of the line connecting the two transformation points can be calculated based on their position information. This length can be used as the current wheelbase information, or the current wheelbase information can be obtained by pre-calculating this length.

[0116] In addition, template information can be pre-configured, which records the correspondence between wheelbase information and 3D dimension information. Based on this correspondence and the current wheelbase information, second candidate 3D dimension information can be obtained.

[0117] Furthermore, based on the positional information of the two transformation points, the positional information of the midpoint between these two transformation points, such as (x0, y0), can be calculated. Then, the 3D positional information of the vehicle is determined according to the current wheelbase information D and the height information in the second candidate 3D dimension information. For example, if the second candidate 3D dimension information is l1*w1*h1, in the coordinate system shown in Figure 9, assuming that the two transformation points are the wheel contact points in the length direction of the vehicle, then the 3D positional information of the vehicle is (x0+D / 2, y0, h1 / 2). Alternatively, assuming that the two transformation points are the wheel contact points in the width direction of the vehicle, then the 3D positional information of the vehicle is (x0, y0+D / 2, h1 / 2).

[0118] In this embodiment, the second candidate 3D detection result can be obtained simply and efficiently based on the wheel grounding point.

[0119] For obtaining the third candidate 3D detection result based on the boundary points of the passable area:

[0120] The third candidate 3D detection result includes: third candidate orientation angle, third candidate 3D position information, and third candidate 3D size information;

[0121] The two-dimensional sensing information also includes: current category information;

[0122] The step of determining the third candidate 3D detection result based on the boundary points of the passable area includes:

[0123] The boundary points of the passable area are fitted with rectangular boxes to obtain fitted rectangular boxes.

[0124] Based on the rectangle, the third candidate orientation angle is determined;

[0125] Based on the correspondence between the current category information and the preset category information and three-dimensional size information, the third candidate three-dimensional size information is determined;

[0126] The third candidate three-dimensional position information is determined based on the two-dimensional position information of the matrix frame and the height information in the third candidate three-dimensional size information.

[0127] After determining the boundary points of the passable area, a rectangular bounding box can be fitted using algorithms such as Hough to obtain the fitted rectangular bounding box. This bounding box can correspond to two visible edges or one visible edge. As shown in Figure 10, fitting a set of boundary points of the passable area yields a matrix bounding box 1001 with two visible edges, or fitting another set of boundary points of the passable area yields a rectangular bounding box 1002 with one visible edge.

[0128] After obtaining the rectangular frame, the included angle between two adjacent sides of the frame can be used as the third candidate direction angle. For two visible sides, this is the angle between those two visible sides. For a single visible side, this is the angle between that visible side and its adjacent side.

[0129] In addition, template information can be pre-configured, which records the correspondence between category information and 3D size information. For example, the 3D size information for a car is l1*w1*h1, and the 3D size information for a bus is l2*w2*h2. If the current category information is a car, then the third candidate 3D size information is l1*w1*h1.

[0130] In addition, based on the fitted rectangle, the coordinates of the center point of the rectangle can be calculated, such as (x1, y1). Assuming that the height information obtained based on the template information is h1, the 3D position information of the vehicle is (x1, y1, h1 / 2).

[0131] In this embodiment, the third candidate 3D detection result can be obtained simply and efficiently based on the boundary points of the passable area.

[0132] 705. Perform fusion processing on multiple sets of candidate 3D detection results for the same target to obtain the final 3D detection result for the same target.

[0133] Among them, for the same goal:

[0134] The final three-dimensional detection results include: final orientation angle, final three-dimensional position information, and final three-dimensional size information;

[0135] Each set of candidate 3D detection results includes: candidate orientation angle, candidate 3D position information, and candidate 3D size information;

[0136] The process of fusing the multiple sets of candidate 3D detection results to determine the final 3D detection result of the target includes:

[0137] A weighted summation operation is performed on multiple candidate direction angles to determine the final direction angle;

[0138] The mean value of multiple candidate 3D position information is calculated to determine the final 3D position information;

[0139] The final three-dimensional dimension information is determined by averaging multiple candidate three-dimensional dimension information.

[0140] Taking the above three sets of candidate 3D detection results as examples, the formula for calculating the final orientation angle can be:

[0141] Final direction angle = w1 * first candidate direction angle + w2 * second candidate direction angle + w3 * third candidate direction angle.

[0142] Where w1, w2, and w3 are preset weighted values, and the order of w1 + w2 + w3 can be: w2 > w3 (if the fitted matrix box has two visible sides) > w1, or w2 > w1 > w3 (if the fitted matrix box has one visible side).

[0143] The formulas for calculating the final 3D position information and the final 3D size information can be:

[0144] Final 3D position information = (first candidate 3D position information + second candidate 3D position information + third candidate 3D position information) / 3.

[0145] Final 3D size information = (first candidate 3D size information + second candidate 3D size information + third candidate 3D size information) / 3.

[0146] Since 3D position information (including three dimensions: x, y, z) and 3D size information (including three dimensions: l, w, h) are multi-dimensional, the above mean calculation is performed on the mean of each dimension. For example, for the x-coordinate, it is the mean of the x-coordinates x1, x2, and x3 in the x-direction of the three candidate 3D position information.

[0147] In this embodiment, the final direction angle is obtained by performing a weighted summation operation on the candidate direction angles, which can yield a more accurate final direction angle; the final 3D position information and final 3D size information can be easily and accurately obtained by performing an average operation on the candidate 3D position information and candidate 3D size information.

[0148] Figure 11 is a schematic diagram according to a third embodiment of the present disclosure. This embodiment provides a three-dimensional target detection device 1100, which includes: a determination module 1101, an acquisition module 1102, and a fusion module 1103.

[0149] The determining module 1101 is used to process the image to determine the two-dimensional perception information of the target in the image; the acquiring module 1102 is used to acquire multiple sets of candidate three-dimensional detection results of the target based on the two-dimensional perception information; and the fusing module 1103 is used to perform fusing processing on the multiple sets of candidate three-dimensional detection results to determine the final three-dimensional detection result of the target.

[0150] In this embodiment, two-dimensional perception information is obtained by processing the image, and three-dimensional target detection is performed based on this information, eliminating the need for complex 3D sensor equipment. Furthermore, the final three-dimensional detection result is obtained by fusing multiple candidate three-dimensional detection results based on the two-dimensional perception information, without the need to calculate depth information, and therefore without the need for complex deep learning networks. Thus, the three-dimensional detection result of the target can be obtained simply and efficiently.

[0151] In some embodiments, the target is a vehicle; the two-dimensional perception information includes at least two of the following: vehicle body segmentation lines, wheel contact points, and traversable area boundary points; the acquisition module 1102 is further configured to: acquire a first candidate three-dimensional detection result based on the vehicle body segmentation lines; acquire a second candidate three-dimensional detection result based on the wheel contact points; and acquire a third candidate three-dimensional detection result based on the traversable area boundary points.

[0152] In this embodiment, since the vehicle body dividing line, wheel contact point and passable area boundary point are closely related to the vehicle, the candidate 3D detection results of the vehicle can be obtained by acquiring the above three types of perception information, which can lead to more accurate candidate 3D detection results. Based on the candidate 3D detection results, a more accurate final 3D detection result can be obtained.

[0153] In some embodiments, the first candidate 3D detection result includes: a first candidate orientation angle, first candidate 3D position information, and first candidate 3D size information; the 2D perception information further includes: current category information and 2D position information; the acquisition module 1102 is further configured to: determine the current ratio of the vehicle body dividing line to the target's 2D bounding box; determine the first candidate orientation angle based on the current ratio and the correspondence between the preset ratio and the orientation angle; determine the first candidate 3D size information based on the current category information and the correspondence between the preset category information and the 3D size information; and determine the first candidate 3D position information based on the 2D position information and the height information in the first candidate 3D size information.

[0154] In this embodiment, the first candidate 3D detection result can be obtained simply and efficiently based on the vehicle body segmentation line.

[0155] In some embodiments, the second candidate 3D detection result includes: a second candidate orientation angle, second candidate 3D position information, and second candidate 3D size information; the wheel contact points are two, and the acquisition module is further configured to: perform inverse perspective transformation on the two wheel contact points to obtain two transformation points, and determine the line connecting the two transformation points; determine the second candidate orientation angle based on the orientation angle of the line connecting the two transformation points, and determine the current wheelbase information of the target based on the length of the line connecting the two transformation points; determine the second candidate 3D size information based on the current wheelbase information and the correspondence between preset wheelbase information and preset wheelbase information and 3D size information; and determine the second candidate 3D position information based on the position information of the two transformation points, the current wheelbase information, and the height information in the second candidate 3D size information.

[0156] In this embodiment, the second candidate 3D detection result can be obtained simply and efficiently based on the wheel grounding point.

[0157] In some embodiments, the third candidate 3D detection result includes: a third candidate orientation angle, third candidate 3D position information, and third candidate 3D size information; the 2D perception information further includes: current category information; the acquisition module 1102 is further configured to: perform rectangular box fitting processing on the boundary points of the passable area to obtain a fitted rectangular box; determine the third candidate orientation angle based on the rectangular box; determine the third candidate 3D size information based on the correspondence between the current category information and the preset category information and 3D size information; and determine the third candidate 3D position information based on the 2D position information of the matrix box and the height information in the third candidate 3D size information.

[0158] In this embodiment, the third candidate 3D detection result can be obtained simply and efficiently based on the boundary points of the passable area.

[0159] In some embodiments, the final 3D detection result includes: final orientation angle, final 3D position information, and final 3D size information; each group of candidate 3D detection results includes: candidate orientation angle, candidate 3D position information, and candidate 3D size information; the fusion module 1103 is further configured to: perform a weighted summation operation on multiple candidate orientation angles to determine the final orientation angle; perform an average operation on multiple candidate 3D position information to determine the final 3D position information; and perform an average operation on multiple candidate 3D size information to determine the final 3D size information.

[0160] In this embodiment, the final direction angle is obtained by performing a weighted summation operation on the candidate direction angles, which can yield a more accurate final direction angle; the final 3D position information and final 3D size information can be easily and accurately obtained by performing an average operation on the candidate 3D position information and candidate 3D size information.

[0161] In some embodiments, there are multiple targets, and the acquisition module 1102 is further configured to: perform data preprocessing on the two-dimensional perception information to obtain preprocessed two-dimensional perception information; perform data association processing on the preprocessed two-dimensional perception information to obtain associated two-dimensional perception information of the same target among the multiple targets; and obtain multiple sets of candidate three-dimensional detection results of the same target based on the associated two-dimensional perception information of the same target.

[0162] In this embodiment, by performing data preprocessing on the two-dimensional perception information, more accurate data can be obtained, thereby improving the accuracy of three-dimensional target detection; by performing data association processing on the two-dimensional perception information, two-dimensional perception information of the same target can be obtained, thereby improving the accuracy of 3D target detection.

[0163] In some embodiments, the acquisition module 1102 is further configured to perform at least one of the following: time synchronization processing of the two-dimensional sensing information; spatial synchronization processing of the two-dimensional sensing information; and noise filtering processing of the two-dimensional sensing information.

[0164] In this embodiment, accurate preprocessed two-dimensional perception information can be obtained through time synchronization processing, spatial synchronization processing, and noise filtering processing, thereby improving the accuracy of 3D target detection.

[0165] In some embodiments, the acquisition module 1102 is further configured to: determine the positional relationship between the preprocessed two-dimensional sensing information and the two-dimensional bounding box of the target; and determine the associated two-dimensional sensing information of the same target based on the positional relationship.

[0166] In this embodiment, based on the positional relationship between the two-dimensional sensing information and the two-dimensional bounding box, the two-dimensional sensing information of the same target can be obtained easily and efficiently.

[0167] It is understood that the same or similar content in different embodiments of this disclosure can be referred to each other.

[0168] It is understood that the terms "first" and "second" in the embodiments of this disclosure are only used for distinction and do not indicate the degree of importance or the order of events.

[0169] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0170] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0171] Figure 12 illustrates a schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 1200 may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0172] As shown in Figure 12, the electronic device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. The RAM 1203 may also store various programs and data required for the operation of the electronic device 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0173] Multiple components in electronic device 1200 are connected to I / O interface 1205, including: input unit 1206, such as keyboard, mouse, etc.; output unit 1207, such as various types of displays, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows electronic device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0174] The computing unit 1201 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as the three-dimensional object detection method. For example, in some embodiments, the control method or the training method of the control model may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by the computing unit 1201, one or more steps of the three-dimensional object detection method described above may be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured to perform a three-dimensional target detection method by any other suitable means (e.g., by means of firmware).

[0175] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0176] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to the processor or controller of a general-purpose computer, special-purpose computer, or other programmable load balancing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0177] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0178] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0179] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0180] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0181] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0182] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A three-dimensional target detection method, comprising: The image is processed to determine the two-dimensional perceptual information of the target in the image; Based on the two-dimensional perception information, multiple sets of candidate three-dimensional detection results for the target are obtained; The multiple sets of candidate 3D detection results are fused to determine the final 3D detection result of the target. The final 3D detection result includes: final orientation angle, final 3D position information, and final 3D size information; each group of candidate 3D detection results includes: candidate orientation angle, candidate 3D position information, and candidate 3D size information; the fusion processing of the multiple groups of candidate 3D detection results to determine the final 3D detection result of the target includes: performing a weighted summation operation on multiple candidate orientation angles to determine the final orientation angle; performing an average operation on multiple candidate 3D position information to determine the final 3D position information; and performing an average operation on multiple candidate 3D size information to determine the final 3D size information.

2. The method according to claim 1, wherein, The target is a vehicle; the two-dimensional perception information includes at least two of the following: vehicle body segmentation lines, wheel contact points, and traversable area boundary points; the acquisition of multiple candidate three-dimensional detection results of the target based on the two-dimensional perception information includes at least two of the following: acquiring a first candidate three-dimensional detection result based on the vehicle body segmentation lines; acquiring a second candidate three-dimensional detection result based on the wheel contact points; and acquiring a third candidate three-dimensional detection result based on the traversable area boundary points.

3. The method according to claim 2, wherein, The first candidate 3D detection result includes: a first candidate orientation angle, first candidate 3D position information, and first candidate 3D size information; the 2D perception information further includes: current category information and 2D position information; obtaining the first candidate 3D detection result based on the vehicle body dividing line includes: determining the current ratio of the vehicle body dividing line to the target's 2D bounding box; determining the first candidate orientation angle based on the current ratio and the correspondence between the preset ratio and the orientation angle; determining the first candidate 3D size information based on the current category information and the correspondence between the preset category information and the 3D size information; and determining the first candidate 3D position information based on the 2D position information and the height information in the first candidate 3D size information.

4. The method according to claim 2, wherein, The second candidate 3D detection result includes: a second candidate orientation angle, second candidate 3D position information, and second candidate 3D size information; there are two wheel contact points, and obtaining the second candidate 3D detection result based on the wheel contact points includes: performing inverse perspective transformation on the two wheel contact points to obtain two transformation points, and determining the line connecting the two transformation points; determining the second candidate orientation angle based on the orientation angle of the line, and determining the current wheelbase information of the target based on the length of the line; determining the second candidate 3D size information based on the correspondence between the current wheelbase information and the preset wheelbase information and 3D size information; and determining the second candidate 3D position information based on the position information of the two transformation points, the current wheelbase information, and the height information in the second candidate 3D size information.

5. The method according to claim 2, wherein, The third candidate 3D detection result includes: a third candidate orientation angle, third candidate 3D position information, and third candidate 3D size information; the 2D perception information also includes: current category information; obtaining the third candidate 3D detection result based on the boundary points of the passable area includes: performing rectangular box fitting processing on the boundary points of the passable area to obtain a fitted rectangular box; determining the third candidate orientation angle based on the rectangular box; determining the third candidate 3D size information based on the correspondence between the current category information and the preset category information and 3D size information; and determining the third candidate 3D position information based on the 2D position information of the rectangular box and the height information in the third candidate 3D size information.

6. The method according to any one of claims 1-5, wherein, The targets are multiple, and the step of obtaining multiple sets of candidate 3D detection results for the targets based on the 2D perception information includes: performing data preprocessing on the 2D perception information to obtain preprocessed 2D perception information; performing data association processing on the preprocessed 2D perception information to obtain associated 2D perception information of the same target among the multiple targets; and obtaining multiple sets of candidate 3D detection results for the same target based on the associated 2D perception information of the same target.

7. The method according to claim 6, wherein, The data preprocessing of the two-dimensional sensing information includes at least one of the following: performing time synchronization processing on the two-dimensional sensing information; performing spatial synchronization processing on the two-dimensional sensing information; and performing noise filtering processing on the two-dimensional sensing information.

8. The method according to claim 6, wherein, The step of performing data association processing on the preprocessed two-dimensional sensing information to obtain the associated two-dimensional sensing information of the same target among the multiple targets includes: determining the positional relationship between the preprocessed two-dimensional sensing information and the two-dimensional bounding box of the target; and determining the associated two-dimensional sensing information of the same target based on the positional relationship.

9. A three-dimensional target detection device, comprising: The determination module is used to process the image to determine the two-dimensional perceptual information of the target in the image; The acquisition module is used to acquire multiple sets of candidate 3D detection results of the target based on the 2D perception information; The fusion module is used to fuse the multiple sets of candidate 3D detection results to determine the final 3D detection result of the target. The final 3D detection result includes: final orientation angle, final 3D position information, and final 3D size information; each group of candidate 3D detection results includes: candidate orientation angle, candidate 3D position information, and candidate 3D size information; the fusion module is further used to: perform a weighted summation operation on multiple candidate orientation angles to determine the final orientation angle; perform an average operation on multiple candidate 3D position information to determine the final 3D position information; and perform an average operation on multiple candidate 3D size information to determine the final 3D size information.

10. The apparatus according to claim 9, wherein, The target is a vehicle; the two-dimensional perception information includes at least two of the following: vehicle body segmentation lines, wheel contact points, and passable area boundary points; the acquisition module is further used to: acquire a first candidate three-dimensional detection result based on the vehicle body segmentation lines; acquire a second candidate three-dimensional detection result based on the wheel contact points; and acquire a third candidate three-dimensional detection result based on the passable area boundary points.

11. The apparatus according to claim 10, wherein, The first candidate 3D detection result includes: a first candidate orientation angle, a first candidate 3D position information, and a first candidate 3D size information; the 2D perception information also includes: current category information and 2D position information; the acquisition module is further used to: determine the current ratio of the vehicle body dividing line to the target's 2D bounding box; determine the first candidate orientation angle based on the current ratio and the correspondence between the preset ratio and the orientation angle; determine the first candidate 3D size information based on the current category information and the correspondence between the preset category information and the 3D size information; and determine the first candidate 3D position information based on the 2D position information and the height information in the first candidate 3D size information.

12. The apparatus according to claim 10, wherein, The second candidate 3D detection result includes: a second candidate orientation angle, second candidate 3D position information, and second candidate 3D size information; there are two wheel contact points, and the acquisition module is further used to: perform inverse perspective transformation on the two wheel contact points to obtain two transformation points, and determine the line connecting the two transformation points; determine the second candidate orientation angle based on the orientation angle of the line, and determine the current wheelbase information of the target based on the length of the line; determine the second candidate 3D size information based on the current wheelbase information and the correspondence between preset wheelbase information and preset wheelbase information and 3D size information; and determine the second candidate 3D position information based on the position information of the two transformation points, the current wheelbase information, and the height information in the second candidate 3D size information.

13. The apparatus according to claim 10, wherein, The third candidate 3D detection result includes: a third candidate orientation angle, third candidate 3D position information, and third candidate 3D size information; the 2D perception information also includes: current category information; the acquisition module is further used to: perform rectangular box fitting processing on the boundary points of the passable area to obtain a fitted rectangular box; determine the third candidate orientation angle based on the rectangular box; determine the third candidate 3D size information based on the correspondence between the current category information and the preset category information and 3D size information; and determine the third candidate 3D position information based on the 2D position information of the rectangular box and the height information in the third candidate 3D size information.

14. The apparatus according to any one of claims 9-13, wherein, The target is multiple, and the acquisition module is further used to: perform data preprocessing on the two-dimensional sensing information to obtain preprocessed two-dimensional sensing information; The preprocessed two-dimensional sensing information is subjected to data association processing to obtain the associated two-dimensional sensing information of the same target among the multiple targets; Based on the associated two-dimensional perception information of the same target, multiple sets of candidate three-dimensional detection results of the same target are obtained.

15. The apparatus according to claim 14, wherein, The acquisition module is further configured to perform at least one of the following: perform time synchronization processing on the two-dimensional sensing information; perform spatial synchronization processing on the two-dimensional sensing information; and perform noise filtering processing on the two-dimensional sensing information.

16. The apparatus according to claim 14, wherein, The acquisition module is further configured to: determine the positional relationship between the preprocessed two-dimensional sensing information and the two-dimensional bounding box of the target; and, based on the positional relationship, determine the associated two-dimensional sensing information of the same target.

17. An electronic device comprising: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images