Target detection method and system for improving longitudinal perception distance and accuracy

Abstract

The application provides a target detection method and system for improving longitudinal perception distance and accuracy, the method comprising: acquiring image information and laser point cloud information of multiple targets in front, wherein the laser point cloud information covers a first distance in front, and the image information covers a second distance in front, the second distance being greater than the first distance; acquiring first position information of the multiple targets based on the image information; acquiring second position information of the multiple targets through a neural network based on the image information and the laser point cloud information; and fusing the second position information and the first position information to form third position information of the multiple targets. By combining the high accuracy of a short-range laser radar and the long-distance observation advantage of a camera, the laser radar high accuracy at a short distance is complemented and propagated to a long distance, and the perception distance and accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular to a target detection method and system for improving longitudinal perception distance and accuracy. BACKGROUND

[0002] The braking distance of a truck is longer, and compared with the braking distance of 40m of a passenger car, the braking distance of a truck exceeds 100m. Therefore, compared with the perception distance requirement of about 100m of a passenger car, the perception distance required by a truck is generally above 200m, or even reaches a kilometer level. The main solutions adopted by the prior art include: 1) based on monocular two-dimensional detection, combined with a traditional two-dimensional to three-dimensional lifting method, to obtain three-dimensional information of a long-distance obstacle; and 2) based on binocular vision to obtain three-dimensional information of a long-distance target. The deficiencies of the prior art mainly include: 1) monocular two-dimensional detection can realize two-dimensional detection of a long-distance object, but the two-dimensional to three-dimensional lifting method generally needs the premise assumption that the ground is flat, which is difficult to meet in actual scenes, resulting in low accuracy; 2) the ranging principle of binocular vision needs the left and right cameras to have a common viewing area of an obstacle, and the distance between the light centers of the left and right cameras (baseline) is proportional to the maximum measurement distance, in order to measure a longer distance, the left and right cameras with a larger distance between the light centers are needed, which is difficult to break through under the condition that the width of the vehicle is constant; 3) generally, the measurement error of a pure vision solution is enlarged exponentially with the distance, and it is difficult to guarantee the ranging accuracy of a long distance. It should be noted that “eye” in the above monocular and binocular refers to a camera, and the above vision refers to visible light imaging based on the camera. SUMMARY

[0003] In view of the problems of the prior art, the present application provides a target detection method and system for improving longitudinal perception distance and accuracy.

[0004] The target detection method for improving longitudinal perception distance and accuracy provided by the present application comprises the following steps:

[0005] obtaining image information and laser point cloud information of a plurality of targets in front, wherein the laser point cloud information covers a first distance in front, and the image information covers a second distance in front, the second distance being greater than the first distance;

[0006] based on the image information, obtaining first position information of the plurality of targets;

[0007] based on the image information and the laser point cloud information, obtaining second position information of the plurality of targets through a neural network;

[0008] fusing the second position information and the first position information to form third position information of the plurality of targets.

[0009] According to the target detection method for improving longitudinal perception distance and precision provided by the application, the method further comprises:

[0010] Obtaining the third position information of the plurality of targets in the current frame;

[0011] Obtaining the final position information of the plurality of targets in the previous frame;

[0012] Projecting the third position information of the plurality of targets in the current frame to the previous frame to form fourth position information;

[0013] Obtaining the re-projection error of the fourth position information and the final position information;

[0014] Based on the minimization of the re-projection error, correcting the third position information of the plurality of targets in the current frame to form the final position information of the plurality of targets in the current frame.

[0015] According to the target detection method for improving longitudinal perception distance and precision provided by the application, the neural network is supervised and corrected based on the principle that the laser point cloud information is more accurate than the image information when training the position information of the targets within the first distance.

[0016] According to the target detection method for improving longitudinal perception distance and precision provided by the application, when training the neural network, a regularization constraint is set in the loss function, and the position information of the plurality of targets outside the first distance and within the second distance is supervised and corrected in the same way as the position information of the plurality of targets within the first distance.

[0017] According to the target detection method for improving longitudinal perception distance and precision provided by the application, the neural network comprises a neural network based on an attention mechanism.

[0018] According to the target detection method for improving longitudinal perception distance and precision provided by the application, the fusion of the second position information and the first position information to form the third position information of the plurality of targets comprises:

[0019] Obtaining a Kalman filter;

[0020] Inputting the second position information and the first position information into the Kalman filter, and outputting the third position information by the Kalman filter.

[0021] The application further provides a target detection system for improving longitudinal perception distance and precision, and the system comprises:

[0022] An acquisition module is configured to acquire image information and laser point cloud information of a plurality of targets in front, wherein the laser point cloud information covers a first distance in front, and the image information covers a second distance in front, the second distance being greater than the first distance.

[0023] A first perception module is configured to acquire first position information of the plurality of targets based on the image information.

[0024] A second perception module is configured to acquire second position information of the plurality of targets by a neural network based on the image information and the laser point cloud information.

[0025] A fusion module is configured to fuse the second position information and the first position information to form third position information of the plurality of targets.

[0026] The application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the target detection method for improving longitudinal perception distance and accuracy according to any one of the above.

[0027] The application further provides an autonomous truck, including the electronic device.

[0028] The application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the steps of the target detection method for improving longitudinal perception distance and accuracy according to any one of the above.

[0029] The application provides a target detection method and system for improving longitudinal perception distance and accuracy, which combines the high precision of a short-range laser radar and the long-distance observation advantage of a camera, forms a complement, propagates the high precision of the short-range laser radar to a long distance, and improves the perception distance and accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the present application or prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0031] Figure 1 A flowchart of a target detection method for improving longitudinal perception distance and accuracy provided by the present application;

[0032] Figure 2 An optical imaging diagram for acquiring target position information based on image information provided by the present application;

[0033] Figure 3 A target detection system structure diagram for improving longitudinal perception distance and precision is provided in the present application.

[0034] Figure 4 An entity structure diagram of an electronic device is provided in the present application. DETAILED DESCRIPTION

[0035] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described below in connection with the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0036] The target detection method for improving longitudinal perception distance and precision provided by the embodiments of the present application will be described in detail below in connection with the drawings, specific embodiments and application scenarios.

[0037] Figure 1 A flow diagram of the target detection method for improving longitudinal perception distance and precision provided in the present application is shown in FIG. 1, and the target detection method for improving longitudinal perception distance and precision provided in the present application can include the following steps. Figure 1

[0038] It should be noted that the present method can be applied to a vehicle to detect obstacle information in front of the vehicle. Further, the present method is more suitable for vehicles for automatic driving, which are equipped with a camera and a laser radar to detect obstacles and form detection data in multiple dimensions, and based on the detection data, higher-precision position information of the obstacles in front is obtained.

[0039] Preferably, the present method can be directly run on the vehicle, or only detection data can be obtained through the vehicle, the vehicle uploads the detection data to a remote server, the remote server analyzes and processes the detection data, and the remote server transmits the processing result to the vehicle to achieve the detection purpose. In addition, the detection data can also be obtained through other vehicles or roadside terminals connected to the vehicle.

[0040] S100, image information and laser point cloud information of a plurality of targets in front are obtained, wherein the laser point cloud information covers a first distance in front, and the image information covers a second distance in front, and the second distance is greater than the first distance.

[0041] Preferably, the targets include vehicles, obstacles, traffic road devices, personnel, ground, etc.

[0042] ​Preferably, the laser point cloud information comes from a vehicle-mounted laser radar, which has a short detection distance and high detection accuracy; and the image information comes from a vehicle-mounted camera, which has a long detection distance and low detection accuracy.

[0043] S200, acquiring first position information of the multiple targets based on the image information.

[0044] It should be noted that the first position information of the multiple targets based on the image information can be understood as judging the depth of field information of each target in the image through the image information. At this time, the judgment is purely based on the image, and the accuracy is relatively low. In particular, the accuracy of the target position judgment at a distance outside the first distance and within the second distance is lower, and is easily affected by other interference factors.

[0045] Preferably, Figure 2 An optical imaging schematic diagram for acquiring target position information based on image information is provided in the present application, as shown in Figure 2 A camera P is arranged on the vehicle A, and the image information of the targets B and C in front of the vehicle A is acquired through the camera. The focal length of the camera P is f, the height is H, the distance between the camera P and the targets B / C is Z1 / Z2, and the coordinate of the projection point of the imaging plane I of the camera P on the y-axis of the coordinate system is y1 / y2. Based on the principle of similar triangles, y=fH / Z, and further transformation is Z=fH / y, wherein y is the universalization of y1 / y2, and Z is the universalization of Z1 / Z2. Correspondingly, y is the image information, and Z is the first position information.

[0046] S300, acquiring second position information of the multiple targets based on the image information and the laser point cloud information through a neural network.

[0047] Optionally, during the training of the neural network, the position information of the targets within the first distance is supervised and corrected based on the principle that the laser point cloud information is more accurate than the image information.

[0048] Preferably, the difference between the target distance prediction value output by the regression network and the target distance measurement value measured by the laser radar within the first distance forms a supervised loss function, and the supervised correction is performed.

[0049] Optionally, during the training of the neural network, a regularization constraint is set in the loss function, and the position information of the multiple targets outside the first distance and within the second distance is supervised and corrected in the same way as the position information of the multiple targets within the first distance.

[0050] Preferably, the first distance is supervised by laser information, and the first distance is more accurate than a range within a second distance.

[0051] Optionally, the neural network comprises an attention-based neural network.

[0052] It should be noted that the laser point cloud information is fused into the network by the attention-based neural network, and the target within the first distance is supervised. Due to the high precision of the laser point cloud information, the neural network can correct the target position information within the first distance with high precision. At the same time, the consistency constraint of the point cloud in the far and near places in the loss function is designed to regularize the network, so that the aforementioned high-precision correction is applied to the range outside the first distance and within the second distance, so that the target position information determined by the image information in the far place can also be corrected with high precision by the laser radar. This is achieved by virtualizing the neural network.

[0053] S400, fuse the second position information and the first position information to form third position information of a plurality of targets.

[0054] Optionally, fusing the second position information and the first position information to form third position information of a plurality of targets comprises:

[0055] Obtain a Kalman filter;

[0056] Input the second position information and the first position information into the Kalman filter, and output the third position information from the Kalman filter.

[0057] It should be noted that the second position information and the first position information are fused by the Kalman filter, so that the target position information contained in the image information and the target position information contained in the laser point cloud information can contribute to the final target position information, thereby improving the accuracy of the target position information.

[0058] The present embodiment combines the high precision of the short-range laser radar and the long-range advantage of the camera to form a complement, and propagates the high precision of the short-range laser radar to a long distance, thereby improving the perception distance and precision.

[0059] On the basis of the foregoing embodiment, in another embodiment, the method further comprises:

[0060] Obtain third position information of a plurality of targets in a current frame;

[0061] Obtain final position information of a plurality of targets in a previous frame;

[0062] The third position information of multiple targets in the current frame is projected onto the previous frame to form the fourth position information;

[0063] The reprojection error between the fourth position information and the final position information is obtained;

[0064] Based on minimizing the reprojection error, the third position information of multiple targets in the current frame is corrected to form the final position information of multiple targets in the current frame.

[0065] It should be noted that, based on the motion of the vehicle and the motion of the monitored target within the previous and next frames, the relative motion between the vehicle and the target can be obtained. Based on this relative motion and the target position information of the current frame, the target position information of the previous frame can be deduced, thereby enabling the projection of the third position information of multiple targets in the current frame onto the previous frame.

[0066] Preferably, as a parallel approach, after obtaining the third position information of multiple targets in the current frame and the final position information of multiple targets in the previous frame, the final position information of multiple targets in the previous frame can be projected onto the current frame to form the fifth position information. Then, based on minimizing the reprojection error as described above, the third position information is corrected to form the final position information of multiple targets in the current frame.

[0067] Preferably, the final position information within the frame from two or three earlier times can be selected, and the target position information can be corrected multiple times by minimizing the reprojection error.

[0068] This embodiment utilizes the continuity of the target's relative position to the vehicle in space between consecutive frames. By employing a reprojection error optimizer, it corrects the target position information in the current frame, adding a new dimension and further improving the accuracy of the target position information.

[0069] The target detection system for improving longitudinal sensing distance and accuracy provided by the present invention is described below. The target detection system for improving longitudinal sensing distance and accuracy described below can be referred to in correspondence with the target detection method for improving longitudinal sensing distance and accuracy described above.

[0070] Figure 3 This invention provides a schematic diagram of a target detection system structure that improves longitudinal sensing distance and accuracy. Figure 3 As shown, the present invention also provides a target detection system for improving longitudinal sensing distance and accuracy, the system comprising:

[0071] The acquisition module is used to acquire image information and laser point cloud information of multiple targets in front. The laser point cloud information covers a first distance in front, and the image information covers a second distance in front, where the second distance is greater than the first distance.

[0072] The first perception module is used to obtain the first location information of multiple targets based on image information;

[0073] The second perception module is used to obtain the second position information of multiple targets through a neural network based on image information and laser point cloud information.

[0074] The fusion module is used to fuse the second location information and the first location information to form the third location information of multiple targets.

[0075] This embodiment combines the high precision of near-range lidar with the long-range capability of a camera, creating a complementary effect. It transmits the high precision of near-range lidar to long-range applications, thereby improving both sensing distance and accuracy.

[0076] Figure 4 A schematic diagram of the physical structure of an electronic device provided by the present invention, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a target detection method that improves longitudinal sensing distance and accuracy, the method including:

[0077] Acquire image information and laser point cloud information of multiple targets in front, wherein the laser point cloud information covers a first distance in front, the image information covers a second distance in front, and the second distance is greater than the first distance;

[0078] Based on the image information, the first location information of the plurality of targets is obtained;

[0079] Based on the image information and the laser point cloud information, the second position information of the multiple targets is obtained through a neural network;

[0080] The second location information and the first location information are fused together to form the third location information of the plurality of targets.

[0081] In addition, the logic instructions in the memory 430 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0082] In another aspect, the present application also provides a truck for automatic driving, which comprises the electronic device described above.

[0083] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions, when the program instructions are executed by a computer, the computer can execute the target detection method for improving longitudinal perception distance and accuracy provided by the above-mentioned methods, and the method comprises:

[0084] obtaining image information and laser point cloud information of a plurality of targets in front, wherein the laser point cloud information covers a first distance in front, and the image information covers a second distance in front, and the second distance is greater than the first distance;

[0085] obtaining first position information of the plurality of targets based on the image information;

[0086] obtaining second position information of the plurality of targets by a neural network based on the image information and the laser point cloud information;

[0087] fusing the second position information and the first position information to form third position information of the plurality of targets.

[0088] In another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the target detection method for improving longitudinal perception distance and accuracy provided by the above-mentioned methods, and the method comprises:

[0089] Obtain image information and laser point cloud information of a plurality of targets in front, wherein the laser point cloud information covers a first distance in front, and the image information covers a second distance in front, the second distance being greater than the first distance;

[0090] Obtain first position information of the plurality of targets based on the image information;

[0091] Obtain second position information of the plurality of targets based on the image information and the laser point cloud information through a neural network;

[0092] Fuse the second position information and the first position information to form third position information of the plurality of targets.

[0093] The above-described device embodiments are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0094] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary general hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0095] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A target detection method for improving longitudinal perception distance and accuracy, characterized in that, The method comprises: acquiring image information and laser point cloud information of a plurality of targets in front, wherein the laser point cloud information covers a first distance in front, and the image information covers a second distance in front, the second distance being greater than the first distance; acquiring first position information of the plurality of targets based on the image information; acquiring second position information of the plurality of targets based on the image information and the laser point cloud information through a neural network, wherein the neural network comprises a neural network based on an attention mechanism; fusing the second position information and the first position information to form third position information of the plurality of targets; the method further comprises: acquiring the third position information of the plurality of targets in a current frame; acquiring final position information of the plurality of targets in a previous frame; projecting the third position information of the plurality of targets in the current frame to the previous frame to form fourth position information; acquiring a re-projection error of the fourth position information and the final position information; correcting the third position information of the plurality of targets in the current frame based on minimization of the re-projection error to form final position information of the plurality of targets in the current frame; during training of the neural network, the position information of the targets within the first distance is supervised and corrected based on the principle that the laser point cloud information is more accurate than the image information; during training of the neural network, a regularization constraint is set in a loss function, and the position information of the plurality of targets outside the first distance and inside the second distance is supervised and corrected in a manner consistent with the position information of the plurality of targets within the first distance; the fusing of the second position information and the first position information to form the third position information of the plurality of targets comprises: acquiring a Kalman filter; inputting the second position information and the first position information into the Kalman filter, and outputting the third position information by the Kalman filter.

2. A target detection system that improves longitudinal perceptual range and accuracy, characterized by, The system comprises: an acquisition module configured to acquire image information and laser point cloud information of a plurality of targets in front, wherein the laser point cloud information covers a first distance in front, and the image information covers a second distance in front, the second distance being greater than the first distance; a first perception module configured to acquire first position information of the plurality of targets based on the image information; a second perception module configured to acquire second position information of the plurality of targets based on the image information and the laser point cloud information through a neural network, wherein the neural network comprises a neural network based on an attention mechanism; a fusion module configured to fuse the second position information and the first position information to form third position information of the plurality of targets; The system further comprises a correction module configured to: obtain the third position information of the plurality of targets in the current frame; obtain final position information of the plurality of targets in a previous frame; project the third position information of the plurality of targets in the current frame to the previous frame to form fourth position information; obtain a re-projection error of the fourth position information and the final position information; and correct the third position information of the plurality of targets in the current frame based on minimization of the re-projection error to form final position information of the plurality of targets in the current frame. During training of the neural network, the position information of the targets within the first distance is supervised and corrected based on the principle that the laser point cloud information is more accurate than the image information. During training of the neural network, a regularization constraint is set in a loss function, and the position information of the plurality of targets outside the first distance and within the second distance is supervised and corrected to be consistent with the position information of the plurality of targets within the first distance. The fusion module is further configured to obtain a Kalman filter, input the second position information and the first position information into the Kalman filter, and output the third position information from the Kalman filter.

3. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the target detection method for improving longitudinal perception distance and accuracy according to claim 1.

4. An autonomously driving truck, characterized in that The truck comprises the electronic device according to claim 3.

5. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the target detection method for improving longitudinal perception distance and accuracy according to claim 1. The computer program is executed by the processor to implement the steps of the target detection method for improving longitudinal perception distance and accuracy according to claim 1.

Images