Target detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a target detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a first target detection result, obtained through all-round target detection, of an environment where a vehicle is located, and obtaining a second target detection result, obtained through monocular target detection, of the environment where the vehicle is located; performing target tracking on a first target in the first target detection result to obtain a first tracking library, and performing target tracking on a second target in the second target detection result to obtain a second tracking library; determining a second target of which the matching rate with the first target is greater than a second threshold value from a second tracking library as a matching target, and determining a second target of which the tracking life cycle meets a preset condition in the second targets except the matching target as a candidate target; and obtaining a third target detection result corresponding to the vehicle according to the first target, the matching target and the candidate target, thereby obtaining a detection result of a long-distance target, and improving the accuracy of target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and more particularly, to a target detection method and device, an electronic device, and a storage medium. BACKGROUND

[0002] With the development of science and technology, the automatic driving technology has made great progress. Target detection and tracking are the core links of the technology, and therefore, the requirement for the accuracy of vehicle target detection is also higher and higher. SUMMARY

[0003] In view of the above, the embodiments of the present application provide a target detection method, device, electronic device, and storage medium to improve the above problems.

[0004] In a first aspect, the embodiments of the present application provide a target detection method, which comprises: obtaining a first target detection result of an environment where a vehicle is located by surround view target detection, and obtaining a second target detection result of the environment where the vehicle is located by monocular target detection; performing target tracking on a first target in the first target detection result to obtain a first tracking library, and performing target tracking on a second target in the second target detection result to obtain a second tracking library, the distance between the first target and the second target and the vehicle being greater than a first threshold; determining, from the second tracking library, a second target whose matching rate with the first target is greater than a second threshold as a matching target, and determining a second target whose tracking life cycle meets a preset condition as a candidate target, the second target being other than the matching target; and obtaining a third target detection result corresponding to the vehicle according to the first target, the matching target, and the candidate target.

[0005] In a second aspect, an embodiment of the present application provides a target detection device, the device comprising: a surround detection module, a tracking library establishing module, a cross matching module, and a target fusion module. The surround detection module is configured to obtain a first target detection result of an environment in which a vehicle is located by surround target detection, and obtain a second target detection result of the environment in which the vehicle is located by monocular target detection. The tracking library establishing module is configured to perform target tracking on first targets in the first target detection result to obtain a first tracking library, and perform target tracking on second targets in the second target detection result to obtain a second tracking library, the distance between the first targets and the second targets and the vehicle being greater than a first threshold. The cross matching module is configured to determine, from the second tracking library, second targets whose matching rate with the first targets is greater than a second threshold as matching targets, and determine second targets whose tracking life cycle meets a preset condition as candidate targets, the second targets being other than the matching targets. The target fusion module is configured to obtain a third target detection result corresponding to the vehicle according to the first targets, the matching targets, and the candidate targets.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, the memory being coupled to the processor, and the memory storing instructions, when the instructions are executed by the processor, the processor executes the target detection method provided in the first aspect.

[0007] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium storing program codes, the program codes being executable by a processor to execute the target detection method provided in the first aspect.

[0008] In the scheme of the present application, the first target detection result of an environment in which a vehicle is located obtained by surround target detection, and the second target detection result of the environment in which the vehicle is located obtained by monocular target detection are acquired, the first targets in the first target detection result whose distance from the vehicle is greater than a first threshold are tracked to obtain a first tracking library, the second targets in the second target detection result whose distance from the vehicle is greater than the first threshold are tracked to obtain a second tracking library, the second targets whose matching rate with the first targets is greater than a second threshold are determined from the second tracking library as matching targets, the second targets other than the matching targets whose tracking life cycle meets a preset condition are determined as candidate targets, and the third target detection result corresponding to the vehicle is obtained according to the first targets, the matching targets, and the candidate targets, so that the long-distance targets in the surround target detection result and the monocular target detection result whose distance from the vehicle is greater than the first threshold are fused and tracked to obtain the detection result of the long-distance targets, and the accuracy of target detection is improved. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0010] Figure 1 A flowchart of a target detection method provided by an embodiment of the present application is shown; Figure 2 A flowchart of image processing provided by an embodiment of the present application is shown; Figure 3 A flowchart of target tracking provided by an embodiment of the present application is shown; Figure 4 A flowchart of a target detection method provided by an embodiment of the present application is shown; Figure 5 A module block diagram of a target detection device provided by an embodiment of the present application is shown; Figure 6 A block diagram of an electronic device for executing the target detection method according to the embodiments of the present application is shown. DETAILED DESCRIPTION

[0011] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0012] In order to better understand the solutions of the embodiments of the present application, the technical terms used in the embodiments of the present application will be explained first.

[0013] Bird's Eye View (BEV) is a technology for converting a traditional two-dimensional image into a three-dimensional overhead view through an algorithm.

[0014] The implementation details of the technical solutions of the embodiments of the present application will be described in detail as follows: In the field of autonomous driving, the most common target detection can be classified into surround view BEV detection, monocular 3D detection and 2D target detection. The surround view BEV detection generally uses the images of seven surround view cameras as input to provide the vehicle with the information of obstacles around the vehicle, including the position, number, moving direction, speed and the like of the obstacles, so as to provide the vehicle with the obstacle avoidance information for intelligent driving planning. However, the surround view BEV detection has the disadvantage that it cannot detect small targets at a long distance (such as more than 120 meters) well. The monocular 3D detection uses a single camera image as input and directly detects the target in the image and estimates the actual depth of the target. Since the image has the imaging feature of an infinite distance target in theory, the image is usually used for long-distance target detection. However, the monocular 3D detection also has the disadvantage that the depth estimation of a long-distance target jumps greatly. The 2D target detection is usually used for detecting static targets such as traffic lights and signs and estimating the actual position of the target by using the IPM projection method. The 2D target detection is mainly used for target detection with low accuracy requirement for the actual position of the target.

[0015] In the related art, a scheme is proposed for calculating the 3D size and rotation angle of an actual target by estimating the depth of the center position of the target and the length and width of the target and combining the intrinsic parameters. However, since the image lacks effective target depth information, the estimation of the target position depth based on the image has great uncertainty. The farther the distance (the smaller the imaging pixel), the greater the depth jump of the estimation. In addition, in the related art, there is also a scheme for detecting targets within a range of about 100 meters around the vehicle environment by using the input surround view 7-view images. However, the detection of small targets at a long distance (such as more than 120 meters) also has the problem of great jump.

[0016] Therefore, in the related art, there is a challenge of accurately detecting small targets at a long distance.

[0017] To solve the above problems, the inventors have found, after long-term research, the target detection method, device and electronic equipment provided in the embodiments of the present application. The target detection method, device and electronic equipment provided in the embodiments of the present application fuse and track the long-distance targets with a distance greater than a first threshold between the long-distance targets and the vehicle in the surround view target detection result and the monocular target detection result, to obtain the detection result of the long-distance target, thereby improving the accuracy of target detection. The specific target detection method is described in detail in subsequent embodiments.

[0018] The embodiments related to the present application will be described below with reference to the accompanying drawings.

[0019] Please refer to Figure 1 , Figure 1 Fig. 1 shows a flowchart of a target detection method according to an embodiment of the present application. In specific embodiments, the target detection method can be applied to a target detection device 200 as shown in Fig. 2 and an electronic equipment 100 configured with the target detection device 200 (as shown in Fig. 3). Figure 5 ​Figure 6 The specific flow of the present embodiment will be described below with an electronic device as an example. It should be understood that the electronic device to which the present embodiment is applied can include a vehicle, a vehicle-mounted terminal, and the like, which are not limited herein. The target detection method can include the following steps, which will be described in detail below with reference to the flow shown in Figure 1 Step S110: Obtain a first target detection result of the environment in which the vehicle is located obtained through surround view target detection, and obtain a second target detection result of the environment in which the vehicle is located obtained through monocular target detection.

[0020] In the present embodiment, the electronic device can be understood as a vehicle (the vehicle). One or more cameras can be provided in the vehicle, which can be used to collect images of the environment in which the vehicle is located. For example, the vehicle can include seven cameras, which can include 7-way cameras provided at the front wide, front narrow, left front, left rear, right front, right rear, and rear of the vehicle. The vehicle can collect images of the environment in which the vehicle is located based on the seven cameras. The vehicle can obtain the result of surround view target detection of the environment in which the vehicle is located according to the images collected by the seven cameras.

[0021] The vehicle can obtain a first target detection result of the environment in which the vehicle is located obtained through surround view target detection. The vehicle can be pre-provided with a surround BEV algorithm model (e.g., BEV Former algorithm, etc.). Based on this, after the vehicle obtains the images collected by the seven cameras, the vehicle can input the images into the surround BEV algorithm model, perform surround view target detection through the surround BEV algorithm model, and obtain the first target detection result of the environment in which the vehicle is located output by the surround BEV algorithm model.

[0022] The first target detection result can be understood as information of surround view targets around the vehicle in the environment in which the vehicle is located. The first target detection result can include at least one target (e.g., a vehicle, a pedestrian, a road sign, etc.) and position information and motion information corresponding to each target. For example, the first target detection result can include targets { , , } and position information ​={x, y, z, w, l, h, yaw, vx, vy}. Wherein, n can represent the total number of detected targets; wherein, x, y, z can represent the position of the center point of the target in the vehicle coordinate system; wherein, w can represent the width corresponding to the target, l can represent the length corresponding to the target, and h can represent the height corresponding to the target; wherein, yaw can represent the direction of the target, vx can represent the directional velocity of the target in the X-axis of the vehicle coordinate system, and vy can represent the directional velocity of the target in the Y-axis of the vehicle coordinate system.

[0023] In some embodiments, it is considered that there is a situation where the target detection distance is close in the surround view target detection process. Based on this, in the present embodiment, the vehicle can also obtain the image collected by the front-view camera, and obtain the second target detection result of monocular target detection of the environment in which the vehicle is located according to the image, so as to detect small targets at a long distance through multiple perception sources, and improve the accuracy of target detection.

[0024] Wherein, a monocular 3D algorithm model (such as an FCOS3D algorithm model, etc.) can be pre-set in the vehicle, which can be used to detect the 3D position of the target on the image. Based on this, after the vehicle obtains the image collected by the front-view camera (such as a front-view FOV30 degree camera, etc.), the image can be input into the monocular 3D algorithm model, monocular target detection is performed through the monocular 3D algorithm model, and the second target detection result of the environment in which the vehicle is located output by the monocular 3D algorithm model is obtained.

[0025] Wherein, the second target detection result can be understood as the 3D information of the target in the front-view direction of the vehicle. Wherein, the second target information can include at least one target (such as a vehicle, a pedestrian, a road sign, etc.) and the position information and motion information corresponding to each target. For example, the second target detection result includes the target ={ , ,..., } and the position information of the target ={u, v, depth, w, h, l, vx, vy, yaw}. Wherein, m can represent the total number of detected targets; wherein, u, v can represent the pixel position of the center point of the target in the image in the camera coordinate system, w can represent the width corresponding to the target, l can represent the length corresponding to the target, and h can represent the height corresponding to the target; wherein, yaw can represent the direction of the target, vx can represent the directional velocity of the target in the X-axis of the camera coordinate system, and vy can represent the directional velocity of the target in the Y-axis of the camera coordinate system. Wherein, depth can represent the depth of the center point of the target from the camera position. Optionally, the vehicle can combine the depth of , the intrinsic and extrinsic parameters of the camera, and Convert to the vehicle coordinate system. Thus, the target obtained by the second target detection result is in the same coordinate system as the target obtained by the first target detection result, which facilitates the fusion of the targets detected by multiple perception sources and improves the efficiency of target detection.

[0026] For example, refer to Figure 2 which shows a flowchart of image processing provided by an embodiment of the present application. After obtaining the images captured by the seven cameras, the vehicle can perform feature extraction on the images corresponding to the seven viewing angles of the vehicle (7V images), and can input the extracted features of the seven viewing angle images into the BEV module of the vehicle, and perform surround view target detection through the surround view BEV module to obtain the first target detection result of the environment in which the vehicle is located. The vehicle can also perform front view image feature extraction on the front view images in the 7V images through the front view image feature extraction module, and input the extracted front view image features into the Mono3D module of the vehicle, and can perform monocular target detection of the front view through the Mono3D module to obtain the second target detection result of the environment in which the vehicle is located. The first target detection result and the second target detection result are fused by the fusion module in the subsequent vehicle to output the third target detection result.

[0027] Step S120: performing target tracking on the first target in the first target detection result to obtain a first tracking library, and performing target tracking on the second target in the second target detection result to obtain a second tracking library, the distance between the first target and the second target and the vehicle being greater than a first threshold.

[0028] In some embodiments, after the vehicle obtains the first target detection result, the vehicle can filter the targets included in the first target detection result to obtain the first target whose distance to the vehicle is greater than the first threshold. After the vehicle obtains the second target detection result, the vehicle can filter the targets included in the second target detection result to obtain the second target whose distance to the vehicle is greater than the first threshold. Thus, the targets in the close range are filtered, and the recall rate of the long-distance target is improved.

[0029] The first threshold can be set in the vehicle in advance, which can be set by the user or obtained through third-party experimental data, and is not limited herein. For example, the first threshold can include 100m, 110m, 120m, etc. Alternatively, the vehicle can also obtain the first target and the second target from the first target detection result and the second target detection result based on the height of the target, the lateral distance between the target and the vehicle, etc.

[0030] For example, the target included in the first target detection result is , and the target included in the second target detection result is In the process of filtering the targets included in the first target detection result and the second target detection result to obtain the first target and the second target, the target filtering can be performed according to a filtering formula to obtain the first target and the second target. The filtering formula includes: = filter (x > 100, -15 < y < 15, -1 < z < 5) = filter (x > 100, -15 < y < 15, -1 < z < 5) = filter (x > 100, -15 < y < 15, -1 < z < 5) condition = (x > 100, -15 < y < 15, -1 < z < 5) x > 100 can represent that the target beyond 100 m in front of the host vehicle is obtained, -15 < y < 15 can represent that the target within 15 m from the leftmost point and the rightmost point of the host vehicle is obtained, and -1 < z < 5 can represent that the target within the range from 5 m above the center point of the host vehicle to 1 m below the center point of the host vehicle is obtained.

[0031] Step S130: determining, from the second tracking library, a second target with a matching rate greater than a second threshold to the first target as a matching target, and determining a second target with a tracking life cycle satisfying a preset condition from the second targets other than the matching target as a candidate target.

[0032] In some embodiments, after the host vehicle obtains the first target and the second target, the host vehicle can track the first target and the second target respectively to obtain a first tracking library and a second tracking library. The first tracking library can include information of tracking the first target in the first target detection result, and the second tracking library can include information of tracking the second target in the second target detection result. For example, the first tracking library can include tracking state (such as active state, tracking state, lost state, death state, etc.), position information, direction information, speed information, appearance feature information, etc. of the first target, and the second tracking library can include tracking state (such as active state, tracking state, lost state, death state, etc.), position information, direction information, speed information, appearance feature information, etc. of the first target.

[0033] For example, refer to Figure 3 ​Fig. 1 shows a flowchart of a target tracking method according to an embodiment of the present application. As shown in Fig. 1, the vehicle can obtain a first target detection result and a second target detection result, filter the targets in the first target detection result and the second target detection result based on a filtering module, obtain a first target and a second target with a distance greater than a first threshold, fuse the long-distance targets, improve the accuracy of long-distance target detection, track the first target by a first tracking module and track the second target by a second tracking module, obtain a first tracking library and a second tracking library, fuse the long-distance targets in the first tracking library and the second tracking library by a subsequent cross fusion module, improve the recall rate of long-distance target detection, and plan an automatic driving trajectory to avoid obstacles, thereby improving the safety of vehicle driving.

[0034] Step S130: determining a second target with a matching rate greater than a second threshold as a matching target and determining a second target with a tracking life cycle satisfying a preset condition as a candidate target from the second tracking library.

[0035] In some embodiments, after obtaining the first tracking library and the second tracking library, the vehicle can determine a second target with a matching rate greater than a second threshold as a matching target from the second tracking library. The vehicle can match each first target with each second target in the second tracking library, obtain a matching rate between each first target and each second target, and obtain a second target with a matching rate greater than a second threshold as a matching target from the second tracking library for any first target, thereby obtaining the matched targets in the monocular target detection result and the surround view target detection result.

[0036] The matching rate of the first target and the second target can include a similarity of position information of the first target and the second target in the vehicle coordinate system, a similarity of motion direction of the first target and the second target, a similarity of appearance of the first target and the second target, etc.

[0037] In some embodiments, after determining the matching target from the second tracking library, the vehicle can determine a second target with a tracking life cycle satisfying a preset condition as a candidate target from the second tracking library. The preset condition can include a tracking life cycle greater than a preset period, thereby obtaining a target from the monocular target detection result that is not matched with the surround view target detection result but has a tracking period greater than a preset period, and improving the effectiveness of long-distance target detection.

[0038] Step S140: obtaining a third target detection result corresponding to the vehicle according to the first target, the matching target, and the candidate target.

[0039] In some embodiments, after the ego vehicle determines the first target, the matching target and the candidate target, the ego vehicle can obtain a third target detection result corresponding to the ego vehicle according to the first target, the matching target and the candidate target. The ego vehicle can exclude the matching target from the first target to obtain a target that is not matched under the surround-view BEV detection and the monocular target detection, thereby reducing the influence on the surround-view target detection result and supplementing the candidate target output to obtain a small target at a long distance, so as to combine the Mono3D long-distance target detection result and the surround-view BEV target detection result for association matching and fusion, output the small target at a long distance, and effectively improve the recall rate of the small target at a long distance, thereby providing a safety guarantee for vehicle automatic driving.

[0040] The ego vehicle can fuse the result of excluding the matching target from the first target with the candidate target to obtain the third target detection result corresponding to the ego vehicle. Alternatively, the ego vehicle can fuse the position information of the target obtained by excluding the matching target from the first target with the position information of the candidate target to obtain the third target test result. Alternatively, the ego vehicle can combine the detection box corresponding to the result of excluding the matching target from the first target with the detection box corresponding to the candidate target to obtain the third target detection result.

[0041] The target detection method provided by an embodiment of the present application obtains a first target detection result of an environment in which an ego vehicle is located through surround-view target detection, and obtains a second target detection result of the environment in which the ego vehicle is located through monocular target detection. The target detection method performs target tracking on a first target in the first target detection result that is greater than a first threshold from the ego vehicle to obtain a first tracking library, and performs target tracking on a second target in the second target detection result that is greater than the first threshold from the ego vehicle to obtain a second tracking library. The target detection method determines a second target in the second tracking library that has a matching rate greater than a second threshold with the first target as a matching target, and determines a second target in the second target detection result that has a tracking life cycle satisfying a preset condition as a candidate target. The target detection method obtains a third target detection result corresponding to the ego vehicle according to the first target, the matching target and the candidate target, thereby fusing and tracking a long-distance target in the surround-view target detection result and the monocular target detection result that is greater than the first threshold from the ego vehicle to obtain a detection result of the long-distance target, improving the accuracy of target detection, avoiding obstacles for vehicle driving, and improving the safety of vehicle driving.

[0042] Please refer to Figure 4 , Figure 4 The flowchart of the target detection method provided by an embodiment of the present application is shown. The method is applied to the electronic device described above, and the following will be described in detail with respect to the flowchart shown in Figure 4 The target detection method can specifically include the following steps: Step S201: Obtain a first target detection result of an environment in which the vehicle is located obtained through surround target detection, and obtain a second target detection result of the environment in which the vehicle is located obtained through monocular target detection.

[0043] Step S202: Perform target tracking on first targets in the first target detection result to obtain a first tracking library, and perform target tracking on second targets in the second target detection result to obtain a second tracking library, distances between the first targets and the second targets and the vehicle being greater than a first threshold.

[0044] For specific descriptions of steps S201-S202, refer to the descriptions of steps S110-S120 above, which will not be repeated here.

[0045] Step S203: Obtain position information of a currently detected target to be matched as to-be-matched position information.

[0046] In some embodiments, after obtaining the first tracking library and the second tracking library, the vehicle can obtain an image captured by a camera of the vehicle in real time, and can obtain position information of a currently detected target to be matched as to-be-matched position information according to the image. The number of targets to be matched can be one or more, and the vehicle can obtain to-be-matched position information corresponding to each target to be matched.

[0047] The to-be-matched position information corresponding to the target to be matched can include position information of a detection box corresponding to the target to be matched in a vehicle coordinate system, such as length, width, height, and direction of the detection box.

[0048] Step S204: Obtain targets in the first tracking library and the second tracking library in an activated state as targets to be updated.

[0049] In some embodiments, after obtaining the first tracking library and the second tracking library, the vehicle can obtain targets in the first tracking library and the second tracking library in an activated state as targets to be updated. The number of targets to be updated can be one or more.

[0050] Step S205: Perform position prediction on the targets to be updated to obtain predicted position information.

[0051] In some embodiments, after determining the targets to be updated, the vehicle can perform position prediction on the targets to be updated to obtain predicted position information. If the number of targets to be updated is greater than one, the vehicle can obtain predicted position information corresponding to each target to be updated.

[0052] The predicted position information corresponding to the target to be updated can include a predicted position of the target to be updated in the vehicle coordinate system, such as a length, a width, a height, a direction, and the like of the predicted detection box.

[0053] In some embodiments, the ego vehicle can predict the position of the target to be updated based on a Kalman filter to obtain the predicted position information corresponding to the target to be updated. The ego vehicle can predict the trajectory of the target to be updated based on the Kalman filter to obtain the predicted position information corresponding to the target to be updated.

[0054] Step S206: If the predicted position information matches the matching position information, the position information corresponding to the target to be updated is updated based on the matching position information.

[0055] In some embodiments, after obtaining the predicted position information, the ego vehicle can match the matching position information corresponding to the target to be matched with the predicted position information corresponding to the target to be updated, and determine whether to update the first tracking library and / or the second tracking library according to the matching result. For example, if it is determined that the matching position information corresponding to the target to be matched matches the predicted position information corresponding to the target to be updated, the position information corresponding to the target to be updated can be updated based on the matching position information.

[0056] In some embodiments, the ego vehicle can obtain the overlap between the predicted position information and the matching position information based on the IoU algorithm, and if it is determined that the overlap is greater than a fourth threshold, it can be determined that the predicted position information matches the matching position information. The fourth threshold can be pre-set in the ego vehicle, can be set by the user, or can be obtained through third-party experimental data. For example, the fourth threshold can be obtained through third-party experimental data and is 0.9, 0.8, and the like.

[0057] The ego vehicle can obtain the overlap between the detection box included in the predicted position information in the vehicle coordinate system and the detection box included in the matching position information in the vehicle coordinate system through the IoU algorithm, and can determine that the predicted position information matches the matching position information if it is determined that the overlap is greater than the fourth threshold.

[0058] The ego vehicle can match the matching position information with the predicted position information corresponding to the target to be updated included in the first tracking library and with the predicted position information corresponding to the target to be updated included in the second tracking library, respectively, and can update the position information of the target to be updated whose predicted position information matches the matching position information in the first tracking library and the second tracking library based on the matching position information, thereby obtaining the updated first tracking library and / or the updated second tracking library.

[0059] In some embodiments, if the to-be-matched position information matches the predicted position information corresponding to a plurality of to-be-updated targets in one tracking library, the ego vehicle can update the position information of the to-be-updated target in the tracking library with the highest overlap degree with the to-be-matched position information based on the to-be-matched position information, to obtain an updated tracking library. Wherein, the ego vehicle can replace the position information corresponding to the to-be-updated target corresponding to the predicted position information matched with the to-be-matched position information with the to-be-matched position information, so as to update the position information of the target in the tracking library in time and improve the accuracy of target detection.

[0060] Step S207: Based on the intersection over union algorithm, match each first target in the first tracking library with a tracking state of an active state with each second target in the second tracking library with a tracking state of an active state, to obtain a matching rate between each first target and each second target.

[0061] In some embodiments, after obtaining the first tracking library and the second tracking library, the ego vehicle can match each first target in the first tracking library with a tracking state of an active state with each second target in the second tracking library with a tracking state of an active state based on the intersection over union algorithm, to obtain a matching rate between each first target and each second target.

[0062] Wherein, the ego vehicle can match the position information of each first target in the first tracking library with an active state with the position information of each second target in the second tracking library with an active state one by one based on the IoU method, to obtain a matching rate between each first target and each second target. Wherein, the matching rate between each first target and each second target can represent the overlap degree between the position of the detection frame corresponding to each first target and the position of the detection frame corresponding to each second target.

[0063] Step S208: Obtain a second target with a matching rate greater than the second threshold value with any first target from the second tracking library as the matching target.

[0064] In some embodiments, after the ego vehicle obtains the matching rate between each active first target in the first tracking library and each active second target in the second tracking library, the ego vehicle can obtain a second target with a matching rate greater than a second threshold value with any first target from the second tracking library as a matching target. Thus, the intersection target in the surround view target detection result and the monocular target detection result is obtained.

[0065] Wherein, the second threshold value can be pre-set in the ego vehicle, can be set by the user independently, or can be determined by third-party experimental data, which is not limited here. For example, the second threshold value can be determined by third-party experimental data, which is 0.85, 0.86, etc.

[0066] Step S209: excluding the matched target from the second targets in the second tracking library in an active state to obtain second targets without matched objects.

[0067] In some embodiments, after obtaining the second targets without matched objects from the second tracking library, the ego vehicle can obtain the second targets with a tracking life cycle greater than a third threshold from the second targets without matched objects as the candidate targets, thereby performing reactivation output on the monocular target detection result to obtain the targets tracked by the monocular target detection result which do not affect the surround view detection result.

[0068] Step S210: obtaining the second targets with a tracking life cycle greater than a third threshold from the second targets without matched objects as the candidate targets.

[0069] In some embodiments, after obtaining the second targets without matched objects from the second tracking library, the ego vehicle can obtain the second targets with a tracking life cycle greater than a third threshold from the second targets without matched objects as the candidate targets, thereby performing reactivation output on the monocular target detection result to obtain the targets tracked by the monocular target detection result which do not affect the surround view detection result.

[0070] The third threshold can be pre-set in the ego vehicle, can be set by the user independently, or can be determined by third-party experimental data, and is not limited herein. For example, the third threshold can be determined by third-party experimental data and is 1s, 2s, etc.

[0071] Step S211: excluding the first target corresponding to the matched target from the first tracking library to obtain a sub-detection result.

[0072] In some embodiments, after determining the matched target, the ego vehicle can exclude the first target corresponding to the matched target from the first tracking library to obtain a sub-detection result. The first target corresponding to the matched target can be understood as the first target in the first tracking library with a matching rate greater than the second threshold and in an active state. The sub-detection result can include position information, direction information, motion information, appearance information, etc.

[0073] Step S212: obtaining the third target detection result according to the sub-detection result and information corresponding to the candidate target in the second tracking library.

[0074] In some implementations, after obtaining the sub-detection results and candidate targets, the vehicle can obtain a third target detection result based on the sub-detection results and the information corresponding to the candidate targets in the second tracking database. This integrates the results of monocular 3D detection of distant small targets and the results of surround-view target detection to output a distant target detection result, thus improving the recall rate of distant targets.

[0075] For example, please refer to again Figure 3 Among them, this vehicle can obtain the first target detection result (e.g., including the target) and goals Location information, etc.) and the second target detection results (e.g., including target location ... and goals After obtaining location information, etc., targets in the first target detection result and the second target detection result whose distance from the vehicle is less than or equal to a first threshold are filtered out to obtain the first target. And the second objective .

[0076] This vehicle can target the first objective separately. And the second objective Track and, based on the first objective The tracking information, including tracking status, position, direction, speed, and appearance features, is used to establish a first tracking database; and based on the second target... A second tracking library is established based on tracking information such as tracking status, position, direction, speed, and appearance features. The first tracking library may correspond to a first tracking management module, which is used for updating and managing the first tracking library. Similarly, the second tracking library may correspond to a second tracking management module, which is used for updating and managing the second tracking library.

[0077] The first tracking management module can use the first target in the active state of the first tracking library as the target to be updated, and can perform trajectory prediction on the target to be updated based on a Kalman filter to obtain the predicted position information corresponding to the target to be updated. The first tracking management module can also obtain the position information of the target to be matched currently detected by the vehicle as the matching position information, and can determine the overlap between the matching position information and the predicted position information based on the intersection-union algorithm. If the overlap is greater than a fourth threshold (e.g., 0.9), it can be determined that the matching target and the target to be updated are the same object, and the position information of the target to be updated in the first tracking library can be updated based on the matching position information, thereby obtaining an updated first tracking library and improving the accuracy of target detection.

[0078] The second tracking management module can use the second target in the active state of the second tracking library as the target to be updated, and can perform trajectory prediction on the target to be updated based on a Kalman filter to obtain the predicted position information of the target to be updated. The second tracking management module can also obtain the position information of the target to be matched currently detected by the vehicle as the matching position information, and can determine the overlap between the matching position information and the predicted position information based on the intersection-union algorithm. If the overlap is greater than a fourth threshold (e.g., 0.9), it can be determined that the matching target and the target to be updated are the same object, and the position information of the target to be updated in the second tracking library can be updated based on the matching position information, thereby obtaining an updated second tracking library and improving the accuracy of target detection.

[0079] Specifically, this vehicle can perform cross-association matching on targets in the first and second tracking databases based on the cross-fusion module. Specifically, this vehicle can obtain the first target in the first tracking database that is in the active state. and the second target in the second tracking library in its active state Among them, this vehicle can use the IoU method to... one by one Perform matching to obtain each With each The matching rate between them, and can be obtained from the second tracking library with any The matching rate between them is greater than the second threshold (e.g., 0.85). , as a matching target .

[0080] This vehicle can also reactivate the output of the second tracking library, and can select targets from the second tracking library using a candidate target selection formula. Select from matching targets Other tracking lifetimes exceeding the third threshold as candidate targets The candidate target selection formula includes: = - If alife > threshold Here, `alife` represents the duration of continuous tracking of the current target in the second tracking library; and `threshold` represents the third threshold. For example, `threshold` is 1, meaning that the target will only be selected if it has been continuously tracked for more than 1 second.

[0081] Among them, this vehicle can exclude the first target corresponding to the matching target from the first tracking database to obtain the sub-detection result ( - ), and can obtain a third target detection result (T3) according to the sub-detection result and information corresponding to the candidate target in the second tracking library = + - . Thus, the monocular 3D detection result of a long-distance small target and the surround-view target detection result are cross-correlated, matched, fused, and processed, and a long-distance small target is output, which improves the recall rate of a long-distance target, improves the accuracy and consistency of the position of the detected long-distance small target, provides accurate long-distance small target obstacle avoidance information for downstream driving planning, and ensures the safety and comfort of automatic driving.

[0082] Compared with the target detection method shown in Figure 1 , the target detection method provided by an embodiment of the present application can exclude the first target corresponding to the matched target from the first tracking library to obtain a sub-detection result, and obtain a third target detection result according to the sub-detection result and information corresponding to the candidate target in the second tracking library, thereby obtaining a monocular target detection tracking result and a target output of the surround-view target detection and monocular target detection without matching management, avoiding the redundancy of the target output, and improving the accuracy of target detection.

[0083] Meanwhile, the embodiment can also match each first target in the first tracking library in an activated state with each second target in the second tracking library in an activated state based on an intersection-over-union algorithm to obtain a matching rate between each first target and each second target, and obtain, as a matched target, a second target in the second tracking library having a matching rate greater than a second threshold with any first target, thereby determining the target of the surround-view target detection and monocular target detection matching management in a correlated matching manner, and improving the accuracy of target detection.

[0084] Meanwhile, the embodiment can also exclude the matched target from the second target in the second tracking library in an activated state to obtain a second target without a matching object, and obtain, as a candidate target, a second target having a tracking life cycle greater than a third threshold from the second target without a matching object, thereby excluding the target of the surround-view target detection result and the monocular target detection matching management, avoiding the redundancy of the target output, and reducing the influence of the monocular target detection result on the surround-view target detection result.

[0085] In addition, the embodiment can further acquire position information of the currently detected matching target as matching position information before determining the second target with a matching rate greater than the second threshold as the matching target from the second tracking library, and determining the second target with a tracking life cycle satisfying a preset condition as the candidate target from the second target other than the matching target; acquire the target with an active state in the first tracking library and the second tracking library as the target to be updated; perform position prediction on the target to be updated to obtain predicted position information; and update the position information corresponding to the target to be updated based on the matching position information if the predicted position information matches the matching position information, so as to update the position information of the target in the tracking library in time and improve the accuracy of target detection.

[0086] Please refer to Figure 5 , Figure 5 A module block diagram of a target detection device provided by an embodiment of the present application is shown. The target detection device 200 is applied to the electronic device described above, and the following will be described in detail with respect to the flow shown in Figure 5 The target detection device 200 includes a surround detection module 210, a tracking library establishing module 220, a cross matching module 230, and a target fusion module 240, wherein: The surround detection module 210 is configured to acquire a first target detection result of an environment in which the vehicle is located obtained through surround target detection, and acquire a second target detection result of the environment in which the vehicle is located obtained through monocular target detection.

[0087] The tracking library establishing module 220 is configured to perform target tracking on a first target in the first target detection result to obtain a first tracking library, and perform target tracking on a second target in the second target detection result to obtain a second tracking library, the distance between the first target and the second target and the vehicle being greater than a first threshold.

[0088] The cross matching module 230 is configured to determine a second target with a matching rate greater than a second threshold as a matching target from the second tracking library, and determine a second target with a tracking life cycle satisfying a preset condition as a candidate target from the second target other than the matching target.

[0089] The target fusion module 240 is configured to acquire a third target detection result corresponding to the vehicle according to the first target, the matching target, and the candidate target.

[0090] Further, the target fusion module 240 can include a sub-detection result obtaining unit and a target fusion sub-unit, wherein: The sub-detection result obtaining unit is configured to exclude the first target corresponding to the matching target from the first tracking library to obtain a sub-detection result.

[0091] a target fusion subunit, configured to obtain the third target detection result according to the sub-detection result and information corresponding to the candidate target in the second tracking library.

[0092] Further, the cross matching module 230 can include a matching rate obtaining unit and a matching target obtaining unit, wherein: The matching rate obtaining unit is configured to match each first target with a tracking state of an active state in the first tracking library and each second target with a tracking state of an active state in the second tracking library based on an intersection over union algorithm, and obtain a matching rate between each first target and each second target.

[0093] The matching target obtaining unit is configured to obtain, as the matching target, a second target with a matching rate greater than the second threshold value with any first target from the second tracking library.

[0094] Further, the cross matching module 230 can include a matching target excluding unit and a candidate target obtaining unit, wherein: The matching target excluding unit is configured to exclude the matching target from the second target with a tracking state of an active state in the second tracking library, and obtain a second target without a matching object.

[0095] The candidate target obtaining unit is configured to obtain, as the candidate target, a second target with a tracking life cycle greater than a third threshold value from the second target without a matching object.

[0096] Further, before determining, as the matching target, a second target with a matching rate greater than a second threshold value with the first target from the second tracking library, and determining, as the candidate target, a second target with a tracking life cycle satisfying a preset condition from the second target other than the matching target, the target detection device 200 can further include a to-be-matched position information obtaining unit, a to-be-updated target obtaining unit, a predicted position information obtaining unit, and a tracking library updating unit, wherein: The to-be-matched position information obtaining unit is configured to obtain position information of a to-be-matched target currently detected as to-be-matched position information.

[0097] The to-be-updated target obtaining unit is configured to obtain a target with a tracking state of an active state in the first tracking library and the second tracking library as a to-be-updated target.

[0098] The predicted position information obtaining unit is configured to perform position prediction on the to-be-updated target, and obtain predicted position information.

[0099] The tracking library updating unit is configured to update the position information corresponding to the target to be updated based on the position information to be matched if the predicted position information matches the position information to be matched.

[0100] Further, the predicted position information obtaining unit can comprise a position predicting unit, wherein: The position predicting unit is configured to predict the position of the target to be updated based on a Kalman filter to obtain the predicted position information.

[0101] Further, before the target detecting apparatus 200 determines that the target to be matched matches the target to be updated if the predicted position information matches the position information to be matched, the target detecting apparatus 200 can further comprise an overlap degree obtaining unit and a position information matching unit, wherein: The overlap degree obtaining unit is configured to obtain the overlap degree between the predicted position information and the position information to be matched based on an intersection over union algorithm.

[0102] The position information matching unit is configured to determine that the predicted position information matches the position information to be matched if the overlap degree is greater than a fourth threshold.

[0103] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described apparatuses and modules can refer to the corresponding process in the foregoing method embodiments, which will not be described herein.

[0104] In several embodiments provided in the present application, the coupling between the modules can be electrical, mechanical or other forms of coupling.

[0105] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can exist physically independently, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0106] Please refer to Figure 6 which shows a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device 100 can be a vehicle, a vehicle-mounted terminal, a server, a computer or the like, which has processing capability. The electronic device 100 in the present application can comprise one or more of the following components: a processor 110, a memory 120 and one or more application programs, wherein the one or more application programs can be stored in the memory 120 and configured to be executed by the one or more processors 110, and the one or more programs are configured to perform the method described in the foregoing method embodiments.

[0107] The processor 110 can include one or more processing cores. The processor 110 connects various parts within the vehicle 100 by various interfaces and lines, performs various functions of the vehicle 100 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Alternatively, the processor 110 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 110 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs; the GPU is responsible for rendering and drawing the content to be displayed; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 110, but can be implemented separately by a communication chip.

[0108] The memory 120 can include a random access memory (RAM) and can also include a read-only memory (ROM). The memory 120 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing each of the method embodiments described below, etc. The data storage area can also store data created by the electronic device 100 in use (such as a phone book, audio and video data, chat record data, etc.).

[0109] In this embodiment, the computer-readable medium stores program codes, which can be called and executed by the processor to perform the methods described in the above method embodiments.

[0110] The computer-readable storage medium can be an electronic, magnetic, optical, or other physical storage device that stores executable computer program code. The computer-readable storage medium can alternatively or additionally include a non-transitory computer-readable storage medium. The computer-readable storage medium has a storage space for storing program code that executes any of the method steps described above. The program code can be read from or written to one or more computer program products. The program code can be compressed, for example, in a suitable form.

[0111] In the present application, multiple refers to two or more than two.

[0112] In the present application, unless specifically defined otherwise, the terms "mount", "connected", "connection" should be understood broadly, for example, can be fixedly connected, can be detachably connected, or integrally connected; can be mechanically connected, can be electrically connected; can be directly connected, can be indirectly connected through an intermediate medium, can be internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0113] The terms "first", "second", "third", "fourth" and the like (if any) in the present application are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0114] The term "and / or" in the present application is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. In addition, the character " / " in the present application generally represents an "or" relationship between the front and rear associated objects.

[0115] If there is no special description, all the steps of the present application can be performed in sequence or randomly. For example, the method comprises steps A and B, which means that the method can comprise steps A and B performed in sequence, or steps B and A performed in sequence. For example, the method also comprises step C, which means that step C can be added to the method in any order, for example, the method can comprise steps A, B and C, or steps A, C and B, or steps C, A and B, etc.

[0116] The above only describes the preferred embodiments of the present application and does not limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A target detection method, characterized in that, The method includes: Obtain the first target detection result of the vehicle's environment obtained through surround view target detection, and obtain the second target detection result of the vehicle's environment obtained through monocular target detection; The first target in the first target detection result is tracked to obtain a first tracking library, and the second target in the second target detection result is tracked to obtain a second tracking library, wherein the distance between the first target and the second target and the vehicle is greater than a first threshold. A second target is determined from the second tracking library whose matching rate with the first target is greater than a second threshold, and is used as a matching target; and a second target whose tracking lifecycle meets a preset condition among the second targets other than the matching target is determined as a candidate target. Based on the first target, the matching target, and the candidate target, obtain the detection result of the third target corresponding to this vehicle.

2. The method according to claim 1, characterized in that, The step of obtaining the detection result of the third target corresponding to the vehicle based on the first target, the matching target, and the candidate target includes: Exclude the first target corresponding to the matching target from the first tracking library to obtain the sub-detection result; The third target detection result is obtained based on the sub-detection result and the information corresponding to the candidate target in the second tracking library.

3. The method according to claim 1, characterized in that, The step of determining a second target from the second tracking library whose matching rate with the first target is greater than a second threshold, as the matching target, includes: Based on the cross-union algorithm, each first target in the first tracking library that is in an active tracking state is matched with each second target in the second tracking library that is in an active tracking state to obtain the matching rate between each first target and each second target; A second target with a matching rate greater than the second threshold is obtained from the second tracking library and is used as the matching target.

4. The method according to claim 1, characterized in that, The step of determining a second target, other than the matching target, whose tracking lifecycle meets a preset condition, as a candidate target includes: The matching targets are excluded from the second targets that are in an active state in the second tracking library to obtain second targets with no matching objects; From the second targets with no matching objects, obtain a second target whose tracking lifecycle is greater than a third threshold, and use it as the candidate target.

5. The method according to any one of claims 1-4, characterized in that, Before determining a second target from the second tracking library whose matching rate with the first target is greater than a second threshold as a matching target, and determining a second target among the second targets other than the matching target whose tracking lifecycle meets a preset condition as a candidate target, the method further includes: Obtain the location information of the currently detected target to be matched, and use it as the location information to be matched; Obtain targets in the first and second tracking libraries that are in an active tracking state, and use them as targets to be updated; The location of the target to be updated is predicted to obtain the predicted location information; If the predicted location information matches the location information to be matched, then the location information corresponding to the target to be updated is updated based on the location information to be matched.

6. The method according to claim 5, characterized in that, The step of predicting the location of the target to be updated to obtain predicted location information includes: The location of the target to be updated is predicted based on a Kalman filter to obtain the predicted location information.

7. The method according to claim 5, characterized in that, Before determining that the target to be matched matches the target to be updated if the predicted location information matches the location information to be matched, the method further includes: Based on the intersection-union algorithm, the overlap between the predicted location information and the location information to be matched is obtained; If the overlap is greater than the fourth threshold, then the predicted location information is determined to match the location information to be matched.

8. A target detection device, characterized in that, The device includes: The surround view detection module is used to obtain the first target detection result of the environment in which the vehicle is located through surround view target detection, and to obtain the second target detection result of the environment in which the vehicle is located through monocular target detection. The tracking library establishment module is used to perform target tracking on the first target in the first target detection result to obtain a first tracking library, and to perform target tracking on the second target in the second target detection result to obtain a second tracking library, wherein the distance between the first target and the second target and the vehicle is greater than a first threshold. The cross-matching module is used to determine a second target from the second tracking library whose matching rate with the first target is greater than a second threshold, as a matching target, and to determine a second target among the second targets other than the matching target whose tracking life cycle meets a preset condition, as a candidate target; The target fusion module is used to obtain the detection result of the third target corresponding to the vehicle based on the first target, the matching target and the candidate target.

9. An electronic device, characterized in that, include: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1-7.

Images