Target detection method, device and equipment, medium and vehicle

Abstract

The invention relates to a target detection method, device and equipment, a medium and a vehicle, and the method comprises the steps: obtaining a to-be-detected image collected by a camera, and internal and external parameters of the camera; inputting the to-be-detected image and the internal and external parameters of the camera into the BEV model, outputting a two-dimensional feature map of a target object in the to-be-detected image by a target segmentation network in the BEV model, and outputting a three-dimensional feature map of the target object by the BEV model; projecting the three-dimensional feature map to a plane where the two-dimensional feature map is located by using internal and external parameters of the camera to obtain a three-dimensional projection map; and calculating the intersection-to-union ratio of the three-dimensional projection drawing and the two-dimensional feature map, and determining that the three-dimensional feature map of the target object is effective when the intersection-to-union ratio is greater than a preset threshold value, thereby improving the accuracy of target detection when target detection is performed by using the BEV perception model.

Description

Technical Field

[0001] The present disclosure relates to the field of information processing technology, and in particular to a target detection method, device, equipment, medium and vehicle. Background Art

[0002] For intelligent driving vehicles, accurate and real-time target object detection is one of the important functions that enables intelligent driving vehicles to accurately perceive the complex environment around them. However, how to accurately judge and identify the three-dimensional information such as the position, size, and orientation of target objects in the complex environment around the vehicle is the main difficulty in target detection.

[0003] At present, the main method for detecting the three-dimensional information of target objects in the complex environment around the vehicle is to obtain camera images around the vehicle, input the camera images into the Bird's-Eye-View (BEV) perception model, and use the BEV perception model to recognize the camera images to obtain the three-dimensional information of the target objects around the vehicle.

[0004] However, when the three-dimensional information of the target object is detected through the BEV perception model, there are problems such as inaccurate target object detection results; therefore, how to determine the accuracy of target detection when using the BEV perception model for target object detection becomes an urgent problem to be solved. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides a target detection method, device, equipment, medium and vehicle.

[0006] A first aspect of an embodiment of the present disclosure provides a target detection method, the method comprising:

[0007] Obtain the image to be detected captured by the camera and the internal and external parameters of the camera;

[0008] The image to be detected and the internal and external parameters of the camera are input into the BEV model, and the target segmentation network in the BEV model outputs a two-dimensional feature map of the target object in the image to be detected, and the BEV model outputs a three-dimensional feature map of the target object;

[0009] The three-dimensional feature map is projected onto the plane where the two-dimensional feature map is located using the internal and external parameters of the camera to obtain a three-dimensional projection map;

[0010] The intersection-and-union ratio of the three-dimensional projection image and the two-dimensional feature image is calculated. When the intersection-and-union ratio is greater than a preset threshold, the three-dimensional feature image of the target object is determined to be valid.

[0011] A second aspect of an embodiment of the present disclosure provides a target detection device, the device comprising:

[0012] The image acquisition module is used to obtain the image to be detected collected by the camera and the internal and external parameters of the camera;

[0013] A feature extraction module is used to input the image to be detected and the internal and external parameters of the camera into the BEV model, and the target segmentation network in the BEV model outputs a two-dimensional feature map of the target object in the image to be detected, and the BEV model outputs a three-dimensional feature map of the target object;

[0014] A feature map projection module is used to project the three-dimensional feature map onto the plane where the two-dimensional feature map is located using the internal and external parameters of the camera to obtain a three-dimensional projection map;

[0015] The result determination module is used to calculate the intersection-and-union ratio of the three-dimensional projection image and the two-dimensional feature image. When the intersection-and-union ratio is greater than a preset threshold, the three-dimensional feature image of the target object is determined to be valid.

[0016] A third aspect of an embodiment of the present disclosure provides an electronic device, the device comprising:

[0017] Memory;

[0018] Processor; and

[0019] A computer program, wherein the computer program is stored in a memory and is configured to be executed by a processor to implement the target detection method of the first aspect as described above.

[0020] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the target detection method as described in the first aspect above is implemented.

[0021] A fifth aspect of an embodiment of the present disclosure provides a vehicle, comprising the electronic device of the third aspect described above.

[0022] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0023] The target detection method, apparatus, device, medium and vehicle provided by the embodiments of the present disclosure can obtain the image to be detected collected by the camera and the internal and external parameters of the camera. After obtaining the image to be detected and the internal and external parameters of the camera, the image to be detected and the internal and external parameters of the camera are input into the BEV model, and the target segmentation network in the BEV model outputs a two-dimensional feature map of the target object in the image to be detected, and the BEV model outputs a three-dimensional feature map of the target object, and the three-dimensional feature map is projected to the plane where the two-dimensional feature map is located using the internal and external parameters of the camera to obtain a three-dimensional projection map, and the intersection and union ratio of the three-dimensional projection map and the two-dimensional feature map is calculated. When the intersection and union ratio is greater than a preset threshold, the three-dimensional feature map of the target object is determined to be valid. Therefore, the two-dimensional feature map of the target object can be compared with the three-dimensional projection map corresponding to the three-dimensional feature map to verify the three-dimensional feature map output by the BEV model, thereby improving the accuracy of target detection when the BEV perception model is used for target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 is a flow chart of a target detection method provided by an embodiment of the present disclosure;

[0027] Figure 2 is a flow chart of another target detection method provided by an embodiment of the present disclosure;

[0028] Figure 3 is a structural schematic diagram of a target detection device provided by an embodiment of the present disclosure;

[0029] Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0031] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0032] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0033] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0034] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0035] Normally, the main method for detecting the three-dimensional information of target objects in the complex environment around the vehicle is to obtain camera images around the vehicle, input the camera images into the bird's-eye-view (BEV) perception model, and use the BEV perception model to recognize the camera images to obtain the three-dimensional information of the target objects around the vehicle. However, when the three-dimensional information of the target object is detected by the BEV perception model, there are problems such as inaccurate target object detection results. Therefore, how to determine the accuracy of target detection when using the BEV perception model for target object detection becomes an urgent problem to be solved. In response to this problem, the embodiments of the present disclosure provide a target detection method, which is introduced below in conjunction with specific embodiments.

[0036] Figure 1It is a flow chart of a target detection method provided by an embodiment of the present disclosure. The method can be executed by a target detection device. The target detection device can be implemented in software and / or hardware. The target detection device can be configured in an electronic device, such as a server or a terminal or a server cluster, wherein the terminal can specifically include a computer or a tablet computer, a vehicle-mounted terminal, etc.

[0037] like Figure 1 As shown, the target detection method provided by the embodiment of the present disclosure includes the following steps:

[0038] S110, obtaining the image to be detected captured by the camera and the internal and external parameters of the camera.

[0039] In the embodiment of the present disclosure, after receiving a target detection task or when target detection is required, the electronic device obtains the image to be detected captured by the camera and the internal and external parameters of the camera.

[0040] The target detection task includes an image to be detected corresponding to the target detection task and / or a unique identifier corresponding to the target detection task.

[0041] In the disclosed embodiment, the number of cameras may be at least one, and at least one camera may be respectively set at multiple positions of the vehicle, wherein the multiple positions may include the left front, front, right front, left rear, rear, and right rear of the vehicle, and the positions may also be set according to the angle of the camera and actual conditions, which is not limited here.

[0042] In the embodiments of the present disclosure, the internal and external parameters of the camera refer to the internal and external parameters of the camera, wherein the internal parameters of the camera are parameters related to the camera's own characteristics, such as the camera's aperture, focal length, pixel size, etc., and the external parameters of the camera are parameters that reflect the relative position relationship between the camera and the vehicle, such as parameters in the vehicle coordinate system or the world coordinate system, or the camera's position, posture, orientation, etc.

[0043] In the embodiment of the present disclosure, the camera may be one or more, and the camera may be arranged on the vehicle. If multiple cameras are arranged on the vehicle, a corresponding relationship between the inner and outer sides of each camera and the images to be detected collected by the camera is established.

[0044] In some embodiments of the present disclosure, after receiving a target detection task, the electronic device parses the instruction to obtain at least one image to be detected corresponding to the target detection task, or obtains at least one image to be detected that is pre-captured and corresponds to the target detection task from a preset memory according to the unique identifier of the target detection task.

[0045] In other embodiments of the present disclosure, when target detection is needed, the electronic device controls at least one camera to collect images around the vehicle. After collecting the images, the at least one camera uploads the images to the electronic device, so that the electronic device obtains the images to be detected. It should be noted that since each camera on the vehicle faces different directions and angles, the collected images are also different. At this time, the different images collected by different cameras can be stitched together using existing methods such as image stitching, so that the electronic device can obtain the images to be detected.

[0046] S120, input the image to be detected and the internal and external parameters of the camera into the BEV model, and the target segmentation network in the BEV model outputs a two-dimensional feature map of the target object in the image to be detected, and the BEV model outputs a three-dimensional feature map of the target object.

[0047] In the embodiment of the present disclosure, after acquiring the image to be detected and the internal and external parameters of the camera, the electronic device inputs the image to be detected and the internal and external parameters of the camera into the BEV model, and the target segmentation network in the BEV model outputs a two-dimensional feature map of the target object in the image to be detected, and the BEV model outputs a three-dimensional feature map of the target object.

[0048] Among them, the BEV model is an existing model, which is mainly used to determine the spatial information of the two-dimensional image using the internal and external parameters of the camera that collects the two-dimensional image, and generate a three-dimensional feature image using the two-dimensional image information; the BEV model can include multiple different networks, and using these networks, specific processes such as identifying the target object from the two-dimensional image, segmenting the image, adding annotation boxes, determining the depth information of the target object, generating the detection box of the target object in the three-dimensional space, and the three-dimensional feature image can be completed; in these networks, the target segmentation network can generate a corresponding two-dimensional feature map for the target object in the two-dimensional image, and this two-dimensional feature map can contain the detection box of the target object, that is, the corresponding detection box in the two-dimensional image.

[0049] The target object may be an object around the vehicle that creates obstacles to the vehicle's travel, such as buildings, other vehicles around the vehicle, trees, etc.

[0050] The target segmentation network is the network structure in the existing BEV model. Its function is to segment and detect the image to be detected and obtain the two-dimensional feature map corresponding to the target object.

[0051] In the embodiment of the present disclosure, the two-dimensional feature map may include a two-dimensional image and a two-dimensional detection box of the target object.

[0052] Furthermore, the BEV model outputs a three-dimensional feature map of the target object, which may specifically include:

[0053] The visual transformation network in the BEV model converts the two-dimensional feature map from the two-dimensional coordinate system to the three-dimensional coordinate system according to the internal and external parameters of the camera to obtain a three-dimensional feature map.

[0054] The 3D feature map may include the 3D image and 3D detection frame of the target object, as well as the position and pose of the 3D image in the vehicle coordinate system.

[0055] In the disclosed embodiment, the specific implementation method of converting the two-dimensional feature map from the two-dimensional coordinate system to the three-dimensional coordinate system according to the internal and external parameters of the camera is similar to the implementation method of coordinate system conversion of the existing BEV model, and will not be repeated here.

[0056] S130, using the internal and external parameters of the camera to project the three-dimensional feature map onto the plane where the two-dimensional feature map is located, to obtain a three-dimensional projection map.

[0057] In some embodiments of the present disclosure, the three-dimensional feature map is projected onto the plane where the two-dimensional feature map is located using the internal and external parameters of the camera to obtain a three-dimensional projection map, which may specifically include: performing a coordinate system conversion on the two-dimensional feature map based on the internal and external parameters of the camera to obtain the plane where the two-dimensional feature map is located in the vehicle coordinate system; using a perspective projection method to project the three-dimensional feature map onto the plane where the two-dimensional feature map is located in the vehicle coordinate system to obtain a three-dimensional projection map. In this process, because the internal and external parameters of the camera are known, the internal parameters of the camera and the existing methods can be used to determine the plane where the two-dimensional image captured by the camera is located in the camera coordinate system, and the two-dimensional feature map is obtained based on the two-dimensional image captured by the camera, which is the same as the plane where the two-dimensional image is located in the camera coordinate system; in addition, the internal and external parameters of the camera can be used to determine the coordinate conversion relationship between the camera coordinate system and the vehicle coordinate system of the vehicle where the camera is located, and even further determine the conversion relationship between the camera coordinate system and the world coordinate system; in addition, the three-dimensional feature map of the target object is obtained using the BEV model, which clarifies the target object and its detection frame in the three-dimensional space (such as in the vehicle coordinate system). 3D information in a three-dimensional space); after obtaining the above-mentioned coordinate transformation relationship, the two-dimensional feature map and the three-dimensional feature map, the three-dimensional feature map can be projected onto the plane where the two-dimensional image is located by using existing methods such as perspective projection (such as projecting the three-dimensional feature map along the perpendicular direction of the plane where the two-dimensional image is located). Since the three-dimensional feature map contains the target object and its three-dimensional detection frame, the three-dimensional projection map obtained after projecting the three-dimensional feature map contains the projection image information of the target object and its three-dimensional detection frame; at this time, since the three-dimensional projection map and the two-dimensional feature map are already in the same plane, the three-dimensional projection map and the two-dimensional feature map can be subsequently processed, such as calculating the overlap degree or the intersection-union ratio and so on.

[0058] S140, calculating an intersection-over-union ratio of the three-dimensional projection image and the two-dimensional feature image, and when the intersection-over-union ratio is greater than a preset threshold, determining that the three-dimensional feature image of the target object is valid.

[0059] In some embodiments of the present disclosure, after obtaining the three-dimensional projection image, the electronic device calculates the intersection and union ratio of a first detection box corresponding to the target object in the three-dimensional projection image and a second detection box corresponding to the target object in the two-dimensional feature image. When the intersection and union ratio is greater than a preset threshold, the three-dimensional feature map of the target object is determined to be valid.

[0060] In other embodiments of the present disclosure, after obtaining the three-dimensional projection image, the electronic device respectively performs mask processing on the three-dimensional projection image and the two-dimensional feature map, and calculates the intersection and union ratio of the three-dimensional projection image after the mask processing and the two-dimensional feature map after the mask processing. When the intersection and union ratio is greater than a preset threshold, the three-dimensional feature map of the target object is determined to be valid. The specific implementation method of performing mask processing on the three-dimensional projection image and the two-dimensional feature map is similar to the existing implementation method of performing mask processing on an image, and will not be repeated here.

[0061] In the disclosed embodiment, the image to be detected captured by the camera and the internal and external parameters of the camera can be obtained. After the image to be detected and the internal and external parameters of the camera are obtained, the image to be detected and the internal and external parameters of the camera are input into the BEV model, and the target segmentation network in the BEV model outputs a two-dimensional feature map of the target object in the image to be detected, and the BEV model outputs a three-dimensional feature map of the target object. The three-dimensional feature map is projected to the plane where the two-dimensional feature map is located using the internal and external parameters of the camera to obtain a three-dimensional projection map, and the intersection and union ratio of the three-dimensional projection map and the two-dimensional feature map is calculated. When the intersection and union ratio is greater than a preset threshold, the three-dimensional feature map of the target object is determined to be valid. Therefore, the two-dimensional feature map of the target object can be compared with the three-dimensional projection map corresponding to the three-dimensional feature map to verify the three-dimensional feature map output by the BEV model, thereby improving the accuracy of target detection when the BEV perception model is used for target detection.

[0062] In the disclosed embodiment, the intersection and union scores of the three-dimensional projection map and the two-dimensional feature map are calculated in three cases: the first case is that the three-dimensional projection map includes the first detection frame corresponding to the target object, and the two-dimensional feature map also includes the second detection frame corresponding to the target object; the second case is that the three-dimensional projection map includes the first detection frame corresponding to the target object, but the two-dimensional feature map does not include the second detection frame corresponding to the target object; the third case is that the three-dimensional projection map does not include the first detection frame corresponding to the target object, but the two-dimensional feature map includes the second detection frame corresponding to the target object. The following is an introduction based on these three cases respectively.

[0063] In some embodiments of the present disclosure, calculating the intersection-and-union ratio of a three-dimensional projection image and a two-dimensional feature image may specifically include: when the three-dimensional projection image includes a first detection frame corresponding to the target object and the two-dimensional feature image includes a second detection frame corresponding to the target object, calculating the intersection-and-union ratio of the first detection frame and the second detection frame; comparing the intersection-and-union ratio with a preset threshold, and when the intersection-and-union ratio is greater than or equal to the preset threshold, determining that the three-dimensional feature image of the target object is valid.

[0064] The preset threshold may be 0.5, or may be set according to specific circumstances, and is not limited here.

[0065] Specifically, when the electronic device includes a first detection frame corresponding to the target object in the three-dimensional projection image and includes a second detection frame corresponding to the target object in the two-dimensional feature map, it calculates the intersection and union ratio of the first detection frame and the second detection frame, and compares the intersection and union ratio with a preset threshold. When the intersection and union ratio is greater than or equal to the preset threshold, it means that the target object has been confirmed in both two-dimensional and three-dimensional features, that is, the credibility of the detection result is high, and the three-dimensional feature map of the target object is determined to be valid.

[0066] Furthermore, when the intersection-and-union ratio is less than a preset threshold, a data return instruction is generated; based on the data return instruction, the target data is returned to the data platform, the target data including the image to be detected and the internal and external parameters of the camera; and the BEV model is optimized based on the target data.

[0067] The target data may also include a three-dimensional projection image and a two-dimensional feature image.

[0068] Specifically, when the intersection-and-union ratio is less than a preset threshold value, the electronic device indicates that the credibility of the detection result needs to be improved or further confirmed, and then generates a data return instruction. Based on the data return instruction, the target data is returned to the data platform. The data platform calculates the difference between the second detection box corresponding to the target object in the two-dimensional feature map and the first detection box corresponding to the target object in the three-dimensional projection map, and determines the difference as the loss value of the BEV model. The loss value is input into a preset loss function, and the parameters of the BEV model are adjusted. Then, the second detection box corresponding to the target object in the two-dimensional feature map is determined as the annotation result corresponding to the image to be detected. The target data and the annotation result corresponding to the target data are used as verification sample data to verify the accuracy of the adjusted BEV model, thereby achieving optimization of the BEV model and improving the credibility of the BEV model.

[0069] In the disclosed embodiment, the BEV model can be continuously optimized according to the comparison results of the three-dimensional projection image and the two-dimensional feature image, thereby further improving the accuracy and credibility of the target detection results.

[0070] In other embodiments of the present disclosure, for the case where the three-dimensional projection image includes a first detection frame corresponding to the target object, but the two-dimensional feature image does not include a second detection frame corresponding to the target object; and the three-dimensional projection image does not include the first detection frame corresponding to the target object, but the two-dimensional feature image includes the second detection frame corresponding to the target object, the following can be specifically performed. Figure 2 Steps S210-S230 are shown.

[0071] S210, determining whether the three-dimensional projection image includes the first detection frame, and whether the two-dimensional feature image includes the second detection frame.

[0072] S220. When the first detection frame corresponding to the target object is not included in the three-dimensional projection image, and the second detection frame corresponding to the target object is included in the two-dimensional feature image, the target object is recalled by a preset means, and when the target object is included in the recall result, the three-dimensional feature image of the target object is determined to be valid.

[0073] In the embodiment of the present disclosure, when the electronic device does not include a first detection frame corresponding to the target object in the three-dimensional projection image, but includes a second detection frame corresponding to the target object in the two-dimensional feature image, it indicates that the target object may have been missed, and the target object is then recalled through a preset means.

[0074] Among them, the preset means may include lowering a preset recall threshold, re-detecting and identifying the image to be detected through the BEV model to determine whether the target object exists, or other existing means for recalling the target object, which are not limited here.

[0075] In an embodiment of the present disclosure, when the target object is included in the recall result, the three-dimensional feature map of the target object is directly determined to be valid; when the target object does not exist in the recall result, the three-dimensional feature map of the target object is determined to be invalid.

[0076] S230. When the three-dimensional projection image includes a first detection frame corresponding to the target object, and the two-dimensional feature image does not include a second detection frame corresponding to the target object, filter out the target object from the three-dimensional projection image, and determine that the three-dimensional feature image of the target object is invalid.

[0077] In the disclosed embodiment, when the electronic device does not include the second detection frame corresponding to the target object in the two-dimensional feature map, the two-dimensional feature map is input into the visual transformation network model of the BEV model to perform a conversion from the two-dimensional coordinate system to the three-dimensional coordinate system, thereby obtaining a three-dimensional feature map, and the three-dimensional feature map is projected onto the plane where the two-dimensional feature map is located using the internal and external parameters of the camera to obtain a three-dimensional projection map. In the process of annotating the three-dimensional projection map to obtain the target detection frame in the three-dimensional projection map, the BEV model may incorrectly identify and annotate the target object in the three-dimensional projection map, thereby causing the three-dimensional projection map to include the first detection frame corresponding to the target object. Therefore, when the electronic device includes the first detection frame corresponding to the target object in the three-dimensional projection map and does not include the second detection frame corresponding to the target object in the two-dimensional feature map, the target object is determined as a falsely detected object in the three-dimensional projection map, and the target object is filtered out from the three-dimensional projection map, and the three-dimensional feature map of the target object is determined to be invalid.

[0078] In the disclosed embodiment, mutual supervision and confirmation of detection results can be achieved by combining a two-dimensional feature map and a three-dimensional projection map, thereby improving the accuracy of the target detection results.

[0079] Figure 3 : is a schematic diagram of the structure of a target detection device provided in an embodiment of the present disclosure. The target detection device in the embodiment of the present disclosure can be set in an electronic device, and the electronic device can be a server or a terminal or a server cluster, wherein the terminal can specifically include a computer or a tablet computer, a vehicle-mounted terminal, etc., which is not limited here.

[0080] like Figure 3 As shown, the target detection device 300 may include an image acquisition module 310 , a feature extraction module 320 , a feature map projection module 330 and a result determination module 340 .

[0081] The image acquisition module 310 can be used to acquire the image to be detected captured by the camera and the internal and external parameters of the camera.

[0082] The feature extraction module 320 can be used to input the image to be detected and the internal and external parameters of the camera into the BEV model, and the target segmentation network in the BEV model outputs a two-dimensional feature map of the target object in the image to be detected, and the BEV model outputs a three-dimensional feature map of the target object.

[0083] The feature map projection module 330 can be used to project the three-dimensional feature map onto the plane where the two-dimensional feature map is located using the internal and external parameters of the camera to obtain a three-dimensional projection map.

[0084] The result determination module 340 may be used to calculate an intersection-and-union ratio between the three-dimensional projection image and the two-dimensional feature image, and when the intersection-and-union ratio is greater than a preset threshold, the three-dimensional feature image of the target object is determined to be valid.

[0085] In the disclosed embodiment, the image to be detected captured by the camera and the internal and external parameters of the camera can be obtained. After the image to be detected and the internal and external parameters of the camera are obtained, the image to be detected and the internal and external parameters of the camera are input into the BEV model, and the target segmentation network in the BEV model outputs a two-dimensional feature map of the target object in the image to be detected, and the BEV model outputs a three-dimensional feature map of the target object. The three-dimensional feature map is projected to the plane where the two-dimensional feature map is located using the internal and external parameters of the camera to obtain a three-dimensional projection map, and the intersection and union ratio of the three-dimensional projection map and the two-dimensional feature map is calculated. When the intersection and union ratio is greater than a preset threshold, the three-dimensional feature map of the target object is determined to be valid. Therefore, the two-dimensional feature map of the target object can be compared with the three-dimensional projection map corresponding to the three-dimensional feature map to verify the three-dimensional feature map output by the BEV model, thereby improving the accuracy of target detection when the BEV perception model is used for target detection.

[0086] In some embodiments of the present disclosure, the feature map projection module 330 can be specifically used to perform coordinate system conversion on the two-dimensional feature map based on the internal and external parameters of the camera to obtain the plane where the two-dimensional feature map is located in the vehicle coordinate system; use the perspective projection method to project the three-dimensional feature map to the plane where the two-dimensional feature map is located in the vehicle coordinate system to obtain a three-dimensional projection map.

[0087] In some embodiments of the present disclosure, the result determination module 340 can be used to calculate the intersection and union ratio of the first detection frame and the second detection frame when the three-dimensional projection image includes a first detection frame corresponding to the target object and the two-dimensional feature image includes a second detection frame corresponding to the target object.

[0088] In some embodiments of the present disclosure, the target detection device 300 may further include a data feedback module.

[0089] The data feedback module can be used to generate a data feedback instruction when the intersection-and-union ratio is less than a preset threshold; based on the data feedback instruction, the target data is returned to the data platform, and the target data includes the image to be detected and the internal and external parameters of the camera; and the BEV model is optimized based on the target data.

[0090] In some embodiments of the present disclosure, the target detection device 300 may further include a target recall module.

[0091] The target recall module can be used to recall the target object by preset means when the first detection frame corresponding to the target object is not included in the three-dimensional projection image, and the second detection frame corresponding to the target object is included in the two-dimensional feature map. When the target object is included in the recall result, it is determined that the three-dimensional feature map is valid.

[0092] In some embodiments of the present disclosure, the feature extraction module 320 can be specifically used to convert the two-dimensional feature map from a two-dimensional coordinate system to a three-dimensional coordinate system according to the internal and external parameters of the camera by the visual transformation network in the BEV model to obtain a three-dimensional feature map.

[0093] It should be noted that Figure 3 The target detection device 300 shown can execute each step in the above method embodiment and realize each process and effect in the above method embodiment, which will not be described in detail here.

[0094] Figure 4 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown.

[0095] In the disclosed embodiment, Figure 4 The electronic device shown may be a server or a terminal or a server cluster, wherein the terminal may specifically include a mobile computer or a tablet computer, a vehicle-mounted terminal, etc., which is not limited here.

[0096] like Figure 4 As shown, the electronic device may include a processor 410 and a memory 420 storing computer program instructions.

[0097] Specifically, the processor 410 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0098] The memory 420 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 420 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 420 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 420 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 420 is a non-volatile solid-state memory. In a particular embodiment, the memory 420 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically rewritable ROM (EAROM) or a flash memory, or a combination of two or more of these.

[0099] The processor 410 reads and executes the computer program instructions stored in the memory 420 to perform the steps of the target detection method provided in the embodiment of the present disclosure.

[0100] In one example, the electronic device may further include a transceiver 430 and a bus 440. Figure 4 As shown, the processor 410, the memory 420 and the transceiver 430 are connected via a bus 440 and communicate with each other.

[0101] The bus 440 includes hardware, software, or both. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a Memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 440 may include one or more buses.

[0102] The embodiments of the present disclosure further provide a computer-readable storage medium, which may store a computer program. When the computer program is executed by a processor, the processor implements the target detection method provided by the embodiments of the present disclosure.

[0103] The above-mentioned storage medium may, for example, include a memory 420 of computer program instructions, and the above-mentioned instructions may be executed by the processor 410 of the electronic device to complete the target detection method provided in the embodiment of the present disclosure. Optionally, the storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc ROM, CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0104] The embodiments of the present disclosure also provide a vehicle, which includes electronic equipment and can implement the various processes and effects in the above embodiments of the present disclosure, which will not be elaborated here.

[0105] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising" is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0106] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A target detection method, characterized in that: The method comprises: Acquire the image to be detected captured by the camera and the internal and external parameters of the camera; Inputting the image to be detected and the internal and external parameters of the camera into the BEV model, the target segmentation network in the BEV model outputs a two-dimensional feature map of the target object in the image to be detected, and the BEV model outputs a three-dimensional feature map of the target object; Projecting the three-dimensional feature map onto the plane where the two-dimensional feature map is located using the internal and external parameters of the camera to obtain a three-dimensional projection map; The intersection-and-union ratio of the three-dimensional projection image and the two-dimensional feature image is calculated, and when the intersection-and-union ratio is greater than a preset threshold, the three-dimensional feature image of the target object is determined to be valid.

2. The method according to claim 1, characterized in that The method of projecting the three-dimensional feature map onto the plane where the two-dimensional feature map is located using the internal and external parameters of the camera to obtain a three-dimensional projection map includes: Performing coordinate system transformation on the two-dimensional feature map based on the internal and external parameters of the camera to obtain a plane where the two-dimensional feature map is located in the vehicle coordinate system; The three-dimensional feature map is projected onto the plane where the two-dimensional feature map is located in the vehicle coordinate system by using a perspective projection method to obtain the three-dimensional projection map.

3. The method according to claim 1, characterized in that The calculating the intersection-over-union ratio of the three-dimensional projection image and the two-dimensional feature image includes: When the three-dimensional projection image includes a first detection frame corresponding to the target object and the two-dimensional feature image includes a second detection frame corresponding to the target object, an intersection-over-union ratio of the first detection frame and the second detection frame is calculated.

4. The method according to claim 3, characterized in that The method further comprises: When the intersection-to-combination ratio is less than the preset threshold, generating a data return instruction; Based on the data return instruction, the target data is returned to the data platform, wherein the target data includes the image to be detected and the internal and external parameters of the camera; The BEV model is optimized based on the target data.

5. The method according to claim 1, characterized in that The method further comprises: When the first detection frame corresponding to the target object is not included in the three-dimensional projection image, and the second detection frame corresponding to the target object is included in the two-dimensional feature image, the target object is recalled by a preset means, and when the target object is included in the recall result, it is determined that the three-dimensional feature image is valid.

6. The method according to claim 1, characterized in that The outputting of the three-dimensional feature map of the target object by the BEV model includes: The visual transformation network in the BEV model transforms the two-dimensional feature map from a two-dimensional coordinate system to a three-dimensional coordinate system according to the internal and external parameters of the camera to obtain the three-dimensional feature map.

7. A target detection device, characterized in that: include: An image acquisition module is used to acquire the image to be detected captured by the camera and the internal and external parameters of the camera; A feature extraction module, used for inputting the image to be detected and the internal and external parameters of the camera into the BEV model, and outputting a two-dimensional feature map of the target object in the image to be detected by the target segmentation network in the BEV model, and outputting a three-dimensional feature map of the target object by the BEV model; A feature map projection module, used to project the three-dimensional feature map onto the plane where the two-dimensional feature map is located using the internal and external parameters of the camera to obtain a three-dimensional projection map; The result determination module is used to calculate the intersection-and-union ratio of the three-dimensional projection image and the two-dimensional feature image, and when the intersection-and-union ratio is greater than a preset threshold, determine that the three-dimensional feature image of the target object is valid.

8. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A vehicle, characterized in that: Comprising the electronic device as claimed in claim 8.

Images