Target perception method, computer device, storage medium and vehicle

Abstract

The present application relates to the technical field of automatic driving, and particularly provides a target perception method, computer equipment, storage medium and vehicle, aiming to solve the problem of improving target perception efficiency. The method provided by the present application comprises: obtaining 2D target points obtained by performing target detection on images; projecting the 2D target points to a normalized image plane of a camera corresponding to the images in 3D space to obtain 3D projection points of the 2D target points in 3D space; obtaining Key features and Value features according to the coordinates of the 3D projection points respectively; obtaining Query features according to the coordinates of each spatial point in a BEV space; performing cross-attention processing on the Key features, the Value features and the Query features to obtain BEV features of the target in the BEV space; and performing target perception on the BEV features to obtain 3D perception results of the target. Through the above method, the 3D perception efficiency of the target can be improved under the premise of ensuring the perception accuracy.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically to a target perception method, computer equipment, storage medium, and vehicle. Background Technology

[0002] Autonomous driving technology mainly includes several key technologies such as perception, localization, planning, and control. Among them, perception technology is mainly used to determine the obstacle information of the vehicle's driving environment, localization technology is mainly used to determine the vehicle's position, planning technology is mainly used to plan the vehicle's driving trajectory based on obstacle information and vehicle position, and control technology is mainly used to control the vehicle to drive according to the planned driving trajectory.

[0003] To improve the accuracy of 3D obstacle information and ensure safe vehicle operation, the current approach primarily involves training a multi-sensor perception model based on the BEV (Bird Eye View) perspective using images. This trained model is then used to infer and determine the 3D obstacle information of the vehicle's driving environment. However, using images as model input for training and inference results in a large number of model parameters and reduces the speed of training and inference, failing to meet the real-time perception requirements of 3D obstacle information and impacting vehicle driving safety.

[0004] Accordingly, a new technical solution is needed in this field to solve the above problems. Summary of the Invention

[0005] To overcome the above-mentioned deficiencies, the present invention is proposed to provide a target perception method, computer device, storage medium, and vehicle that solve or at least partially solve the technical problem of improving the efficiency of 3D target perception while ensuring the accuracy of 3D target perception.

[0006] In a first aspect, a target perception method is provided, the method comprising:

[0007] Obtain 2D target points obtained by target detection of the image, and project the 2D target points onto the normalized image plane of the camera corresponding to the image in 3D space to obtain the 3D projection points of the 2D target points in 3D space.

[0008] Based on the coordinates of the 3D projection points, the Key feature and Value feature are obtained respectively;

[0009] Based on the coordinates of each spatial point within the BEV space, obtain the Query features;

[0010] Cross-attention processing is performed based on the Key feature, the Value feature, and the Query feature to obtain the BEV feature of the target in the BEV space;

[0011] The BEV features are used for target perception to obtain the 3D perception results of the target.

[0012] In one technical solution of the above target perception method, the step of "obtaining Key features and Value features respectively based on the coordinates of the 3D projection points" specifically includes:

[0013] Obtain the coordinate difference between the coordinates of the 3D projection point and the coordinates of the optical center of the camera in 3D space;

[0014] Based on the coordinate difference, the Key feature and Value feature are obtained respectively.

[0015] In one technical solution of the above target perception method, the step of "obtaining Key features and Value features respectively based on the coordinate difference" specifically includes:

[0016] The coordinate difference is encoded using a neural network to form a first encoded feature;

[0017] The first encoded feature is used as the Key feature and the Value feature, respectively.

[0018] In one technical solution of the above target perception method, the step of "obtaining Query features based on the coordinates of each spatial point in the BEV space" specifically includes:

[0019] A neural network is used to encode the coordinates of each spatial point in the BEV space to form a second encoded feature. The initial query feature is obtained based on the feature sum between the second encoded feature and the initialized query feature.

[0020] A neural network is used to encode the coordinates of the optical center of the camera in 3D space to form a third encoded feature. The final query feature is obtained based on the feature difference between the third encoded feature and the initial query feature.

[0021] In one technical solution of the above-mentioned target perception method, after the step of "acquiring the BEV features of the target in the BEV space", the method further includes performing at least one more cross-attention process in the following manner, so as to use the BEV features obtained from the last cross-attention process as the final BEV features:

[0022] For each of the at least one cross-attention processes,

[0023] Based on the BEV features obtained from the previous cross-attention processing, obtain the Query features used in this cross-attention processing;

[0024] Based on the Key and Value features used in the previous cross-attention processing, obtain the Key and Value features used in this cross-attention processing respectively;

[0025] The BEV feature is obtained by performing this cross-attention process based on the Query feature, Key feature, and Value feature used in this cross-attention process.

[0026] In one technical solution of the above-mentioned target perception method, the step of "obtaining the Query features used in this cross-attention processing" specifically includes:

[0027] A neural network is used to encode the coordinates of the optical center of the camera in 3D space to form a third encoded feature;

[0028] Based on the feature difference between the third encoded feature and the BEV feature obtained from the previous cross-attention processing, the Query feature used in this cross-attention processing is obtained.

[0029] In one technical solution of the above-mentioned target perception method, the step of "obtaining the Key features and Value features used in this cross-attention processing" specifically includes:

[0030] A multilayer perceptron is used to process the key features and value features used in the previous cross-attention process separately.

[0031] The Key and Value features after perception processing will be used as the Key and Value features for this cross-attention processing, respectively.

[0032] In one technical solution of the above-mentioned target perception method, the step of "performing target perception on the BEV features to obtain the 3D perception result of the target" includes:

[0033] Target perception is performed on the BEV features, and multiple 3D perception results of the target and the confidence level of each 3D perception result are obtained based on the perception results.

[0034] The final 3D perception result is selected from the plurality of 3D perception results based on the confidence level.

[0035] In a second aspect, a computer device is provided, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and executed by the processor to perform the method described in any of the above-described technical solutions of the target perception method.

[0036] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the method described in any of the above-described target perception method techniques.

[0037] In a fourth aspect, a vehicle is provided that includes the computer equipment described in the above-described computer equipment technical solution.

[0038] Solution 1. A target perception method, characterized in that the method includes:

[0039] Obtain 2D target points obtained by target detection of the image, and project the 2D target points onto the normalized image plane of the camera corresponding to the image in 3D space to obtain the 3D projection points of the 2D target points in 3D space.

[0040] Based on the coordinates of the 3D projection points, the Key feature and Value feature are obtained respectively;

[0041] Based on the coordinates of each spatial point within the BEV space, obtain the Query features;

[0042] Cross-attention processing is performed based on the Key feature, the Value feature, and the Query feature to obtain the BEV feature of the target in the BEV space;

[0043] The BEV features are used for target perception to obtain the 3D perception results of the target.

[0044] Solution 2. The target perception method according to Solution 1, characterized in that the step of "obtaining Key features and Value features respectively based on the coordinates of the 3D projection points" specifically includes:

[0045] Obtain the coordinate difference between the coordinates of the 3D projection point and the coordinates of the optical center of the camera in 3D space;

[0046] Based on the coordinate difference, the Key feature and Value feature are obtained respectively.

[0047] Solution 3. The target perception method according to Solution 2, characterized in that the step of "obtaining Key features and Value features respectively based on the coordinate difference" specifically includes:

[0048] The coordinate difference is encoded using a neural network to form a first encoded feature;

[0049] The first encoded feature is used as the Key feature and the Value feature, respectively.

[0050] Solution 4. The target perception method according to Solution 1, characterized in that the step of "obtaining Query features based on the coordinates of each spatial point in the BEV space" specifically includes:

[0051] A neural network is used to encode the coordinates of each spatial point in the BEV space to form a second encoded feature. The initial query feature is obtained based on the feature sum between the second encoded feature and the initialized query feature.

[0052] A neural network is used to encode the coordinates of the optical center of the camera in 3D space to form a third encoded feature. The final query feature is obtained based on the feature difference between the third encoded feature and the initial query feature.

[0053] Solution 5. The target perception method according to any one of Solutions 1 to 4, characterized in that, after the step of "acquiring the BEV features of the target in the BEV space", the method further includes performing at least one more cross-attention process in the following manner, so as to use the BEV features obtained from the last cross-attention process as the final BEV features:

[0054] For each of the at least one cross-attention processes,

[0055] Based on the BEV features obtained from the previous cross-attention processing, obtain the Query features used in this cross-attention processing;

[0056] Based on the Key and Value features used in the previous cross-attention processing, obtain the Key and Value features used in this cross-attention processing respectively;

[0057] The BEV feature is obtained by performing this cross-attention process based on the Query feature, Key feature, and Value feature used in this cross-attention process.

[0058] Solution 6. The target perception method according to Solution 5, characterized in that the step of "obtaining the Query features used in this cross-attention processing" specifically includes:

[0059] A neural network is used to encode the coordinates of the optical center of the camera in 3D space to form a third encoded feature;

[0060] Based on the feature difference between the third encoded feature and the BEV feature obtained from the previous cross-attention processing, the Query feature used in this cross-attention processing is obtained.

[0061] Solution 7. The target perception method according to Solution 5, characterized in that the step of "obtaining the Key features and Value features used in this cross-attention processing" specifically includes:

[0062] A multilayer perceptron is used to process the key features and value features used in the previous cross-attention process separately.

[0063] The Key and Value features after perception processing will be used as the Key and Value features for this cross-attention processing, respectively.

[0064] Solution 8. The target perception method according to Solution 1, characterized in that the step of "performing target perception on the BEV features to obtain the 3D perception result of the target" includes:

[0065] Target perception is performed on the BEV features, and multiple 3D perception results of the target and the confidence level of each 3D perception result are obtained based on the perception results.

[0066] The final 3D perception result is selected from the plurality of 3D perception results based on the confidence level.

[0067] Scheme 9. A computer device comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by the processor to perform the target perception method as described in any one of Schemes 1 to 8.

[0068] Scheme 10. A computer-readable storage medium storing a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by a processor to perform the target perception method according to any one of Schemes 1 to 8.

[0069] Option 11. A vehicle, characterized in that the vehicle includes the computer equipment described in Option 9.

[0070] The above-described technical solutions of the present invention have at least one or more of the following beneficial effects:

[0071] In the technical solution of the target perception method provided by this invention, 2D target points obtained by target detection of images from the upstream system can be acquired. These 2D target points are then used as input data for data processing to output 3D perception results. In this process, since no image is used as input, image processing (such as image cropping), image feature extraction, and image feature processing are unnecessary, thereby greatly reducing the computational load of target perception, improving the speed of target perception, and overcoming the problem of poor real-time perception caused by using images as input data in existing technologies, which affects vehicle driving safety. Furthermore, in this technical solution, after obtaining the 2D target points, the 2D target points can be projected onto the normalized image plane of the corresponding camera in 3D space to obtain the 3D projection points of the 2D target points in 3D space. Key features and Value features are obtained based on the coordinates of the 3D projection points. Query features are obtained based on the coordinates of each spatial point in the BEV space. Cross-attention processing is performed on the Key features, Value features, and Query features to obtain the BEV features of the target in the BEV space. Finally, target perception is performed on the BEV features to obtain the 3D perception result of the target. Cross-attention processing can utilize the target's 2D information to accurately obtain the target's features in the BEV space, thereby effectively ensuring the accuracy of 3D perception results. Attached Figure Description

[0072] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Wherein:

[0073] Figure 1 This is a schematic flowchart of the main steps of a target perception method according to an embodiment of the present invention;

[0074] Figure 2 This is a schematic flowchart of the main steps of a method for obtaining Key features and Value features according to an embodiment of the present invention;

[0075] Figure 3 This is a schematic flowchart of the main steps of a method for obtaining query features according to an embodiment of the present invention;

[0076] Figure 4 This is a schematic flowchart of the main steps of a method for cross-attention processing according to an embodiment of the present invention;

[0077] Figure 5 This is a schematic diagram of the main structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0078] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0079] In the description of this invention, "processor" can include hardware, software, or a combination of both. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B.

[0080] The relevant user personal information that may be involved in the various embodiments of this application is processed in strict accordance with the requirements of laws and regulations, following the principles of legality, legitimacy, and necessity, based on the reasonable purpose of the business scenario, and includes personal information that users actively provide or that is generated as a result of using the product / service, as well as personal information obtained with user authorization.

[0081] The personal information of users processed by the applicant will vary depending on the specific product / service scenario and will be based on the specific scenario in which the user uses the product / service. This may involve the user's account information, device information, driving information, vehicle information, or other related information. The applicant will treat the user's personal information and its processing with a high degree of diligence.

[0082] The applicant attaches great importance to the security of users' personal information and has taken reasonable and feasible security protection measures that comply with industry standards to protect users' information and prevent unauthorized access, disclosure, use, modification, damage or loss of personal information.

[0083] The following describes an embodiment of the target perception method provided by the present invention.

[0084] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of a target perception method according to an embodiment of the present invention. Figure 1 As shown, the target perception method in this embodiment of the invention mainly includes the following steps S101 to S105.

[0085] Step S101: Obtain the 2D target points obtained by target detection of the image, and project the 2D target points onto the normalized image plane of the corresponding camera in 3D space to obtain the 3D projection points of the 2D target points in 3D space.

[0086] Images can be acquired through visual sensors (such as cameras) on the vehicle and input into a pre-defined perception model. This model performs object detection on the image, obtaining semantic information of the objects in the image, vector information of each target point (including at least the position and pose of the target point), etc. This perception model can be constructed using conventional methods in the field of neural network technology. This embodiment of the invention does not limit the specific structure and training method of the perception model, as long as it possesses the aforementioned perception capabilities.

[0087] The normalized image plane refers to the XY plane in 3D space with a depth of 1 from the optical center of the camera. The XY plane is a plane that is parallel to both the X-axis and Y-axis of this 3D space. Since each 2D target point is projected onto the same normalized image plane, each 3D projection point has the same depth value, which is 1. In this embodiment of the invention, conventional projection methods in the field of image technology can be used to project the 2D target point onto the above-mentioned normalized image plane based on the vector information of the 2D target point. This embodiment of the invention does not specifically limit the projection method.

[0088] Step S102: Obtain Key features and Value features based on the coordinates of the 3D projection points. Key features and Value features represent the key and value in the attention mechanism, respectively. In this embodiment of the invention, a neural network can be used to encode the coordinates of each projection point to form encoded features, which are then used as Key features and Value features, respectively. For example, a multilayer perceptron can be used to encode the coordinates of each projection point to form encoded features.

[0089] Step S103: Obtain Query features based on the coordinates of each spatial point in the BEV space. Query features represent the query in the attention mechanism. In this embodiment of the invention, a neural network can be used to encode the coordinates of each spatial point to form encoded features, which are then used as Query features. For example, a multilayer perceptron can be used to encode the coordinates of each spatial point to form encoded features.

[0090] Step S104: Perform cross-attention processing based on Key features, Value features, and Query features to obtain the BEV features of the target in the BEV space.

[0091] Since the Key feature comes from the coordinates of 3D projection points in 3D space, and the Query feature comes from the coordinates of spatial points in BEV space, the attention weight of the target at each spatial point in BEV space can be obtained through the cross-attention algorithm. Then, the attention weights and Value features are fused to obtain the BEV features of the target in BEV space. In this embodiment of the invention, the conventional cross-attention algorithm in the field of attention mechanism technology can be used for the above cross-attention processing; this embodiment of the invention does not impose specific limitations on this.

[0092] Step S105: Perform target perception on the BEV features to obtain the 3D perception result of the target. In this embodiment of the invention, a preset model with the ability to perform target perception on BEV features can be used. The BEV features obtained in step S104 are input into this model, and the target perception is performed through this model to obtain the 3D perception result of the target. This embodiment of the invention does not limit the specific structure and training method of the above model, as long as it has the above perception capability. For example, in some implementations, a multilayer perceptron with the above perception capability can be used to perform target perception on the BEV features.

[0093] In the method described in steps S101 to S105 above, the input data is the 2D target points obtained by target detection of the image, and the output data is the 3D perception result of the target. Since no image is used as input data, there is no need for image processing (such as image cropping), image feature extraction, or image feature processing, which can greatly reduce the computational load of target perception and improve the speed of target perception. In addition, by performing cross-attention processing based on Key features, Value features, and Query features, the BEV features of the target in the BEV space can be accurately obtained, thereby effectively ensuring the accuracy of the 3D perception result obtained based on the BEV features.

[0094] The following sections will provide further explanation of steps S102, S103, and S105.

[0095] I. Explanation of step S102.

[0096] See appendix Figure 2 In some embodiments of step S102 above, the Key feature and Value feature can be obtained through the following steps S1021 to S1022 respectively.

[0097] Step S1021: Obtain the coordinate difference between the coordinates of the 3D projection point and the coordinates of the camera's optical center in 3D space. Step S1022: Based on the coordinate difference, obtain the Key feature and Value feature respectively. In this step, a neural network can be used to encode the coordinate difference to form the first encoded feature, which is then used as the Key feature and Value feature respectively. For example, a multilayer perceptron can be used to encode the coordinate difference to form the first encoded feature.

[0098] In the method described in steps S1021 to S1022 above, replacing the real coordinates of the 3D projection point in 3D space with the coordinate difference (relative coordinates) between it and the camera optical center can associate the 3D projection point with the camera, that is, associate the Key feature and Value feature with the camera respectively, which is beneficial to obtaining more accurate BEV features in subsequent steps.

[0099] II. Explanation of step S103.

[0100] See appendix Figure 3 In some embodiments of step S103 above, the Query features can be obtained through the following steps S1031 to S1034.

[0101] Step S1031: Use a neural network to encode the coordinates of each spatial point in the BEV space to form a second encoded feature.

[0102] Step S1032: Obtain the initial Query feature based on the feature sum between the second encoded feature and the initialized Query feature. In this step, a conventional Query feature initialization method from the attention mechanism technology field can be used to initialize the feature, forming a Query feature suitable for the BEV space. Each feature point in this Query feature corresponds one-to-one with each spatial point in the BEV space. After obtaining this initialized Query feature, the second encoded feature of each spatial point in the BEV space is added to the corresponding feature point in the initialized Query feature, and the summed feature is used as the initial Query feature.

[0103] Step S1033: Use a neural network to encode the coordinates of the camera's optical center in 3D space to form a third encoded feature.

[0104] Step S1034: Obtain the final query feature based on the feature difference between the third encoded feature and the initial query feature. In this step, the initial query feature corresponding to each spatial point in the BEV space is subtracted from the third encoded feature, and the feature obtained after subtraction is taken as the final query feature.

[0105] In steps S1031 and S1033, a multilayer perceptron can be used to encode the coordinate differences to form a second encoded feature and a third encoded feature.

[0106] Similar to the methods described in steps S1021 to S1022, the final query features are obtained in the methods described in steps S1031 to S1034 based on the feature difference between the third encoded features and the initial query features. The query features can be associated with the camera, which is beneficial for obtaining more accurate BEV features in subsequent steps.

[0107] III. Explanation of step S105.

[0108] In some embodiments of step S105 above, target perception can be performed on the BEV features. Based on the perception results, multiple 3D perception results of the target and the corresponding confidence level for each 3D perception result are obtained. The confidence level represents the degree of credibility of the 3D perception result. Then, the final 3D perception result can be selected from the multiple 3D perception results based on the confidence level. For example, the 3D perception result with the highest confidence level can be selected as the final 3D perception result. Alternatively, multiple 3D perception results with the highest confidence levels can be selected, and the average of these 3D perception results can be used as the final 3D perception result.

[0109] Furthermore, in an embodiment of the target perception method provided by the present invention, after obtaining BEV features through cross-attention processing in step S104, at least one more cross-attention processing can be performed, and the BEV features obtained from the last cross-attention processing are used as the final BEV features. Target perception is then performed on this final BEV feature during step S105 to obtain the 3D perception result of the target. In some preferred embodiments, another cross-attention processing can be performed. In the embodiments of the present invention, the method of each cross-attention processing in the above-mentioned at least one cross-attention processing is the same, as described below in conjunction with the appendix. Figure 4 The method used in one instance of cross-attention processing will be explained. For example... Figure 4 As shown, in this embodiment of the invention, cross-attention processing can be performed through the following steps S201 to S203.

[0110] Step S201: Based on the BEV features obtained from the previous cross-attention processing, obtain the Query features used in this cross-attention processing.

[0111] In some implementations, a neural network can be used to encode the coordinates of the camera's optical center in 3D space to form a third encoded feature. Based on the feature difference between the third encoded feature and the BEV feature obtained from the previous cross-attention processing, the Query feature used in this cross-attention processing is obtained. Similar to the method described in steps S1021 to S1022 above, this approach associates the Query feature with the camera, thereby facilitating the obtaining of more accurate BEV features in subsequent steps.

[0112] Step S202: Based on the Key features and Value features used in the previous cross-attention processing, obtain the Key features and Value features used in this cross-attention processing.

[0113] In some implementations, a multilayer perceptron can be used to process the key and value features used in the previous cross-attention process separately, and the processed key and value features can then be used as the key and value features in the current cross-attention process. By using a multilayer perceptron for processing, the key and value features can be re-encoded to form new key and value features, which are then used in the current cross-attention process.

[0114] Step S203: Perform this cross-attention process based on the Query features, Key features, and Value features used in this cross-attention process to obtain the BEV features.

[0115] In the method described in steps S201 to S203 above, the Query features used in the current cross-attention processing are obtained from the BEV features obtained in the previous cross-attention processing. The results of the previous cross-attention processing can be fused into the current cross-attention processing. In this way, the accuracy of the BEV features can be gradually improved by performing multiple cross-attention processing.

[0116] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted solutions are equivalent to the technical solutions described in the present invention and therefore will also fall within the protection scope of the present invention.

[0117] Those skilled in the art will understand that all or part of the processes in the method of the above embodiment of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0118] Furthermore, the present invention also provides a computer device.

[0119] See appendix Figure 5 , Figure 5 This is a schematic diagram of the main structure of a computer device according to an embodiment of the present invention. Figure 5 As shown, the computer device in this embodiment of the invention mainly includes a storage device and a processor. The storage device can be configured to store a program for executing the target perception method of the above-described method embodiments, and the processor can be configured to execute the program in the storage device. This program includes, but is not limited to, the program for executing the target perception method of the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of the present invention are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of the present invention.

[0120] In embodiments of the present invention, the computer device may be a control device comprising various electronic devices. In some possible implementations, the computer device may include multiple storage devices and multiple processors. The program executing the target perception method of the above method embodiments may be divided into multiple subroutines, each subroutine may be loaded and run by a processor to execute different steps of the target perception method of the above method embodiments. Specifically, each subroutine may be stored in different storage devices, and each processor may be configured to execute programs in one or more storage devices to jointly implement the target perception method of the above method embodiments, that is, each processor executes different steps of the target perception method of the above method embodiments to jointly implement the target perception method of the above method embodiments.

[0121] The aforementioned multiple processors can be processors deployed on the same device. For example, the aforementioned computer device can be a high-performance device composed of multiple processors, and the aforementioned multiple processors can be processors configured on that high-performance device. Alternatively, the aforementioned multiple processors can also be processors deployed on different devices. For example, the aforementioned computer device can be a server cluster, and the aforementioned multiple processors can be processors on different servers within the server cluster.

[0122] Furthermore, the present invention also provides a computer-readable storage medium.

[0123] In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium may be configured to store a program for performing the target perception method of the above-described method embodiments. This program may be loaded and run by a processor to implement the target perception method. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The computer-readable storage medium may be a storage device comprising various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0124] Furthermore, the present invention also provides a vehicle.

[0125] In one embodiment of a vehicle according to the present invention, the vehicle may include the computer equipment described in the above-described computer equipment embodiments. In this embodiment, the vehicle may be an autonomous vehicle, an unmanned vehicle, or the like. Furthermore, according to the type of power source, the vehicle in this embodiment may be a gasoline vehicle, an electric vehicle, a hybrid vehicle that combines electric and gasoline power, or a vehicle using other new energy sources, etc.

[0126] The technical solution of the present invention has been described above with reference to one embodiment shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions resulting from such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A target perception method, characterized in that, The method includes: Obtain 2D target points obtained by target detection of the image, and project the 2D target points onto the normalized image plane of the camera corresponding to the image in 3D space to obtain the 3D projection points of the 2D target points in 3D space. Based on the coordinates of the 3D projection points, the Key feature and Value feature are obtained respectively; Based on the coordinates of each spatial point within the BEV space, obtain the Query features; Cross-attention processing is performed based on the Key feature, the Value feature, and the Query feature to obtain the BEV feature of the target in the BEV space; Target perception is performed on the BEV features to obtain the 3D perception results of the target; The steps of obtaining the Key feature and Value feature respectively include: Obtain the coordinate difference between the coordinates of the 3D projection point and the coordinates of the optical center of the camera in 3D space; based on the coordinate difference, obtain the Key feature and Value feature respectively.

2. The target perception method according to claim 1, characterized in that, The steps of "obtaining the Key feature and Value feature based on the coordinate difference" specifically include: The coordinate difference is encoded using a neural network to form a first encoded feature; The first encoded feature is used as the Key feature and the Value feature, respectively.

3. The target perception method according to claim 1, characterized in that, The steps of "obtaining query features based on the coordinates of each spatial point in the BEV space" specifically include: A neural network is used to encode the coordinates of each spatial point in the BEV space to form a second encoded feature. The initial query feature is obtained based on the feature sum between the second encoded feature and the initialized query feature. A neural network is used to encode the coordinates of the optical center of the camera in 3D space to form a third encoded feature. The final query feature is obtained based on the feature difference between the third encoded feature and the initial query feature.

4. The target perception method according to any one of claims 1 to 3, characterized in that, After the step of "obtaining the BEV features of the target in the BEV space", the method further includes performing at least one more cross-attention process in the following manner, so as to use the BEV features obtained from the last cross-attention process as the final BEV features: For each of the at least one cross-attention processes, Based on the BEV features obtained from the previous cross-attention processing, obtain the Query features used in this cross-attention processing; Based on the Key and Value features used in the previous cross-attention processing, obtain the Key and Value features used in this cross-attention processing respectively; The BEV feature is obtained by performing this cross-attention process based on the Query feature, Key feature, and Value feature used in this cross-attention process.

5. The target perception method according to claim 4, characterized in that, The steps for "obtaining the Query features used in this cross-attention processing" specifically include: A neural network is used to encode the coordinates of the optical center of the camera in 3D space to form a third encoded feature; Based on the feature difference between the third encoded feature and the BEV feature obtained from the previous cross-attention processing, the Query feature used in this cross-attention processing is obtained.

6. The target perception method according to claim 4, characterized in that, The steps for "obtaining the Key and Value features used in this cross-attention processing" specifically include: A multilayer perceptron is used to process the key features and value features used in the previous cross-attention process. The Key and Value features after perception processing will be used as the Key and Value features for this cross-attention processing, respectively.

7. The target perception method according to claim 1, characterized in that, The steps of "performing target perception on the BEV features to obtain 3D perception results of the target" include: Target perception is performed on the BEV features, and multiple 3D perception results of the target and the confidence level of each 3D perception result are obtained based on the perception results. The final 3D perception result is selected from the plurality of 3D perception results based on the confidence level.

8. A computer device comprising a processor and a storage device, said storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to perform the target perception method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the target perception method according to any one of claims 1 to 7.

10. A vehicle, characterized in that, The vehicle includes the computer equipment as described in claim 8.

Images