Image perception method, computer device, computer readable storage medium and vehicle

Abstract

The present application relates to the technical field of automatic driving, and particularly provides an image perception method, a computer device, a computer readable storage medium and a vehicle, aiming at solving the problem of improving the accuracy of image perception. To this end, the image perception method provided by the present application comprises acquiring an image frame collected by a vehicle, and performing image target perception on the image frame by using an image perception model. The training method of the image perception model comprises acquiring a teacher model capable of performing image depth estimation on the image frame, using a knowledge distillation method to make the teacher model guide the pre-training of the image perception model to use the image frame for image depth estimation, and performing final training of the image perception model which has completed the pre-training on the image target perception according to the image frame and its image target label information. Through the above image perception method, not only the accuracy of image perception can be improved, but also the robustness of image perception can be improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and specifically to an image perception method, a computer device, a computer-readable storage medium, and a vehicle. Background Technology

[0002] When controlling autonomous driving, visual sensors typically capture images of the vehicle's surroundings. These images are then processed by a perception model to identify objects such as lane lines, traffic signs, pedestrians, and obstacles. However, current conventional image perception methods often fail to obtain accurate depth information when identifying objects, thus reducing the accuracy of image perception.

[0003] Accordingly, a new technical solution is needed in this field to solve the above problems. Summary of the Invention

[0004] In order to overcome the above-mentioned defects, the present invention is proposed to provide an image sensing method, computer device, computer-readable storage medium and vehicle that solve or at least partially solve the technical problem of how to improve the accuracy of image perception.

[0005] In a first aspect, an image perception method is provided, the method comprising:

[0006] Acquire image frames captured by the vehicle;

[0007] Image target perception is performed on the image frame using an image perception model;

[0008] The image perception model is trained in the following manner:

[0009] Obtain a teacher model capable of performing image depth estimation on image frames;

[0010] A knowledge distillation method is used to enable the teacher model to guide the image perception model in pre-training for image depth estimation using image frames;

[0011] Based on the image frames and their image target annotation information, the pre-trained image perception model is used for final training of image target perception.

[0012] In one technical solution of the above image perception method, the step of "using a knowledge distillation method to guide the image perception model to perform pre-training for image depth estimation using image frames" specifically includes:

[0013] The teacher model is used to estimate the depth and confidence level at each pixel location in the image frame;

[0014] Obtain the positions of pixels whose confidence level at the depth is greater than a preset confidence threshold;

[0015] Based on the pixel position, obtain the high confidence region on the image frame;

[0016] A knowledge distillation method is employed to enable the teacher model to guide the image perception model in pre-training for image depth estimation using high-confidence regions on image frames.

[0017] In one technical solution of the above image perception method, the step of "obtaining the high-confidence region on the image frame based on the pixel position" specifically includes:

[0018] Based on the pixel positions, an image mask for the high-confidence region is generated;

[0019] Based on the image mask, obtain the high-confidence region on the image frame.

[0020] In one technical solution of the above image perception method, the step of "estimating the depth and confidence of each pixel location in the image frame using a teacher model" specifically includes:

[0021] If the teacher model is trained by image depth estimation based on image frames and their non-dense depth annotation information, then the teacher model is used to estimate the depth at the position of each pixel in the image frame and to estimate the probability that each pixel in the image frame will be scanned by the vehicle radar.

[0022] Based on the probability, the confidence level of the depth at each pixel location is determined.

[0023] In one technical solution of the above image perception method, the step of "estimating the depth and confidence of each pixel location in the image frame using a teacher model" specifically includes:

[0024] If the teacher model is trained by image depth estimation based on image frames and their dense depth annotation information, then the teacher model is used to estimate the depth at the location of each pixel in the image frame.

[0025] Based on the uncertainty estimation method, the uncertainty of the depth at each pixel location is estimated;

[0026] Based on the uncertainty, the confidence level of the depth at each pixel location is determined.

[0027] In one technical solution of the above-mentioned image perception method, before the step of "using a knowledge distillation method to guide the image perception model to perform pre-training for image depth estimation using image frames", the method further includes constructing the image perception model in the following manner:

[0028] A feature extraction network, an object perception network, and a depth estimation network are constructed respectively to form the image perception model;

[0029] The feature extraction network is used to extract image features from the image frame, the target perception network is used to perform image target perception based on the image features, and the depth estimation network is used to estimate the depth at the position of each pixel on the image frame based on the image features.

[0030] In one technical solution of the above image perception method, the step of "using a knowledge distillation method to guide the image perception model to perform pre-training for image depth estimation using image frames" specifically includes:

[0031] A knowledge distillation method is used to enable the teacher model to guide the image perception model in pre-training for image depth estimation using image frames;

[0032] Remove the depth estimation network after pre-training is complete.

[0033] In one technical solution of the above image perception method, the method further includes constructing the feature extraction network and the target perception network in the following manner:

[0034] Construct a feature pyramid network to form the feature extraction network;

[0035] Multiple target perception networks are constructed, each of which is used to perceive different types of image targets based on the image features extracted by the feature pyramid network.

[0036] In a second aspect, a computer device is provided, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and executed by the processor to perform the image perception method described in any of the above-described image perception methods.

[0037] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the image perception method described in any of the above-described image perception methods.

[0038] In a fourth aspect, a vehicle is provided that includes the computer equipment described in the above-described computer equipment technical solution.

[0039] Solution 1. An image perception method, characterized in that the method includes:

[0040] Acquire image frames captured by the vehicle;

[0041] Image target perception is performed on the image frame using an image perception model;

[0042] The image perception model is trained in the following manner:

[0043] Obtain a teacher model capable of performing image depth estimation on image frames;

[0044] A knowledge distillation method is used to enable the teacher model to guide the image perception model in pre-training for image depth estimation using image frames;

[0045] Based on the image frames and their image target annotation information, the pre-trained image perception model is used for final training of image target perception.

[0046] Scheme 2. The image perception method according to Scheme 1, characterized in that the step of "using a knowledge distillation method to guide the image perception model to perform pre-training for image depth estimation using image frames" specifically includes:

[0047] The teacher model is used to estimate the depth and confidence level at each pixel location in the image frame;

[0048] Obtain the positions of pixels whose confidence level at the depth is greater than a preset confidence threshold;

[0049] Based on the pixel position, obtain the high confidence region on the image frame;

[0050] A knowledge distillation method is employed to enable the teacher model to guide the image perception model in pre-training for image depth estimation using high-confidence regions on image frames.

[0051] Solution 3. The image perception method according to Solution 2, characterized in that the step of "obtaining the high-confidence region on the image frame based on the pixel position" specifically includes:

[0052] Based on the pixel positions, an image mask for the high-confidence region is generated;

[0053] Based on the image mask, obtain the high-confidence region on the image frame.

[0054] Solution 4. The image perception method according to Solution 2, characterized in that the step of "estimating the depth and confidence of each pixel location on the image frame using a teacher model" specifically includes:

[0055] If the teacher model is trained by image depth estimation based on image frames and their non-dense depth annotation information, then the teacher model is used to estimate the depth at the position of each pixel in the image frame and to estimate the probability that each pixel in the image frame will be scanned by the vehicle radar.

[0056] Based on the probability, the confidence level of the depth at each pixel location is determined.

[0057] Scheme 5. The image perception method according to Scheme 2, characterized in that the step of "estimating the depth and confidence level of each pixel location on the image frame using a teacher model" specifically includes:

[0058] If the teacher model is trained by image depth estimation based on image frames and their dense depth annotation information, then the teacher model is used to estimate the depth at the location of each pixel in the image frame.

[0059] Based on the uncertainty estimation method, the uncertainty of the depth at each pixel location is estimated;

[0060] Based on the uncertainty, the confidence level of the depth at each pixel location is determined.

[0061] Solution 6. The image perception method according to Solution 1, characterized in that, before the step of "using a knowledge distillation method to guide the image perception model to perform pre-training for image depth estimation using image frames", the method further includes constructing the image perception model in the following manner:

[0062] A feature extraction network, an object perception network, and a depth estimation network are constructed respectively to form the image perception model;

[0063] The feature extraction network is used to extract image features from the image frame, the target perception network is used to perform image target perception based on the image features, and the depth estimation network is used to estimate the depth at the position of each pixel on the image frame based on the image features.

[0064] Scheme 7. The image perception method according to Scheme 6, characterized in that the step of "using a knowledge distillation method to guide the image perception model to perform pre-training for image depth estimation using image frames" specifically includes:

[0065] A knowledge distillation method is used to enable the teacher model to guide the image perception model in pre-training for image depth estimation using image frames;

[0066] Remove the depth estimation network after pre-training is complete.

[0067] Solution 8. The image perception method according to Solution 6, characterized in that the method further includes constructing the feature extraction network and the target perception network in the following manner:

[0068] Construct a feature pyramid network to form the feature extraction network;

[0069] Multiple target perception networks are constructed, each of which is used to perceive different types of image targets based on the image features extracted by the feature pyramid network.

[0070] Scheme 9. A computer device comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by the processor to perform the image perception method as described in any one of Schemes 1 to 8.

[0071] Scheme 10. A computer-readable storage medium storing a plurality of program codes, characterized in that the program codes are adapted to be loaded and run by a processor to perform the image perception method of any one of Schemes 1 to 8.

[0072] Option 11. A vehicle, characterized in that the vehicle includes the computer equipment described in Option 9.

[0073] The above-described technical solutions of the present invention have at least one or more of the following beneficial effects:

[0074] In the technical solution of the image perception method provided by this invention, a teacher model capable of performing image depth estimation on image frames can be obtained. A knowledge distillation method is used to guide the image perception model in pre-training for image depth estimation using image frames. Then, based on the image frames, the pre-trained image perception model is finally trained for image target perception. When image perception is required on image frames acquired from vehicles, the trained image perception model is used to perform image target perception on the image frames.

[0075] The above method allows for the improvement of image depth estimation capabilities of image perception models without the need for depth information annotation on image frames. After training the depth-estimation-enabled model on image frames for final image target perception, the trained model can accurately identify targets within the image frames. Because the model possesses depth estimation capabilities, it can accurately identify both two-dimensional and three-dimensional image targets, thus improving the robustness of image target perception. Attached Figure Description

[0076] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Wherein:

[0077] Figure 1 This is a schematic flowchart of the main steps of an image perception method according to an embodiment of the present invention;

[0078] Figure 2This is a schematic diagram of the main steps of an image perception model training method according to an embodiment of the present invention;

[0079] Figure 3 This is a schematic diagram of the network structure of an image perception model according to an embodiment of the present invention;

[0080] Figure 4 This is a schematic diagram of the network structure of a teacher model according to an embodiment of the present invention;

[0081] Figure 5 This is a schematic diagram of the main structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0082] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0083] In the description of this invention, "processor" can include hardware, software, or a combination of both. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc.

[0084] The following describes an embodiment of the image perception method provided by the present invention.

[0085] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of an image perception method according to an embodiment of the present invention. Figure 1 As shown, the image perception method in this embodiment of the invention mainly includes the following steps S101 to S102.

[0086] Step S101: Acquire image frames captured by the vehicle.

[0087] In this embodiment of the invention, image frames can be acquired using an image acquisition device on the vehicle while the vehicle is traveling along the map acquisition path.

[0088] Step S102: Use an image perception model to perform image target perception on the image frame.

[0089] Prior to this step, a training step (S100) for an image perception model may be included to train the image perception model. Specifically, in embodiments of the present invention, this can be achieved through... Figure 2 The following steps S1001 to S1003 are shown to train the image perception model.

[0090] Step S1001: Obtain a teacher model capable of performing image depth estimation on image frames. The teacher model is a model trained using image frames and their depth annotation information for image depth estimation. The depth annotation information includes the depth at the pixel locations on the image frame.

[0091] Step S1002: Use the knowledge distillation method to guide the image perception model in pre-training using image frames for image depth estimation. In other words, use the image perception model as the student model and use the knowledge distillation method to distill the image depth estimation capability of the teacher model onto the image perception model, so that the image perception model also has image depth estimation capability.

[0092] Step S1003: Perform final training of the pre-trained image perception model for image target perception based on the image frames and their image target annotation information.

[0093] Image target annotation information is based on pre-annotated image frames. For example, it can be done manually by annotating the image targets contained in the image frame according to the image targets to be perceived, thus forming image target annotation information. This embodiment of the invention does not specifically limit the method for obtaining image target annotation information.

[0094] Through final training, the image perception model can acquire the ability to perceive image objects. Since the image perception model already has the ability to estimate image depth after pre-training, it can be used not only for two-dimensional image object perception but also for three-dimensional image object perception after the final training is completed, thus improving the robustness of image object perception.

[0095] Therefore, the method described in steps S101 to S102 above can not only improve the accuracy of the image perception model, but also improve the robustness of image target perception.

[0096] The following provides a further explanation of steps S1002 to S1003.

[0097] I. Explanation of step S1002.

[0098] In practical applications, due to limitations such as the accuracy of the image frame depth annotation information used during training, it may be impossible to guarantee that the teacher model can accurately estimate the depth at each pixel location in the image frame. For example, if the depth annotation information at some pixel locations in the image frame is inaccurate, it will affect the teacher model's depth estimation at some pixel locations. Therefore, to ensure the image depth estimation capability of the image perception model, when using the teacher model to guide the training of the image perception model, the more accurate pixel location depths obtained by the teacher model can be used to guide the training of the image perception model. Specifically, in some implementations, the image perception model can be pre-trained for image depth estimation through the following steps 11 to 14.

[0099] Step 11: Use the teacher model to estimate the depth and its confidence level at each pixel location in the image frame. The confidence level of the depth represents the degree of confidence in the depth, which in turn represents the accuracy of the confidence level. The higher the confidence level, the higher the accuracy, and vice versa.

[0100] Step 12: Obtain the locations of pixels whose depth confidence score is greater than a preset confidence threshold. If the confidence score is greater than the preset confidence threshold, it indicates that the accuracy of the confidence score is high and can be used to guide the image perception model in training for image depth estimation.

[0101] Those skilled in the art can flexibly set the specific value of the preset confidence threshold according to actual needs, and the embodiments of the present invention do not impose specific limitations on this.

[0102] Step 13: Based on the pixel locations where the depth confidence score is greater than a preset confidence score threshold, obtain the high-confidence region on the image frame. Specifically, the region formed by these pixel locations can be considered as the high-confidence region.

[0103] Step 14: Employ knowledge distillation to guide the image perception model in pre-training using high-confidence regions on image frames for image depth estimation. In other words, knowledge distillation transfers the teacher model's ability to estimate image depth using high-confidence regions to the image perception model, thereby improving the model's depth estimation capabilities.

[0104] Based on the method described in steps 11 to 14 above, the problem of the image depth estimation capability of the image perception model being affected by the inaccurate depth estimation of some image frame regions by the teacher model can be avoided, thus ensuring that the image perception model has a high image depth estimation capability.

[0105] The following provides further explanation of steps 11 and 13 above.

[0106] 1. Explanation of step 11.

[0107] In practical applications, image frames in different scenarios may be annotated with dense depth annotations or with non-dense depth annotations. For example, for image frames of indoor scenes, to ensure the accuracy of depth estimation, the depth may be annotated at the location of every pixel in the image frame to form dense depth annotations; for image frames of open outdoor scenes, to ensure the efficiency of depth estimation, the depth may only be annotated at a subset of the pixel locations in the image frame to form non-dense depth annotations.

[0108] For teacher models trained with dense depth annotations and those trained with non-dense depth annotations, different methods can be used to obtain the confidence level of the depth estimated by the teacher model. These two cases are explained below.

[0109] (1) The teacher model is trained by image depth estimation based on image frames and their non-dense depth annotation information.

[0110] In this scenario, after inputting the image frame into the teacher model, the teacher model can estimate the depth at each pixel location in the image frame and the probability that each pixel will be scanned by the vehicle's radar. Then, the confidence level of the depth at each pixel location is determined based on the probability. Probability and confidence level are positively correlated; that is, the higher the probability, the higher the confidence level, and vice versa. Therefore, the probability can be directly used as the confidence level. It should be noted that while the image frame is being acquired using the vehicle's image acquisition device, the vehicle's radar is also scanning the environment. Therefore, the confidence level can be obtained by estimating the probability that each pixel will be scanned by the vehicle's radar.

[0111] (2) The teacher model is trained by image depth estimation based on image frames and their dense depth annotation information.

[0112] In this case, after the image frame is input into the teacher model, the teacher model can be used to estimate the depth at each pixel position on the image frame, and the uncertainty of the depth at each pixel position can be estimated based on the uncertainty estimation method. The confidence of the depth at each pixel position can then be determined based on the uncertainty.

[0113] Uncertainty and confidence are negatively correlated; the higher the uncertainty, the lower the confidence, and vice versa. Uncertainty can be inverted, for example, by calculating the reciprocal of the uncertainty and using the result of the inversion as the confidence level.

[0114] In this embodiment of the invention, conventional uncertainty estimation methods can be used to estimate the depth uncertainty at each pixel location. This embodiment does not impose specific limitations on this method. For example, the uncertainty estimation method disclosed in the paper "BoundingBox Regression With Uncertainty for Accurate Object Detection" can be used. This method simplifies the depth confidence level by modeling it as a Gaussian distribution, where the standard deviation in the Gaussian distribution function represents the depth uncertainty. When the standard deviation approaches 0, it indicates that the uncertainty is very high.

[0115] 2. Explanation of step 13.

[0116] In order to quickly and accurately obtain high-confidence regions on an image frame based on pixel positions, some implementations can first generate an image mask of the high-confidence regions based on pixel positions, and then obtain the high-confidence regions on the image frame based on the image mask.

[0117] In this embodiment of the invention, conventional image mask generation methods can be used to generate image masks of high-confidence regions based on pixel positions. This embodiment of the invention does not specifically limit the image mask generation method.

[0118] The above is an explanation of steps 11 and 13.

[0119] In order to train an image perception model to estimate image depth and enable it to estimate image depth, in addition to setting up a feature extraction network and an object perception network, a depth estimation network can also be set up when constructing the image perception model.

[0120] Feature extraction networks can be used to extract image features from image frames, target perception networks can be used to perceive image targets based on image features, and depth estimation networks can be used to estimate the depth at the location of each pixel in an image frame based on image features.

[0121] After inputting an image frame into the image perception model, the depth estimation network can estimate the depth at each pixel location in the image frame based on its image features. For ease of description, the depth estimated by the depth estimation network is referred to as the student estimated depth. Similarly, after inputting an image frame into the teacher model, the teacher model can also estimate the depth at each pixel location in the image frame. For ease of description, the depth estimated by the teacher model is referred to as the teacher estimated depth. After obtaining the student and teacher estimated depths, the model loss value of the image perception model can be calculated based on these values. The parameter gradients of the model parameters are then calculated based on the model loss value. The model parameters are updated via backpropagation based on these parameter gradients until the image perception model meets the convergence condition, at which point training stops. This completes the knowledge distillation training, distilling the image depth estimation capability of the teacher model into the image perception model.

[0122] After completing the pre-training of the image depth estimation, the depth estimation network can be removed. That is, when performing the final training of the image perception model for image object perception, the image perception model no longer includes the aforementioned depth estimation network.

[0123] In practical applications, vehicles may be equipped with multiple image acquisition devices from different perspectives. When performing target perception, image frames from these different perspectives are simultaneously acquired by these devices. To ensure that the image perception model has a high depth estimation capability for each perspective image frame, depth estimation training can be performed separately for each perspective image frame. For example, if a vehicle has two image acquisition devices A and B with different perspectives, image frames acquired by device A are acquired first, and these frames are used to train the image perception model for depth estimation. Then, image frames acquired by device B are acquired, and these frames are used to train the image perception model for depth estimation.

[0124] II. Explanation of step S1003.

[0125] As described above, when training image target perception, the image perception model mainly includes a feature extraction network and a target perception network, but does not include a depth estimation network. This training mainly utilizes the target perception network for image target perception training.

[0126] After inputting the image frame and its image target annotation information into the image perception model, the target perception network can perform image target perception based on the image features of the image frame to obtain image target perception information. Then, based on the image target annotation information and the image target perception information, the model loss value of the image perception model is calculated, the parameter gradient of the model parameters is calculated based on the model loss value, and the model parameters are updated by backpropagation based on the parameter gradient until the image perception model meets the convergence condition and training stops.

[0127] To improve the robustness of image perception models for object perception, multiple object perception models can be incorporated into the overall model. Each object perception network is used to perceive different types of image objects based on the image features of the image frame. For example, 2D object perception networks, 3D object perception networks, and BEV (Bird's Eye View) object perception networks can be set up. Furthermore, to further enhance the robustness of image object perception, a feature pyramid network can be used to construct a feature extraction network. This network can extract image features at different scales from the image frame, allowing for object perception based on these features at each scale.

[0128] like Figure 3 As shown, the feature extraction network in the image perception model mainly includes a backbone network and a feature pyramid network. The feature pyramid network is connected to three multi-task networks. The feature pyramid network can extract image features at three different scales and input the image features at each scale into a separate multi-task network. The multi-task networks include a depth estimation network and an object perception network. The object perception network includes a 2D object perception network, a 3D object perception network, and a BEV object perception network. During the pre-training of the image perception model for image depth estimation, the depth estimation network in the multi-task network is mainly used for training, and this depth estimation network is removed after pre-training. During the final training of the image perception model for image object perception, the object perception network is mainly used for training. The input data of the BEV object perception network is not the original image features extracted by the feature pyramid network, but rather the BEV image features formed by processing the original image features.

[0129] Since the image depth estimation pre-training of the image perception model uses a teacher model for distillation training, the scale of the image features extracted by the teacher model from the previous frame needs to be consistent with the scale of the image features extracted by the image perception model to ensure training accuracy. Therefore, in this embodiment of the invention, if a feature pyramid network is used to construct the feature extraction network of the image perception model, a feature pyramid network can also be used to construct the feature extraction network of the teacher model, ensuring that the image features extracted by the two feature pyramid networks are at the same scale.

[0130] like Figure 4 As shown, the feature extraction network in the teacher model mainly includes a backbone network and a feature pyramid network. The feature pyramid network is connected to two teacher network depth prediction networks. The feature pyramid network can extract image features at two different scales and input the image features at each scale into a separate teacher network depth prediction network. The teacher network depth prediction network includes a depth estimation network and a depth confidence estimation network. The depth estimation network can be used to estimate the depth at each pixel location on the image frame, and the depth confidence estimation network can be used to estimate the confidence of the depth at each pixel location on the image frame. The method for estimating the confidence is the same as that described in step 11 of the aforementioned method embodiment, and will not be repeated here.

[0131] Through experimental comparison in this embodiment of the invention, it can be determined that the performance metrics of the image perception model pre-trained with image depth estimation are significantly improved compared to the image perception model without image depth estimation pre-training. For example, the Average Precision (AP) metric for BEV target perception using the image perception model without image depth estimation pre-training is 34.0, while the AP metric of the image perception model pre-trained with image depth estimation is 37.6, an improvement of 10.6%.

[0132] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.

[0133] Those skilled in the art will understand that all or part of the processes in the method of the above embodiment of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0134] Furthermore, the present invention also provides a computer device.

[0135] See appendix Figure 5 , Figure 5 This is a schematic diagram of the main structure of a computer device according to an embodiment of the present invention. Figure 5 As shown, the computer device in this embodiment of the invention mainly includes a storage device and a processor. The storage device can be configured to store a program for executing the image perception method of the above-described method embodiments, and the processor can be configured to execute the program in the storage device. This program includes, but is not limited to, the program for executing the image perception method of the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of the present invention are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of the present invention.

[0136] In embodiments of the present invention, the computer device may be a control device comprising various electronic devices. In some possible implementations, the computer device may include multiple storage devices and multiple processors. The program executing the image perception method of the above method embodiments may be divided into multiple subroutines, each subroutine being loaded and run by a processor to execute different steps of the image perception method of the above method embodiments. Specifically, each subroutine may be stored in different storage devices, and each processor may be configured to execute programs in one or more storage devices to jointly implement the image perception method of the above method embodiments, that is, each processor executes different steps of the image perception method of the above method embodiments to jointly implement the image perception method of the above method embodiments.

[0137] The aforementioned multiple processors can be processors deployed on the same device. For example, the aforementioned computer device can be a high-performance device composed of multiple processors, and the aforementioned multiple processors can be processors configured on that high-performance device. Alternatively, the aforementioned multiple processors can also be processors deployed on different devices.

[0138] Furthermore, the present invention also provides a computer-readable storage medium.

[0139] In one embodiment of a computer-readable storage medium according to the present invention, the computer-readable storage medium may be configured to store a program for performing the image perception method of the above-described method embodiments. This program may be loaded and run by a processor to implement the image perception method. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The computer-readable storage medium may be a storage device comprising various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0140] Furthermore, the present invention also provides a vehicle.

[0141] In one embodiment of a vehicle according to the present invention, the vehicle may include the computer equipment described in the above-described computer equipment embodiments. In this embodiment, the vehicle may be an autonomous vehicle, an unmanned vehicle, or the like. Furthermore, according to the type of power source, the vehicle in this embodiment may be a gasoline vehicle, an electric vehicle, a hybrid vehicle that combines electric and gasoline power, or a vehicle using other new energy sources, etc.

[0142] The technical solution of the present invention has been described above with reference to one embodiment shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions resulting from such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. An image perception method, characterized in that, The method includes: Acquire image frames captured by the vehicle; Image target perception is performed on the image frame using an image perception model; The image perception model is trained in the following manner: Obtain a teacher model capable of performing image depth estimation on image frames; A feature extraction network, a target perception network, and a depth estimation network are constructed to form the image perception model. The feature extraction network is used to extract image features from the image frame, the target perception network is used to perform image target perception based on the image features, and the depth estimation network is used to estimate the depth at the position of each pixel in the image frame based on the image features. A knowledge distillation method is used to enable the teacher model to guide the image perception model to perform pre-training for image depth estimation using image frames, and the depth estimation network is removed after the pre-training is completed. Based on the image frames and their image target annotation information, the pre-trained image perception model is used for final training of image target perception.

2. The image perception method according to claim 1, characterized in that, The steps of "using knowledge distillation to guide the image perception model in pre-training using image frames for image depth estimation" specifically include: The teacher model is used to estimate the depth and confidence level at each pixel location in the image frame; Obtain the positions of pixels whose confidence level at the depth is greater than a preset confidence threshold; Based on the pixel position, obtain the high confidence region on the image frame; A knowledge distillation method is employed to enable the teacher model to guide the image perception model in pre-training for image depth estimation using high-confidence regions on image frames.

3. The image perception method according to claim 2, characterized in that, The step of "obtaining the high-confidence region on the image frame based on the pixel position" specifically includes: Based on the pixel positions, an image mask for the high-confidence region is generated; Based on the image mask, obtain the high-confidence region on the image frame.

4. The image perception method according to claim 2, characterized in that, The steps of "estimating the depth and confidence level of each pixel in the image frame using a teacher model" specifically include: If the teacher model is trained by image depth estimation based on image frames and their non-dense depth annotation information, then the teacher model is used to estimate the depth at the position of each pixel in the image frame and to estimate the probability that each pixel in the image frame will be scanned by the vehicle radar. Based on the probability, the confidence level of the depth at each pixel location is determined.

5. The image perception method according to claim 2, characterized in that, The steps of "estimating the depth and confidence level of each pixel in the image frame using a teacher model" specifically include: If the teacher model is trained by image depth estimation based on image frames and their dense depth annotation information, then the teacher model is used to estimate the depth at the location of each pixel in the image frame. Based on the uncertainty estimation method, the uncertainty of the depth at each pixel location is estimated; Based on the uncertainty, the confidence level of the depth at each pixel location is determined.

6. The image perception method according to claim 1, characterized in that, The method further includes constructing the feature extraction network and the target perception network in the following manner: Construct a feature pyramid network to form the feature extraction network; Multiple target perception networks are constructed, each of which is used to perceive different types of image targets based on the image features extracted by the feature pyramid network.

7. A computer device comprising a processor and a storage device, said storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to perform the image perception method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the image perception method according to any one of claims 1 to 6.

9. A vehicle, characterized in that, The vehicle includes the computer equipment as described in claim 7.

Images