Environment sensing method, device, equipment, medium and program product

Abstract

The invention discloses an environment sensing method, device and equipment, a medium and a program product, and relates to the technical field of automatic driving. The method comprises the following steps: performing data enhancement on a labeled driving image and an unlabeled driving image to obtain a labeled enhanced image and an unlabeled enhanced image; extracting labeled enhanced image features and unlabeled enhanced image features by adopting a feature extraction network, and calculating domain adaptation loss according to a distribution distance between the labeled enhanced image features and the unlabeled enhanced image features; adopting a classifier to obtain a labeled environment perception result and an unlabeled environment perception result, and calculating supervision loss; and the supervision loss and the domain adaptation loss are integrated to obtain total loss, and an optimizer is adopted to train a feature extraction network and a classifier by taking the minimization of the total loss as a target to obtain an environment perception model. According to the technical scheme of the embodiment of the invention, the recognition precision, generalization ability and calculation efficiency of the environment perception model are improved under the condition that the annotation data are scarce.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and in particular to an environment perception method, device, equipment, medium and program product. BACKGROUND

[0002] The environment perception system of an intelligent connected vehicle is the core of its automatic driving function, relying on a deep learning model to accurately identify targets such as vehicles and pedestrians. However, the training of high-performance models is heavily dependent on massive amounts of high-quality labeled data. In actual applications, the cost of data labeling is extremely high, and there are a large number of rare scenarios in real road conditions, resulting in a lack of labeled data, which is a key bottleneck for technology to land.

[0003] To address the problem of data scarcity, existing technologies such as generative adversarial network (GAN) data augmentation methods are widely used. This type of method generates synthetic images to expand the training dataset, thereby improving model performance. However, this method has significant drawbacks: first, the quality of the generated images is poor under extreme small sample conditions, with blurring and semantic distortion problems, and the noise introduced can mislead model training, leading to decreased recognition accuracy and increased false positive rate; second, this method cannot solve the domain shift problem between labeled data and real and variable driving environments, and the model's generalization ability in unknown scenarios is still insufficient; finally, the training complexity of the generative adversarial network (GAN) model is high, and the computational resource consumption is large, making it difficult to meet the stringent requirements of vehicle-mounted platforms for real-time performance and low power consumption. SUMMARY

[0004] The present application provides an environment perception method, device, equipment, medium and program product, which improves the recognition accuracy, generalization ability and computational efficiency of the environment perception model in the case of scarce labeled data.

[0005] According to an aspect of the present application, an environment perception method is provided, the method comprising:

[0006] Obtaining labeled driving images, labeled labels, and unlabeled driving images, and performing data augmentation on the labeled driving images and the unlabeled driving images to obtain labeled augmented images and unlabeled augmented images;

[0007] Using a feature extraction network to perform feature extraction on the labeled augmented images and the unlabeled augmented images respectively to obtain labeled augmented image features and unlabeled augmented image features, and calculating a domain adaptation loss according to the distribution distance between the labeled augmented image features and the unlabeled augmented image features;

[0008] The classifier is used to classify the labeled enhanced image features to obtain a labeled environment perception result, and a supervision loss is calculated according to a difference between the labeled environment perception result and the labeled label.

[0009] The supervision loss and the domain adaptation loss are integrated to obtain a total loss, and an optimizer is used to train the feature extraction network and the classifier to minimize the total loss, thereby obtaining an environment perception model.

[0010] According to another aspect of the present application, an environment perception device is provided, which comprises:

[0011] A data enhancement module is configured to obtain a labeled driving image, a labeled label, and an unlabeled driving image, and perform data enhancement on the labeled driving image and the unlabeled driving image to obtain a labeled enhanced image and an unlabeled enhanced image.

[0012] A domain adaptation loss calculation module is configured to use a feature extraction network to perform feature extraction on the labeled enhanced image and the unlabeled enhanced image respectively to obtain labeled enhanced image features and unlabeled enhanced image features, and calculate a domain adaptation loss according to a distribution distance between the labeled enhanced image features and the unlabeled enhanced image features.

[0013] A supervision loss calculation module is configured to use a classifier to classify the labeled enhanced image features to obtain a labeled environment perception result, and calculate a supervision loss according to a difference between the labeled environment perception result and the labeled label.

[0014] An environment perception model generation module is configured to integrate the supervision loss and the domain adaptation loss to obtain a total loss, and use an optimizer to train the feature extraction network and the classifier to minimize the total loss, thereby obtaining an environment perception model.

[0015] According to another aspect of the present application, an electronic device is provided, which comprises:

[0016] at least one processor; and

[0017] a memory in communication connection with the at least one processor; wherein

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the environment perception method according to any one of the embodiments of the present application.

[0019] According to another aspect of the present application, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement the environment perception method according to any of the embodiments of the present application when executed.

[0020] According to another aspect of the present application, there is provided a computer program product comprising a computer program for implementing the environment perception method according to any of the embodiments of the present application when executed by a processor.

[0021] The technical scheme of the embodiment of the present application, by performing data augmentation on the annotated driving images and the unannotated driving images, obtains annotated augmented images and unannotated augmented images, greatly expands the diversity of the training samples through simple image transformation, and avoids the problem of introducing noise due to the generation of low-quality images by the traditional generative method; thus, the environment perception model can learn the invariant features of the target from more diversified training samples, significantly improving the recognition accuracy and generalization ability in complex situations such as light changes and local occlusions, while ensuring the efficiency and low cost of the data preparation process; by using a feature extraction network, the annotated augmented images and the unannotated augmented images are respectively subjected to feature extraction to obtain annotated augmented image features and unannotated augmented image features, and according to the distribution distance between the annotated augmented image features and the unannotated augmented image features, a domain adaptation loss is calculated, which overcomes the distribution difference between different driving environments, can drive the feature extraction network to ignore domain-specific interference such as weather, sensor model and background landscape, so as to focus on the essential features of the environment perception result rather than irrelevant environmental factors, thereby maintaining stable and high-precision perception performance when facing rain and fog weather and unfamiliar urban roads that are not seen during training, greatly enhancing the cross-domain adaptability and reliability of the environment perception model; by using a classifier, the annotated augmented image features are classified to obtain annotated environment perception results, and according to the difference between the annotated environment perception results and the annotated labels, a supervision loss is calculated, which considers the overall difference between the model-predicted environment perception results and the real environment perception results; by synthesizing the supervision loss and the domain adaptation loss to obtain a total loss, and using an optimizer to train the feature extraction network and the classifier with the goal of minimizing the total loss, an environment perception model is obtained, which can fully utilize the annotated driving images and the unannotated driving images, does not need to generate complex models, and finally obtains an environment perception model with excellent comprehensive performance, which is computationally efficient and easy to deploy to resource-constrained vehicle-mounted computing platforms, and can meet all requirements of the intelligent networked automobile for the environment perception system in terms of high real-time performance, high accuracy and high reliability.

[0022] It is to be understood that the embodiments described herein are merely exemplary of the application and that a person skilled in the art can devise other embodiments without departing from the scope of the present application. It is also to be understood that not all of the features and aspects of the present application are necessarily implemented in a commercial embodiment of the application. Indeed, various modifications of the application in addition to those described herein will be apparent to those skilled in the art from the foregoing description and accompanying drawings. Such modifications are also intended to fall within the scope of the application. It is further understood that all features of the application are interchangeable and that the various embodiments can be interchanged while still falling within the scope of the application. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0024] Figure 1 is a flow chart of an environment perception method according to an embodiment of the present application;

[0025] Figure 2 is a flow chart of an environment perception method according to an embodiment of the present application;

[0026] Figure 3 is a flow chart of an environment perception method according to an embodiment of the present application;

[0027] Figure 4 is a structural schematic diagram of an environment perception device according to an embodiment of the present application;

[0028] Figure 5 is a structural schematic diagram of an electronic device implementing an environment perception method according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the technical personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present application and the above-described accompanying drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0031] Embodiment one

[0032] Figure 1 A flowchart of an environment perception method provided for the first embodiment of the present application. The embodiments of the present application can be applicable to the case of environment perception for driving environment, and the method is executed by an environment perception device which is realized in the form of hardware and / or software, and the environment perception device can be configured in an electronic device carrying an environment perception function.

[0033] Referring to Figure 1 The environment perception method shown includes:

[0034] S101, obtaining labeled driving images, labeled labels, and unlabeled driving images, and performing data enhancement on the labeled driving images and the unlabeled driving images to obtain labeled enhanced images and unlabeled enhanced images.

[0035] The labeled driving image is a driving image that has been labeled with an actual environment perception result. The labeled label is the actual environment perception result corresponding to the labeled driving image. Optionally, the labeled label is pre-identified and labeled by a technician on the labeled driving image. Illustratively, the labeled label includes vehicles, pedestrians, or obstacles in the labeled driving image, etc. The unlabeled driving image is a driving image that has not been labeled with an actual environment perception result. In comparison, the labeled driving image has a corresponding labeled label; while the unlabeled driving image does not have a corresponding labeled label. The labeled enhanced image is a data enhancement result of the labeled driving image. The unlabeled enhanced image is a data enhancement result of the unlabeled driving image. Optionally, the data enhancement manner of the labeled driving image and the unlabeled driving image includes at least one of small rotation, translation, flipping, color jittering, contrast adjustment, and random occlusion, etc. Among them, the data enhancement manner of the labeled driving image and the unlabeled driving image can be the same or different.

[0036] Specifically, a pre-labeled labeled driving image and a labeled label are obtained. An unlabeled driving image is obtained. At least one of small rotation, translation, flipping, color jittering, contrast adjustment, and random occlusion is used to perform data augmentation on the labeled driving image and the unlabeled driving image respectively to obtain a labeled augmented image and an unlabeled augmented image.

[0037] In S102, a feature extraction network is used to perform feature extraction on the labeled augmented image and the unlabeled augmented image respectively to obtain labeled augmented image features and unlabeled augmented image features, and a domain adaptation loss is calculated according to a distribution distance between the labeled augmented image features and the unlabeled augmented image features.

[0038] The feature extraction network is used to perform feature extraction on the labeled augmented image or the unlabeled augmented image. The labeled augmented image features are the feature extraction result of the labeled augmented image. The unlabeled augmented image features are the feature extraction result of the unlabeled augmented image. The domain adaptation loss is the distribution distance between the labeled augmented image features and the unlabeled augmented image features. The domain adaptation loss represents the difference between the labeled augmented image features and the unlabeled augmented image features from the feature extraction dimension.

[0039] Specifically, the labeled augmented image and the unlabeled augmented image are input into the feature extraction network to obtain the labeled augmented image features and the unlabeled augmented image features. A maximum mean difference algorithm is used to calculate the distribution distance between the labeled augmented image features and the unlabeled augmented image features to obtain the domain adaptation loss.

[0040] In S103, a classifier is used to classify the labeled augmented image features to obtain a labeled environment perception result, and a supervision loss is calculated according to a difference between the labeled environment perception result and the labeled label.

[0041] The classifier is used to classify the labeled augmented image features to predict the labeled environment perception result. The labeled environment perception result is the prediction result obtained by the classifier based on the labeled augmented image features. The supervision loss is used to represent the difference between the labeled environment perception result and the labeled label.

[0042] Specifically, the labeled augmented image features are input into the classifier to obtain the labeled environment perception result. The cross entropy between the labeled environment perception result and the labeled label is calculated to obtain the supervision loss.

[0043] In S104, the supervision loss and the domain adaptation loss are integrated to obtain a total loss, and an optimizer is used to train the feature extraction network and the classifier with the goal of minimizing the total loss to obtain an environment perception model.

[0044] The total loss is used to comprehensively represent the cumulative error of the environment perception model. The total loss is used to reflect the accuracy of the prediction of the environment perception model. It can be understood that the smaller the total loss is, the higher the prediction accuracy of the environment perception model is. The optimizer is used to optimize the training of the environment perception model. The environment perception model is a model that has been trained for environment perception. The input data of the environment perception model is a driving image, and the output result is an environment perception result. The structure of the environment perception model includes a feature extraction network and a classifier.

[0045] Specifically, the supervised loss and the domain adaptation loss are weighted and summed to obtain the total loss. An optimizer is used to train the feature extraction network and the classifier according to the gradient descent algorithm, with the goal of minimizing the loss function, to obtain the environment perception model.

[0046] The technical scheme of the embodiment of the present application obtains the labeled enhanced image and the unlabeled enhanced image by performing data enhancement on the labeled driving image and the unlabeled driving image, greatly expands the diversity of the training samples through simple image transformation, and avoids the problem of introducing noise due to the generation of low-quality images in the traditional generative method. Thus, the environment perception model can learn the invariant features of the target from more diversified training samples, significantly improving the recognition accuracy and generalization ability in complex situations such as light changes and local occlusions, while ensuring the efficiency and low cost of the data preparation process. By using the feature extraction network, the labeled enhanced image features and the unlabeled enhanced image features are obtained by performing feature extraction on the labeled enhanced image and the unlabeled enhanced image respectively, and the domain adaptation loss is calculated according to the distribution distance between the labeled enhanced image features and the unlabeled enhanced image features, which overcomes the distribution difference between different driving environments and drives the feature extraction network to ignore domain-specific interference such as weather, sensor model and background landscape, so as to focus on the essential features of the environment perception result rather than irrelevant environmental factors, thereby maintaining stable and high-precision perception performance when facing rain and fog weather and unfamiliar urban roads that are not seen during training, greatly enhancing the cross-domain adaptability and reliability of the environment perception model. By using the classifier, the labeled environment perception result is obtained by classifying the labeled enhanced image features, and the supervised loss is calculated according to the difference between the labeled environment perception result and the labeled label, which considers the overall difference between the predicted environment perception result of the model and the real environment perception result. By comprehensively obtaining the total loss from the supervised loss and the domain adaptation loss, and using the optimizer to train the feature extraction network and the classifier with the goal of minimizing the total loss, the environment perception model can fully utilize the labeled driving image and the unlabeled driving image, without generating complex models, and finally obtain an environment perception model with excellent comprehensive performance, which is computationally efficient and easy to deploy to resource-constrained vehicle-mounted computing platforms, and can meet all requirements of intelligent networked vehicles for high real-time performance, high accuracy and high reliability of the environment perception system.

[0047] In an optional embodiment of the present application, after the feature extraction network and the classifier are trained by using the optimizer to minimize the total loss to obtain the environment perception model, the method further comprises: obtaining a current driving image; and detecting the current driving image by using the pre-trained environment perception model to obtain a target environment perception result.

[0048] The current driving image is a driving image to be subjected to environment perception. The target environment perception result is an environment perception result of the current driving image. Optionally, the target environment perception result comprises a target bounding box and a corresponding target category. The target bounding box is used to represent the position of the target in the current driving image. The target category is used to represent the category of the target. Exemplarily, the target category comprises a pedestrian, a vehicle, an obstacle, or the like.

[0049] Specifically, the current driving image to be detected is obtained. The current driving image is input into the pre-trained environment perception model to detect the current driving image and obtain the target environment perception result.

[0050] The present scheme improves the perception efficiency and accuracy of the environment perception of the driving image by obtaining the current driving image, detecting the current driving image by using the pre-trained environment perception model, and obtaining the target environment perception result.

[0051] Embodiment Two

[0052] Figure 2 A flowchart of an environment perception method provided by Embodiment Two of the present application is shown. On the basis of the above-mentioned embodiments, the present embodiment specifically implements “performing data enhancement on the labeled driving image and the unlabeled driving image to obtain a labeled enhanced image and an unlabeled enhanced image” as “performing weak enhancement on the labeled driving image and the unlabeled driving image to obtain a labeled weak enhanced image and an unlabeled weak enhanced image; and performing strong enhancement on the unlabeled driving image to obtain an unlabeled strong enhanced image”. Correspondingly, the present embodiment specifically implements “adopting the feature extraction network to perform feature extraction on the labeled enhanced image and the unlabeled enhanced image respectively to obtain labeled enhanced image features and unlabeled enhanced image features, and calculating the domain adaptation loss according to the distribution distance between the labeled enhanced image features and the unlabeled enhanced image features” as “adopting the feature extraction network to perform feature extraction on the labeled weak enhanced image, the unlabeled weak enhanced image, and the unlabeled strong enhanced image respectively to obtain labeled enhanced image features, unlabeled weak enhanced image features, and unlabeled strong enhanced image features; and calculating the domain adaptation loss according to the distribution distance between the labeled weak enhanced image features and the unlabeled weak enhanced image features and the unlabeled strong enhanced image features”, which expands the data diversity without introducing noise. It should be noted that the parts not described in detail in the present embodiment can be referred to the descriptions of other embodiments.

[0053] Referring to Figure 2 The environment perception method shown comprises:

[0054] S201, obtaining labeled driving images, labeled labels and unlabeled driving images.

[0055] S202, performing weak enhancement on the labeled driving images and the unlabeled driving images to obtain labeled weak enhanced images and unlabeled weak enhanced images.

[0056] The labeled weak enhanced images are weak enhancement results of the labeled driving images. The unlabeled weak enhanced images are weak enhancement results of the unlabeled driving images. Exemplarily, the weak enhancement manner comprises at least one of small rotation, translation and flipping. By performing weak enhancement on the labeled driving images and the unlabeled driving images, small degree image transformation of the labeled driving images and the unlabeled driving images is realized. The transformation of the driving images is realized, and the data features of the driving images themselves are not excessively changed.

[0057] Specifically, at least one of small rotation, translation and flipping is adopted to perform weak enhancement on the labeled driving images and the unlabeled driving images respectively to obtain the labeled weak enhanced images and the unlabeled weak enhanced images.

[0058] S203, performing strong enhancement on the unlabeled driving images to obtain unlabeled strong enhanced images.

[0059] The unlabeled strong enhanced images are strong enhancement results of the unlabeled driving images. Exemplarily, the strong enhancement manner comprises at least one of color dithering, contrast adjustment and random occlusion. By performing strong enhancement on the unlabeled driving images, large degree image transformation of the unlabeled driving images is realized, thereby improving the diversity of the unlabeled driving images.

[0060] Specifically, at least one of color dithering, contrast adjustment and random occlusion is adopted to perform strong enhancement on the unlabeled driving images to obtain the unlabeled strong enhanced images.

[0061] S204, performing feature extraction on the labeled weak enhanced images, the unlabeled weak enhanced images and the unlabeled strong enhanced images respectively by using a feature extraction network to obtain labeled enhanced image features, unlabeled weak enhanced image features and unlabeled strong enhanced image features.

[0062] The labeled enhanced image features are feature extraction results of the labeled weak enhanced images. The unlabeled weak enhanced image features are feature extraction results of the unlabeled weak enhanced images. The unlabeled strong enhanced image features are feature extraction results of the unlabeled strong enhanced images.

[0063] Specifically, the labeled weak enhanced image, the unlabeled weak enhanced image and the unlabeled strong enhanced image are input into the feature extraction network respectively to obtain the labeled enhanced image feature, the unlabeled weak enhanced image feature and the unlabeled strong enhanced image feature.

[0064] S205, according to the distribution distance between the labeled weak enhanced image feature and the unlabeled weak enhanced image feature and the unlabeled strong enhanced image feature, the domain adaptation loss is calculated.

[0065] Specifically, the maximum mean difference algorithm is used to calculate the distribution distance between the labeled enhanced image feature and the unlabeled enhanced image feature to obtain the domain adaptation loss.

[0066] S206, using the classifier, the labeled enhanced image feature is classified to obtain the labeled environment perception result, and according to the difference between the labeled environment perception result and the labeled label, the supervision loss is calculated.

[0067] In an optional embodiment of the present application, while using the classifier to classify the labeled enhanced image feature to obtain the labeled environment perception result, it also includes: using the classifier to classify the unlabeled weak enhanced image feature to obtain at least two first unlabeled environment perception results and corresponding first confidence; using the classifier to classify the unlabeled strong enhanced image feature to obtain a second unlabeled environment perception result; in each first unlabeled environment perception result, the first unlabeled environment perception result with the highest first confidence is selected to obtain a pseudo label; according to the second unlabeled environment perception result and the pseudo label, the anti-noise unsupervised loss is determined; and correspondingly, the supervision loss and the domain adaptation loss are integrated to obtain the total loss, including: the supervision loss, the domain adaptation loss and the anti-noise unsupervised loss are integrated to obtain the total loss.

[0068] The first environment perception result is the environment perception result obtained by classifying the unlabeled weak enhanced image feature by the classifier. The number of the first environment perception result is at least two. The first confidence is used to represent the credibility of the first environment perception result. It can be understood that the higher the first confidence, the higher the accuracy of the actual environment perception result as the first environment perception result. The second environment perception result is the environment perception result obtained by classifying the unlabeled strong enhanced image feature by the classifier. The pseudo label is the first unlabeled environment perception result with the highest confidence. That is, the classifier is based on the classification result of the weak enhanced unlabeled weak enhanced image feature. In fact, the pseudo label is the most reliable environment perception result based on the unlabeled weak enhanced image feature. The anti-noise unsupervised loss is used to represent the difference between the second environment perception result and the pseudo label.

[0069] Specifically, the unannotated weakly enhanced image features are input into the classifier to obtain at least two first unannotated environment perception results and corresponding first confidences. The unannotated strongly enhanced image features are input into the classifier to obtain a second unannotated environment perception result. The first confidences are compared, and in the first unannotated environment perception results, a first unannotated environment perception result with the highest first confidence is selected to obtain a pseudo label. Cross entropy between the second unannotated environment perception result and the pseudo label is calculated to obtain an anti-noise unsupervised loss. The supervised loss, the domain adaptation loss, and the anti-noise unsupervised loss are weighted and summed to obtain a total loss.

[0070] The scheme selects a first unannotated environment perception result with the highest first confidence in the first unannotated environment perception results to obtain a pseudo label, determines an anti-noise unsupervised loss according to the second unannotated environment perception result and the pseudo label, and weights and sums the supervised loss, the domain adaptation loss, and the anti-noise unsupervised loss to obtain a total loss, thereby further enriching the data dimension of the total loss, improving the comprehensiveness and richness of the total loss, and improving the accuracy of the environment perception model. Moreover, the difference between the second unannotated environment result and the pseudo label is considered to ensure the stability of the environment perception model learning process, train a more robust environment perception model, and significantly reduce the false positive rate of the environment perception model.

[0071] In an optional embodiment of the present application, the classifier is used to classify the unannotated strongly enhanced image features to obtain a second unannotated environment perception result, including: the classifier is used to classify the unannotated strongly enhanced image features to obtain at least two second unannotated environment perception results and corresponding second confidences; accordingly, the anti-noise unsupervised loss is determined according to the second unannotated environment perception result and the pseudo label, including: the difference between the second unannotated environment perception result with the highest second confidence and the pseudo label is calculated to obtain an unsupervised loss; the difference between the pseudo label and the second unannotated environment perception results with higher second confidences is calculated to obtain an anti-noise loss; and the unsupervised loss and the anti-noise loss are integrated to obtain the anti-noise unsupervised loss.

[0072] The number of the second environment perception results is at least two. The second confidence degree is used to represent the trust degree of the second environment perception result. It can be understood that the higher the second confidence degree is, the higher the accuracy of the actual environment perception result is. The unsupervised loss is used to represent the difference between the second unlabeled environment perception result with the highest confidence degree and the pseudo label. It can be understood that the unsupervised loss is used to represent the difference between the environment perception result with the highest confidence degree corresponding to the unlabeled strong enhanced image feature and the environment perception result with the highest confidence degree corresponding to the unlabeled weak enhanced image feature. The preset number is a number of the second unlabeled environment perception results used to calculate the robust noise loss, which is preset. The preset number is preset and adjusted by the technician. For example, the preset number is 3. The robust noise loss is used to represent the difference between the preset number of the second unlabeled environment perception results and the pseudo label. It can be understood that the robust noise loss is used to represent the difference between the multiple environment perception results with higher confidence degrees corresponding to the unlabeled strong enhanced image feature classification and the environment perception result with the highest confidence degree corresponding to the unlabeled weak enhanced image feature. It can be understood that the unsupervised loss only focuses on the difference between the second unlabeled environment perception result with the highest confidence degree and the pseudo label. The robust noise loss focuses on the difference between the preset number of the second unlabeled environment perception results with higher confidence degrees and the pseudo label. In comparison, the unsupervised loss is more strict, and the robust noise loss is more lenient.

[0073] Specifically, the unlabeled strong enhanced image feature is input into the classifier to obtain at least two second unlabeled environment perception results and corresponding second confidence degrees. The cross entropy between the second unlabeled environment perception result with the highest second confidence degree and the pseudo label is calculated to obtain the unsupervised loss. The cross entropy between the preset number of the second unlabeled environment perception results with higher second confidence degrees and the pseudo label is calculated to obtain the robust noise loss. The unsupervised loss and the robust noise loss are weighted and summed to obtain the robust noise unsupervised loss.

[0074] The present scheme calculates the unsupervised loss according to the difference between the second unlabeled environment perception result with the highest second confidence degree and the pseudo label, calculates the robust noise loss according to the difference between the preset number of the second unlabeled environment perception results with higher second confidence degrees and the pseudo label, and comprehensively obtains the robust noise unsupervised loss by combining the unsupervised loss and the robust noise loss. By combining the strict unsupervised loss and the more lenient robust noise loss, the final robust noise unsupervised loss is obtained, which has higher tolerance to noise in the training process of the environment perception model and has stronger applicability.

[0075] In an optional embodiment of the present application, after selecting the first unlabeled environment perception result with the highest first confidence from each first unlabeled environment perception result and obtaining the pseudo label, the method further comprises: obtaining a preset confidence threshold; and filtering the pseudo label according to the preset confidence threshold and the first confidence corresponding to the pseudo label.

[0076] The preset confidence threshold is a lower limit value of the confidence of the first unlabeled environment perception result preset in advance. The preset confidence threshold is used for screening the pseudo label. The preset confidence threshold is used for detecting the credibility of the pseudo label. For example, the preset confidence threshold is 0.95.

[0077] Specifically, the preset confidence threshold preset by the technical personnel is obtained. The preset confidence threshold and the first confidence corresponding to the pseudo label are compared. When the first confidence corresponding to the pseudo label is greater than or equal to the preset confidence threshold, it is determined that the credibility of the pseudo label is high, and the pseudo label can be used in the subsequent training process of the environment perception model. When the first confidence corresponding to the pseudo label is less than the preset confidence threshold, it is determined that the credibility of the pseudo label is low, and the pseudo label cannot be used in the subsequent training process of the environment perception model, and the pseudo label is filtered.

[0078] The present scheme introduces the preset confidence threshold to filter the pseudo label, which can alleviate the possible errors of the pseudo label, thereby avoiding the continuous accumulation of the error label and the divergence of the model training, and further improving the robustness of the environment perception model.

[0079] S207, the supervised loss and the domain adaptation loss are integrated to obtain a total loss, and an optimizer is adopted to train the feature extraction network and the classifier with the goal of minimizing the total loss, thereby obtaining the environment perception model.

[0080] The technical scheme of the embodiment of the present application weakly enhances the labeled driving images, and takes into account the information integrity of the labeled driving images and the data diversification based on the labeled driving images; the unlabeled driving images are respectively weakly enhanced and strongly enhanced, thereby further expanding the data diversity without introducing noise, and further significantly improving the recognition accuracy and generalization ability in complex situations such as illumination changes and local occlusions, while ensuring the efficiency and low cost of the data preparation process; the feature extraction network is adopted to extract features from the labeled weakly enhanced images, the unlabeled weakly enhanced images and the unlabeled strongly enhanced images, respectively, to obtain labeled enhanced image features, unlabeled weakly enhanced image features and unlabeled strongly enhanced image features, and the domain adaptation loss is calculated according to the distribution distance between the labeled weakly enhanced image features and the unlabeled weakly enhanced image features and the unlabeled strongly enhanced image features, thereby improving the accuracy of the domain adaptation loss and further enhancing the cross-domain adaptability and reliability of the environment perception model.

[0081] Embodiment three

[0082] The embodiment three of the present application provides a flow chart of an environment perception method. On the basis of the above-mentioned embodiment, the embodiment of the present application is a preferred embodiment of the above-mentioned environment perception method. The environment perception method comprises:

[0083] S301, data preparation and enhancement processing.

[0084] The system receives a small amount of labeled annotated driving images with labels and a large amount of unlabeled annotated driving images without labels. Weak enhancement processing is applied to all driving images, including small rotation, translation and flipping. In addition, strong enhancement processing is applied to the unlabeled images, such as color jittering, contrast adjustment and random occlusion.

[0085] As shown in Figure 3 , input the annotated data X (i.e. annotated driving images) and the unlabeled data Y and U (i.e. unlabeled driving images). Weak enhancement is applied to X (i.e. annotated driving images) and Y (i.e. unlabeled driving images) to obtain (i.e. annotated weakly enhanced images) and (i.e. unlabeled weakly enhanced images), and strong enhancement is applied to U (i.e. unlabeled driving images) to obtain (i.e. unlabeled strongly enhanced images).

[0086] For example, the five categories of targets with the highest frequency of occurrence are first taken as the recognition categories, for example, cars, trucks, pedestrians, motorcycles and bicycles are taken as the data set. A public intelligent driving data set is taken as the implementation data source. To simulate the small sample scenario, 40 samples are randomly selected from the training set for each target as the annotated data, and the labels are ensured to be accurate. The remaining data in the training set is taken as the unlabeled data, and a tested and verified validation set is used as the test set to evaluate the model performance.

[0087] This step greatly expands the diversity of training data through simple image transformation, avoiding the problem of introducing noise due to the generation of low-quality images in traditional generative methods. The direct beneficial effect is that the environment perception model can learn the invariant features of the target from more diverse training samples, significantly improving the recognition accuracy and generalization ability in complex situations such as light changes and local occlusions, while ensuring the efficiency and low cost of the data preparation process.

[0088] S302, feature extraction and domain adaptation alignment.

[0089] The data-enhanced driving image is input into a deep convolutional network (i.e., a feature extraction network) sharing weights to extract features. For example, ResNet-18 (Residual Network) is used as the feature extraction network, and the output is a 512-dimensional feature vector. To overcome the distribution difference between different driving environments, a maximum mean discrepancy algorithm is used to explicitly calculate the distribution distance between the extracted features of the labeled driving image and the unlabeled driving image as a domain adaptation loss.

[0090] As shown in Figure 3 , the feature extraction network inputs: a labeled weakly augmented image (i.e., a labeled weakly augmented image), and outputs: a labeled augmented image feature (i.e., a labeled augmented image feature); the feature extraction network inputs: an unlabeled strongly augmented image (i.e., an unlabeled strongly augmented image), and outputs: an unlabeled strongly augmented image feature (i.e., an unlabeled strongly augmented image feature); the feature extraction network inputs: an unlabeled weakly augmented image (i.e., an unlabeled weakly augmented image), and outputs: an unlabeled weakly augmented image feature (i.e., an unlabeled weakly augmented image feature). A maximum mean discrepancy algorithm (MMD algorithm) is used to calculate the distribution distance between the labeled weakly augmented image feature and the unlabeled weakly augmented image feature and the unlabeled strongly augmented image feature, and a domain adaptation loss is obtained.

[0091] This step drives the feature extraction network to ignore domain-specific interference such as weather, sensor model, and background landscape. The beneficial effect is that the environment perception model learns to focus on the essential features of the vehicle and pedestrian targets themselves rather than irrelevant environmental factors, thereby maintaining stable and high-precision perception performance when facing rain and fog weather and unfamiliar urban roads that are not seen during training, greatly enhancing the cross-domain adaptability and reliability of the system.

[0092] S303, pseudo-label generation and noise resistance optimization.

[0093] A classifier is used to obtain the model prediction result. For example, a single linear fully connected layer is used as the classifier. For the unlabeled data (i.e., the unlabeled driving image), the class with the highest confidence in the model prediction result (i.e., the first unlabeled environment perception result) of the weakly augmented version (i.e., the unlabeled weakly augmented image) of the unlabeled data is used as the pseudo-label, and a high threshold (i.e., a preset confidence threshold) is set for filtering. To alleviate the possible errors in the pseudo-label, a loss function is used to calculate the difference between the model prediction result (i.e., the second unlabeled environment perception result) of the strongly augmented version (i.e., the unlabeled strongly augmented image) of the unlabeled data and the pseudo-label, and the loss (noise resistance loss) focuses on the top k classes with the highest prediction probabilities rather than only the first one.

[0094] As Figure 3 shown, the classifier inputs: (i.e., labeled augmented image features), and outputs: (i.e., labeled perception results); the classifier inputs: (i.e., unlabeled strong augmented image features), and outputs: (i.e., second unlabeled perception results); the classifier inputs: (i.e., unlabeled weak augmented image features), and outputs: (i.e., first unlabeled perception results). A supervised loss is calculated using a cross-entropy loss between the labeled labels and the labeled perception results . In each of the first (i.e., first unlabeled perception results), the first confidence score highest (i.e., first unlabeled perception results) is selected to generate a pseudo label. A confidence threshold (i.e., a pre-set confidence threshold) is obtained. The pseudo label is filtered using the confidence threshold . An unsupervised loss is calculated based on the difference between the unfiltered pseudo label and the second confidence score highest (i.e., second unlabeled perception results). A noise-robust loss (i.e., loss) is calculated based on the difference between the unfiltered pseudo label and the second confidence score higher (i.e., second unlabeled perception results). The unsupervised loss and the noise-robust loss (i.e., loss) are summed to obtain a noise-robust unsupervised loss.

[0095] This step effectively controls the pseudo label quality risk through a lenient loss function design. The direct benefits are: the training process has higher tolerance to noise, avoids the continuous accumulation of false labels and the divergence of model training, ensures the stability of the learning process, and finally forges a more robust high-performance model, significantly reducing the false positive rate.

[0096] S304, multi-objective joint optimization.

[0097] ​The supervised loss, the domain adaptation loss and the anti-noise unsupervised loss are combined according to preset weights to form a unified total optimization target (i.e. total loss). The gradient descent algorithm is used to synchronously update all parameters of the feature extraction network and the classifier to minimize the total loss. The optimizer uses Adam (Adaptive Moment Estimation Optimizer). The initial learning rate is set to 0.003, the momentum parameter is 0.9, and the batch size is set to 32. Each batch contains 16 labeled samples and 16 unlabeled samples. The loss weights are set as follows: the domain adaptation loss weight is 1, the unsupervised loss weight is 1, and the Top-k loss weight (i.e. anti-noise loss weight) is 0.5. The Top-k parameter (i.e. preset number) is set to 3. The pseudo-label filtering confidence threshold (i.e. preset confidence threshold) is 0.95. A total of 100 training cycles are trained.

[0098] As shown in Figure 3 , the total loss is calculated using the following formula:

[0099] ;

[0100] In the formula, is the total loss; is the supervised loss; is the domain adaptation loss, is the corresponding weight; is the anti-noise unsupervised loss, is the corresponding weight.

[0101] The parameters of the feature extraction network and the classifier are updated using weight update and backpropagation. It is determined whether the maximum number of iterations is reached. If yes, the process is ended. If no, the process returns to perform the steps of performing weak enhancement on (i.e. labeled driving image) and (i.e. unlabeled driving image) to obtain (i.e. labeled weakly enhanced image) and (i.e. unlabeled weakly enhanced image), and performing strong enhancement on (i.e. unlabeled driving image) to obtain (i.e. unlabeled strongly enhanced image).

[0102] This step integrates the precise guidance of limited labeled samples, the distribution information of massive unlabeled data, and the noise suppression mechanism into an efficient training framework. The core beneficial effect produced is that the environmental perception model can fully utilize labeled and unlabeled data in cooperation, and ultimately achieve excellent comprehensive performance. The entire scheme does not require a complex generation model, is computationally efficient, and is easy to deploy to resource-constrained vehicle computing platforms, meeting all requirements of intelligent networked vehicles for environmental perception systems in terms of high real-time performance, high accuracy, and high reliability.

[0103] The training process is shown in Figure 3 At the end of each training cycle, the average precision value of the model on the test set is calculated as the main performance indicator. After training is complete, the network model parameters with the best performance on the test set are selected as the final model.

[0104] The present application solves the three core problems of low model precision, poor generalization ability, and low computational efficiency caused by the scarcity of labeled data in intelligent networked vehicle environmental perception. The present application integrates strong and weak data augmentation, domain adaptation, and loss function into a unified semi-supervised learning framework. Through strong and weak data augmentation, the data diversity is expanded without introducing noise. Through domain adaptation technology, the feature distribution in different driving environments is explicitly aligned. Through the loss function effectively suppresses the negative effects of false pseudo-labels, ultimately achieving high-precision, strong generalization, and low-power vehicle environmental perception. Specifically, first, the target recognition accuracy is significantly improved and the false positive rate is reduced under the condition of a small amount of labeled samples. Second, the domain differences between different driving environments are effectively overcome by explicitly aligning the feature distribution, greatly enhancing the model generalization ability. Finally, by abandoning complex generation models, the computational overhead is reduced, enabling the scheme to be efficiently deployed on vehicle platforms to meet real-time requirements. The present application provides a high-precision, strong generalization, and low-power environmental perception solution for intelligent networked vehicles.

[0105] Embodiment Four

[0106] Figure 4 A structural schematic diagram of an environmental perception device provided by Embodiment Four of the present application. The embodiment of the present application can be applied to the case of environmental perception of driving environments. The device executes an environmental perception method. The device is implemented in the form of hardware and / or software. The device can be configured in an electronic device that bears the environmental perception function.

[0107] Referring to Figure 4The environment perception device shown comprises a data enhancement module 401, a domain adaptation loss calculation module 402, a supervised loss calculation module 403, and an environment perception model generation module 404. The data enhancement module 401 is configured to obtain labeled driving images, labeled labels, and unlabeled driving images, and perform data enhancement on the labeled driving images and the unlabeled driving images to obtain labeled enhanced images and unlabeled enhanced images. The domain adaptation loss calculation module 402 is configured to use a feature extraction network to perform feature extraction on the labeled enhanced images and the unlabeled enhanced images respectively to obtain labeled enhanced image features and unlabeled enhanced image features, and calculate a domain adaptation loss according to the distribution distance between the labeled enhanced image features and the unlabeled enhanced image features. The supervised loss calculation module 403 is configured to use a classifier to classify the labeled enhanced image features to obtain labeled environment perception results, and calculate a supervised loss according to the difference between the labeled environment perception results and the labeled labels. The environment perception model generation module 404 is configured to synthesize the supervised loss and the domain adaptation loss to obtain a total loss, and use an optimizer to train the feature extraction network and the classifier to minimize the total loss, and obtain an environment perception model.

[0108] The technical scheme of the embodiment of the present application obtains the labeled enhanced image and the unlabeled enhanced image by performing data enhancement on the labeled driving image and the unlabeled driving image, greatly expands the diversity of the training samples through simple image transformation, and avoids the problem of introducing noise due to the generation of low-quality images by the traditional generative method; thus, the environmental perception model can learn the invariant features of the target from more diversified training samples, significantly improves the recognition accuracy and generalization ability under complex conditions such as illumination changes and local occlusions, and at the same time ensures the efficiency and low cost of the data preparation process; by using the feature extraction network, the labeled enhanced image and the unlabeled enhanced image are respectively subjected to feature extraction to obtain the labeled enhanced image features and the unlabeled enhanced image features, and according to the distribution distance between the labeled enhanced image features and the unlabeled enhanced image features, the domain adaptation loss is calculated, which overcomes the distribution difference between different driving environments and drives the feature extraction network to ignore the domain-specific interference such as weather, sensor model and background landscape, so as to focus on the essential features of the environmental perception result rather than irrelevant environmental factors, thereby maintaining stable and high-precision perception performance when facing rain and fog weather and unfamiliar urban roads that are not seen during training, greatly enhancing the cross-domain adaptability and reliability of the environmental perception model; by using the classifier, the labeled enhanced image features are classified to obtain the labeled environmental perception result, and according to the difference between the labeled environmental perception result and the labeled label, the supervision loss is calculated, which considers the overall difference between the model predicted environmental perception result and the real environmental perception result; by comprehensively calculating the supervision loss and the domain adaptation loss to obtain the total loss, and using the optimizer to minimize the total loss as the goal, the feature extraction network and the classifier are trained to obtain the environmental perception model, which can fully utilize the labeled driving image and the unlabeled driving image, without generating a complex model, and finally obtains an environmental perception model with excellent comprehensive performance, which is computationally efficient and easy to deploy to resource-constrained vehicle-mounted computing platforms, and can meet all requirements of intelligent networked vehicles for environmental perception systems in terms of high real-time performance, high accuracy and high reliability.

[0109] In an optional embodiment of the present application, the data enhancement module 401 comprises: a data weak enhancement unit configured to perform weak enhancement on the labeled driving image and the unlabeled driving image to obtain a labeled weak enhanced image and an unlabeled weak enhanced image; and a data strong enhancement unit configured to perform strong enhancement on the unlabeled driving image to obtain an unlabeled strong enhanced image; and the domain adaptation loss calculation module 402 comprises: a feature extraction unit configured to perform feature extraction on the labeled weak enhanced image, the unlabeled weak enhanced image and the unlabeled strong enhanced image respectively by using a feature extraction network to obtain a labeled enhanced image feature, an unlabeled weak enhanced image feature and an unlabeled strong enhanced image feature; and a domain adaptation loss calculation unit configured to calculate a domain adaptation loss according to a distribution distance between the labeled weak enhanced image feature and the unlabeled weak enhanced image feature and the unlabeled strong enhanced image feature.

[0110] In an optional embodiment of the present application, the supervised loss calculation module 403 further comprises: a first unlabeled environment perception result generation unit configured to, while performing classification on the labeled enhanced image feature by using the classifier to obtain a labeled environment perception result, perform classification on the unlabeled weak enhanced image feature by using the classifier to obtain at least two first unlabeled environment perception results and corresponding first confidences; a second unlabeled environment perception result generation unit configured to perform classification on the unlabeled strong enhanced image feature by using the classifier to obtain a second unlabeled environment perception result; a pseudo label generation unit configured to select, from the first unlabeled environment perception results, the first unlabeled environment perception result with the highest first confidence to obtain a pseudo label; and an anti-noise unsupervised loss calculation unit configured to determine an anti-noise unsupervised loss according to the second unlabeled environment perception result and the pseudo label; and correspondingly, the combining of the supervised loss and the domain adaptation loss to obtain a total loss comprises: combining the supervised loss, the domain adaptation loss and the anti-noise unsupervised loss to obtain the total loss.

[0111] In an optional embodiment of the present application, the second unlabeled environment perception result generation unit comprises: a second unlabeled environment perception result generation subunit configured to classify the unlabeled strong enhanced image features by using the classifier to obtain at least two second unlabeled environment perception results and corresponding second confidence; correspondingly, the anti-noise unsupervised loss calculation unit comprises: an unsupervised loss calculation subunit configured to calculate an unsupervised loss according to the difference between the second unlabeled environment perception result with the highest second confidence and the pseudo label; an anti-noise loss calculation subunit configured to calculate an anti-noise loss according to the difference between each of the second unlabeled environment perception results with the preset number and the pseudo label; and an anti-noise unsupervised loss calculation subunit configured to synthesize the unsupervised loss and the anti-noise loss to obtain an anti-noise unsupervised loss.

[0112] In an optional embodiment of the present application, the supervised loss calculation module 403 further comprises: a preset confidence threshold acquisition unit configured to acquire a preset confidence threshold after selecting the first unlabeled environment perception result with the highest first confidence from each of the first unlabeled environment perception results to obtain a pseudo label; and a pseudo label filtering unit configured to filter the pseudo label according to the preset confidence threshold and the first confidence corresponding to the pseudo label.

[0113] In an optional embodiment of the present application, the device further comprises: a current driving image acquisition module configured to acquire a current driving image after the optimizer is used to train the feature extraction network and the classifier to obtain an environment perception model with the goal of minimizing the total loss; and a current driving image perception module configured to detect the current driving image by using the pre-trained environment perception model to obtain a target environment perception result.

[0114] The environment perception device provided in the embodiments of the present application can execute the environment perception method provided in any of the embodiments of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0115] In the technical solution of the embodiments of the present application, the acquisition, storage and application of the labeled driving image, the labeled label, the unlabeled driving image, the preset confidence threshold and the current driving image, etc. all conform to the relevant legal regulations and do not violate public order and good customs.

[0116] Embodiment five

[0117] Figure 5A schematic diagram of an electronic device 500 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0118] like Figure 5 As shown, the electronic device 500 includes at least one processor 501 and a memory, such as a read-only memory (ROM) 502 and a random access memory (RAM) 503, communicatively connected to the at least one processor 501. The memory stores computer programs executable by the at least one processor. The processor 501 can perform various appropriate actions and processes based on the computer program stored in the ROM 502 or loaded into the RAM 503 from storage unit 508. The RAM 503 can also store various programs and data required for the operation of the electronic device 500. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0119] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0120] Processor 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 501 performs the various methods and processes described above, such as environmental awareness methods.

[0121] In some embodiments, the environment perception method can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 508. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 500 via, e.g., ROM 502 and / or communication unit 509. When the computer program is loaded onto RAM 503 and executed by processor 501, one or more steps of the environment perception method described above can be performed. Alternatively, in other embodiments, processor 501 can be configured to perform the environment perception method by way of other means, e.g., with the aid of firmware.

[0122] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0123] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0124] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0126] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0127] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS (Virtual Private Server) service.

[0128] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which are not limited herein.

[0129] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An environmental perception method, characterized in that, The method comprises: obtaining labeled driving images, labeled labels and unlabeled driving images, and performing data enhancement on the labeled driving images and the unlabeled driving images to obtain labeled enhanced images and unlabeled enhanced images; using a feature extraction network to perform feature extraction on the labeled enhanced images and the unlabeled enhanced images respectively to obtain labeled enhanced image features and unlabeled enhanced image features, and calculating a domain adaptation loss according to the distribution distance between the labeled enhanced image features and the unlabeled enhanced image features; using a classifier to classify the labeled enhanced image features to obtain labeled environment perception results, and calculating a supervised loss according to the difference between the labeled environment perception results and the labeled labels; comprehensively obtaining a total loss by comprehensively obtaining the supervised loss and the domain adaptation loss, and using an optimizer to minimize the total loss as the goal to train the feature extraction network and the classifier to obtain an environment perception model.

2. The method of claim 1, wherein, The data enhancement on the labeled driving images and the unlabeled driving images to obtain labeled enhanced images and unlabeled enhanced images comprises: weak enhancement is performed on the labeled driving images and the unlabeled driving images to obtain labeled weak enhanced images and unlabeled weak enhanced images; strong enhancement is performed on the unlabeled driving images to obtain unlabeled strong enhanced images; Correspondingly, the feature extraction network is used to perform feature extraction on the labeled enhanced images and the unlabeled enhanced images respectively to obtain labeled enhanced image features and unlabeled enhanced image features, and the domain adaptation loss is calculated according to the distribution distance between the labeled enhanced image features and the unlabeled enhanced image features, comprising: using a feature extraction network to perform feature extraction on the labeled weak enhanced images, the unlabeled weak enhanced images and the unlabeled strong enhanced images to obtain labeled enhanced image features, unlabeled weak enhanced image features and unlabeled strong enhanced image features; the domain adaptation loss is calculated according to the distribution distance between the labeled weak enhanced image features and the unlabeled weak enhanced image features and the unlabeled strong enhanced image features.

3. The method of claim 2, wherein, At the same time of using the classifier to classify the labeled enhanced image features to obtain the labeled environment perception results, it also comprises: using a classifier to classify the unlabeled weak enhanced image features to obtain at least two first unlabeled environment perception results and corresponding first confidence; using a classifier to classify the unlabeled strong enhanced image features to obtain a second unlabeled environment perception result; in each of the first unlabeled environment perception results, the first unlabeled environment perception result with the highest first confidence is selected to obtain a pseudo label; determining an anti-noise unsupervised loss according to the second unlabeled environment perception result and the pseudo label; Correspondingly, the comprehensive total loss comprises: comprehensively obtaining the total loss by comprehensively obtaining the supervised loss, the domain adaptation loss and the anti-noise unsupervised loss.

4. The method of claim 3, wherein, The adopting a classifier classifies the unlabeled strong enhanced image features to obtain a second unlabeled environment perception result, comprising: Adopting a classifier, classifying the unlabeled strong enhanced image features to obtain at least two second unlabeled environment perception results and corresponding second confidence degrees; Correspondingly, the determining an anti-noise unsupervised loss according to the second unlabeled environment perception result and the pseudo label comprises: According to the difference between the second unlabeled environment perception result with the highest second confidence degree and the pseudo label, an unsupervised loss is calculated; According to the difference between the preset number of second unlabeled environment perception results with higher second confidence degrees and the pseudo label, an anti-noise loss is calculated; The unsupervised loss and the anti-noise loss are integrated to obtain an anti-noise unsupervised loss.

5. The method of claim 3, wherein, After the selecting the first unlabeled environment perception result with the highest first confidence degree from the first unlabeled environment perception results to obtain a pseudo label, the method further comprises: Obtaining a preset confidence threshold; According to the preset confidence threshold and the first confidence degree corresponding to the pseudo label, the pseudo label is filtered.

6. The method of claim 1, wherein, After the adopting an optimizer to train the feature extraction network and the classifier to obtain an environment perception model, the method further comprises: Obtaining a current driving image; Using a pre-trained environment perception model to detect the current driving image to obtain a target environment perception result.

7. An environmental perception apparatus, characterized by, The device comprises: A data enhancement module is configured to obtain labeled driving images, labeled labels, and unlabeled driving images, and perform data enhancement on the labeled driving images and the unlabeled driving images to obtain labeled enhanced images and unlabeled enhanced images; A domain adaptation loss calculation module is configured to use a feature extraction network to perform feature extraction on the labeled enhanced images and the unlabeled enhanced images respectively to obtain labeled enhanced image features and unlabeled enhanced image features, and calculate a domain adaptation loss according to a distribution distance between the labeled enhanced image features and the unlabeled enhanced image features; A supervised loss calculation module is configured to use a classifier to classify the labeled enhanced image features to obtain labeled environment perception results, and calculate a supervised loss according to a difference between the labeled environment perception results and the labeled labels; An environment perception model generation module is configured to integrate the supervised loss and the domain adaptation loss to obtain a total loss, and use an optimizer to train the feature extraction network and the classifier to obtain an environment perception model, with the goal of minimizing the total loss.

8. An electronic device, comprising: The electronic device comprises: At least one processor; and A memory connected in communication with the at least one processor; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the environment perception method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the processor to implement the environment perception method according to any one of claims 1-6 when executed.

10. A computer program product, characterised in that, The computer program product comprises a computer program which, when executed by the processor, implements the environment perception method according to any one of claims 1-6.

Images