Method for training adaptive environmental change perception model for intelligent driving and computing device

Abstract

The invention provides a method for training a self-adaptive environment change perception model for intelligent driving and computing equipment. The method for training the self-adaptive environment change perception model for intelligent driving comprises the following steps: acquiring a content image around a vehicle in a standard scene; obtaining a multi-layer feature map under each scene subjected to fine-grained classification, converting the content image into a generated image of a target scene by using a deconvolution decoder, and generating a training data set of multiple scenes; and training based on the multi-scene training data set to obtain a deep convolutional neural network model for environmental perception. By using the method of the invention, the ability of intelligent driving to adapt to environment change can be improved, and especially in special weather, the problem that the perception ability of a perception algorithm is reduced and even fails can be effectively overcome.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and more specifically to a method and computing device for training an adaptive environmental change perception model for intelligent driving. Background Technology

[0002] In recent years, with the rapid development of deep learning, especially deep convolutional neural networks (DCNNs), autonomous vehicles have made significant progress in environmental perception capabilities. Perception algorithms, such as object detection, semantic segmentation, and instance segmentation, have demonstrated recognition accuracy and robustness approaching or even surpassing human levels in standard, controllable test scenarios, such as well-lit, sunny daytime road environments.

[0003] However, when faced with environmental changes, such as nighttime scenes or severe weather, the capabilities of perception algorithms are greatly reduced or even fail. For example, in the complex and ever-changing operating environment of the real world, environmental perception algorithms still face severe challenges. When encountering drastic changes in lighting (such as dusk, dawn, entering or exiting tunnels), extreme weather conditions (such as heavy rain, heavy snow, dense fog, and sandstorms), or low visibility scenes (such as road sections without streetlights at night), the distribution characteristics of the input image (such as brightness, contrast, texture, color, and noise level) differ significantly from the standard dataset on which the model is trained.

[0004] Therefore, a simple and efficient training device and method are needed to enable perception algorithms to effectively adapt to changes in the surrounding environment. Summary of the Invention

[0005] This application aims to provide a method and system for training an adaptive environmental change perception model for intelligent driving, which can improve the ability of assisted driving or intelligent driving perception algorithms to adapt to environmental changes, especially in special weather conditions, effectively overcoming the problem of decreased perception ability or even failure of perception algorithms, and avoiding traffic accidents.

[0006] According to one aspect of this application, a method is provided for training an adaptive environmental change perception model for intelligent driving, comprising: Acquire images of the area surrounding the vehicle in a standard scenario; The content image is semantically segmented and labeled using a pre-trained convolutional encoder to obtain a multi-layer feature map; Based on various scene statistics and the multi-layer feature map, the content image is converted into a generated image of the target scene using style transfer operation and deconvolution decoder, thereby obtaining a multi-scene training dataset. Based on the training dataset of the aforementioned multiple scenarios, a deep convolutional neural network model for environmental perception is trained.

[0007] According to some embodiments, the method further includes: In each scenario of fine-grained classification, the pre-trained convolutional encoder is used to extract feature maps layer by layer from multiple images in each scenario. Calculate channel statistics for each layer of feature map; The scene statistics are obtained by averaging the channel statistics of each layer obtained from multiple images in each scene, and the scene statistics include the mean and standard deviation. The scene statistics are pre-stored by layer and channel.

[0008] According to some embodiments, the style transfer operation is implemented using the AdaIN operator: , , , Among them, F trans F is the normalized feature map output by the AdaIN operator after injecting the target scene style. c For the feature map of the content image, μ(F) c ) represents the mean of the content image feature map, σ(F) c ) represents the standard deviation of the content image feature map. The average value of the target scene style. Let w be the standard deviation of the target scene style, i represent the scene of the i-th fine-grained classification, k represent the number of scenes to be fused for the target scene, and w be the standard deviation of the target scene style. 1i w represents the scene weight of the mean of the i-th scene. 2i The scene weight represents the standard deviation of the i-th scene. Let be the mean of the i-th scene style. Let be the standard deviation of the i-th scene style.

[0009] According to some embodiments, the method further includes: predefining scene weights independently for each layer of feature maps.

[0010] According to some embodiments, the scene weights are preset for night scenes and weather scenes according to the feature map hierarchy.

[0011] According to some embodiments, training a deep convolutional neural network model for environment perception includes training a corresponding environment perception model for each sub-scene category.

[0012] According to some embodiments, the deep convolutional neural network model trained for environmental perception includes: training an environmental perception model that adapts to scene changes for all types of scenes.

[0013] According to another aspect of the present invention, a method for deploying an environment-aware model is provided: Download the environment perception model trained using the method described in any of the preceding items; When the perception deviation of the environmental perception model exceeds a preset threshold, the scene with the perception error will be uploaded to the cloud. Receive updated models from the cloud.

[0014] According to another aspect of this application, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method as described in any of the preceding claims.

[0015] According to another aspect of the present invention, a computing device is provided, comprising: a processor; and

[0016] A memory storing a computer program that, when executed by the processor, causes the processor to perform the method described in any of the preceding methods.

[0017] According to this example embodiment, by using fine-grained classification and style transfer algorithms, content images are converted into generated images corresponding to various scene categories, generating multi-scene training datasets covering different weather, time, and environmental conditions. This enables deep convolutional neural network models to learn feature representations of objects under different conditions, thus exhibiting stronger adaptability and robustness in the face of environmental changes. Using style transfer algorithms to generate multi-scene training datasets significantly reduces the cost and workload of this process compared to collecting data from multiple complex scenes, thereby improving efficiency. Compared to other methods such as GANs, the data generation scheme of this invention is more lightweight, has no training cost, and boasts extremely fast inference speed. The data generation scheme of this invention has extremely low implementation costs and can be implemented without GPUs. Furthermore, the scheme of this invention allows for fine-grained and controllable adjustment of scenes.

[0018] According to some embodiments, the device, which combines cloud analysis and an adaptive environmental change perception algorithm model, can promptly upload data to the cloud for analysis when the perception algorithm malfunctions, and then send the updated model back to the vehicle, achieving online adaptive updating of the model. This further enhances the system's response speed and adaptability to unknown environmental changes.

[0019] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0021] Figure 1A flowchart illustrating a method for training an adaptive environmental change perception model for intelligent driving, according to an example embodiment, is shown.

[0022] Figure 2 This diagram illustrates the generation of a training dataset for multiple scenarios based on an example embodiment.

[0023] Figure 3 A flowchart illustrating a method for deploying an environment-aware model according to an example embodiment is shown.

[0024] Figure 4 A schematic diagram of an environmental change perception model update system according to an example embodiment is shown.

[0025] Figure 5 A block diagram of a computing device according to an exemplary embodiment is shown. Detailed Implementation

[0026] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0027] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0028] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0029] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0030] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of this application. As used herein, the term "and / or" includes all combinations of any one and more of the associated listed items.

[0031] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0032] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily essential for implementing this application, and therefore cannot be used to limit the scope of protection of this application.

[0033] In recent years, thanks to the development of deep convolutional neural networks, autonomous vehicles have made significant progress in perceiving their surroundings. Environmental perception algorithms (such as object detection and semantic segmentation) have demonstrated high performance in standard scenarios (e.g., daytime scenarios with good lighting conditions). However, when faced with environmental changes, such as nighttime scenarios or inclement weather, the capabilities of perception algorithms decrease significantly or even fail. To address these issues, this invention focuses on the composition of the training network dataset, network training methods, and how to adjust algorithm updates when the external environment changes, enabling perception algorithms to effectively adapt to changes in the surrounding environment.

[0034] Environment perception algorithms based on deep convolutional neural networks exhibit excellent accuracy when trained and tested on the same data distribution. However, when the distribution of the input data changes over time, the accuracy of these algorithms drops sharply, or they even fail. For example, the algorithms are very effective in normal daytime weather conditions, but perform very poorly at night or in inclement weather. One of the main reasons for this problem is the lack of sufficiently diverse datasets covering a wide range of scenarios.

[0035] Typically, collecting and labeling data from various scenarios using conventional methods is extremely costly, requiring significant human and financial resources. For example, while using generative adversarial models (GANs) to synthesize labeled data with the same distribution as the target data can effectively solve the sample labeling problem, generating high-resolution images using GANs is computationally intensive, resulting in low efficiency in synthesizing data.

[0036] To this end, this application proposes a method and computing device for training an adaptive environmental change perception model for intelligent driving, which utilizes a neural network style transfer method to achieve the conversion between data with different distributions.

[0037] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application.

[0038] Figure 1 A flowchart illustrating a method for training an adaptive environmental change perception model for intelligent driving, according to an example embodiment, is shown.

[0039] See Figure 1 In S101, acquire the content image of the area around the vehicle in a standard scene.

[0040] According to the example embodiment, under standard environmental conditions, preferably a sunny daytime road scene with sufficient light and good visibility, images of the surrounding environment are captured by an onboard camera or other image acquisition device during vehicle travel.

[0041] In S103, a pre-trained convolutional encoder is used to perform semantic segmentation and annotation on the content image to obtain a multi-layer feature map.

[0042] According to some embodiments, pre-trained convolutional encoders such as VGG19 or ResNet34 can be used to perform pixel-level semantic annotation (e.g., semantic segmentation labels) on the acquired images, forming a high-quality labeled dataset. This labeled data contains precise masked labels for key objects such as various traffic participants (e.g., vehicles, pedestrians, cyclists) and road structures (lane lines, curbs, traffic signs). The data at this stage serves as the "raw image data" and semantic label source for subsequent style transfer.

[0043] In S105, based on various scene statistics and the multi-layer feature map, the content image is converted into a generated image of the target scene using style transfer operation and deconvolution decoder, thereby obtaining a multi-scene training dataset.

[0044] According to the example embodiment, environmental scenes are classified in a fine-grained manner. The classification dimensions include two orthogonal factors: illumination conditions and weather type. The fine-grained classification is based on the combination of illumination conditions and weather type, including at least cross-combinations of daytime / nighttime with sunny, rainy, snowy, foggy, hail, and dust storms. For example, 12 target scene categories (such as "rainy daytime," "foggy nighttime," etc.) are formed by the combination of illumination conditions and weather type. Furthermore, each weather type can be further subdivided according to intensity; for example, rainy days can be further divided into heavy rain, moderate rain, and light rain, resulting in more subdivided scenes and target scenes. For each scene, multiple representative images are acquired as "style images," whose visual features reflect the typical appearance of the scene (such as raindrop reflection, fog scattering, low-light noise, etc.). Subsequently, using a style transfer operation and a deconvolutional decoder, the content image feature maps are fused with the style images involved in each target scene. Content information (such as object outlines and spatial layout) is retained by the content image feature maps, while style information (such as color distribution, texture, and contrast) is provided by the target scene style.

[0045] See Figure 2 According to some embodiments, the content images are labeled data of daytime scenes under good lighting conditions. Based on this, combined with different scene images, the style transfer algorithm is used to transform this labeled data into target scene generated images of various scenes (or environments, weather), generating a multi-scene training dataset. In the various scene datasets, only the data presentation changes, and the labels in the segmented image data remain unchanged.

[0046] The generated new images visually represent the target scene (e.g., "rainy night"), but their semantic structure is consistent with the original images. Therefore, the original labeled segmented image data can be directly reused as its segmentation labels. Finally, a large-scale synthetic training dataset with precise annotations covering all fine-grained sub-scenes is constructed.

[0047] According to some embodiments, various scene statistics can be pre-acquired and stored. For example, in each scene of fine-grained classification, the pre-trained convolutional encoder is used to extract feature maps layer by layer from multiple images of each scene, and then channel statistics are calculated for each layer of feature maps. After averaging the channel statistics obtained from multiple images of each scene, the scene statistics are obtained, which include the mean and standard deviation, so that the scene statistics can be pre-stored layer by layer and channel.

[0048] According to some embodiments, a pre-trained convolutional neural network (ResNet34) is used as a feature extractor. The source image and style image are input into the pre-trained ResNet34, and features are extracted at different convolutional layers (such as conv1, conv2_3, conv3_4, etc.). The convolutional features of later layers (such as conv4_6) can preserve the semantic structure of the image (such as object outlines and scene layout); the convolutional features of the earlier and middle layers (such as conv1, conv2_3) can preserve style information such as texture, color, and brushstrokes.

[0049] According to the embodiment, AdaIN style transfer is performed using channel-independent mean and variance alignment. The core of this process is to directly align the mean and variance of the style image and the target scene image features in each channel, without needing to calculate the Gram matrix, thus achieving higher efficiency.

[0050] Style transfer operations are implemented using the AdaIN operator: , , , Among them, F trans F is the normalized feature map output by the AdaIN operator after injecting the target scene style. c For the feature map of the content image, μ(F) c ) represents the mean of the content image feature map, σ(F) c ) represents the standard deviation of the content image feature map. The average value of the target scene style. Let w be the standard deviation of the target scene style, i represent the scene of the i-th fine-grained classification, k represent the number of scenes to be fused for the target scene, and w be the standard deviation of the target scene style. 1i w represents the scene weight of the mean of the i-th scene. 2i The scene weight represents the standard deviation of the i-th scene. Let be the mean of the i-th scene style. Let be the standard deviation of the i-th scene style.

[0051] According to some embodiments, scene weights can be predefined independently for each layer of feature maps. These scene weights can be preset for nighttime scenes and weather scenes according to the feature map layer. For example, for nighttime and heavy rain, in shallow layers, the weight of nighttime is smaller, for example, 0.3, while the weight of heavy rain is prioritized, for example, 0.7; in mid-layers, the weight of nighttime increases, for example, 0.4-0.6, while the weight of heavy rain is basically balanced with the weight of nighttime; in deep layers, the weight of nighttime dominates, around 0.7, while the weight of heavy rain is around 0.3. Furthermore, when setting the scene weights for each layer of feature maps, the mean and standard deviation can be assigned different weights. For example, for heavy rain, a higher weight can be assigned to the standard deviation, while the weight of the mean can be lower than the weight of the standard deviation. For example, when the mean weight is set higher for nighttime, the weight of the standard deviation can be relatively lower to prevent loss of detail.

[0052] In addition, for daytime scenes, since the lighting is close to that of standard and weather scenes, only the weather scene can be used for fusion and conversion.

[0053] According to the present invention, only 10-20 images of each scene are needed to achieve diverse coverage of different angles and lighting conditions, without the need for massive amounts of data, which can greatly reduce the cost of implementation.

[0054] According to some embodiments, a pre-trained U-Net can be used as a deconvolution decoder, but the present invention is not limited thereto. The stylized feature maps obtained by the AdaIN operator are fed into the deconvolution decoder to output the final generated image. The deconvolution decoder may include 4-5 layers of transposed convolution + convolution + ReLU, progressively upsampling, convolving, and activating from deep, small feature maps to gradually restore resolution and details, and obtain the original image size.

[0055] In S107, a deep convolutional neural network model for environmental perception is trained based on the training dataset of the multiple scenarios.

[0056] According to the example implementation, a corresponding environment perception model can be trained for each sub-scene category. Alternatively, an environment perception model adapting to scene changes can be trained for all categories of scenes. For example, after generating the required multi-scene training dataset, a corresponding perception model can be trained based on the dataset for each category of scene, such as YOLOv8, YOLOv10, YOLO-NAS, etc., which will not be elaborated here. The trained model can perceive specific sub-scenarios and can switch according to the current environment of the vehicle, that is, use the corresponding environment perception model to deal with the current scene of the vehicle. Another commonly used approach, which is easy to understand, is to use a multi-scene training set to train a unified perception model for all categories of scenes, which will also not be elaborated here.

[0057] According to some embodiments, an apparatus for an adaptive environment change perception algorithm is also provided. This apparatus can implement the method described above for training an adaptive environment change perception model for intelligent driving, and can be used in an in-vehicle system or a mobile terminal device connected thereto. The apparatus may include an image acquisition module, a feature extraction module, a data conversion module, a storage unit, a communication module, and a central processing module.

[0058] According to some embodiments, the image acquisition module is used to acquire content images of the vehicle's surrounding environment, the feature extraction module is used to label the content images acquired by the image acquisition module, and the data conversion module converts the labeled content images overlaid with the environmental scene images into target image data. The storage unit is used to store perception models pre-trained for different environmental scenes. The communication module is used to upload the current scene data to the cloud training device when a significant decrease in perception performance is detected, and to receive the updated perception algorithm model sent by the cloud training device. The central processing module is used to call the corresponding perception model from the storage unit based on the current environment recognition result, or to trigger online model updates through the communication module.

[0059] Figure 3 A flowchart illustrating a method for deploying an environment-aware model according to an example embodiment is shown.

[0060] According to the example embodiment, see Figure 3 The method for deploying the environment-aware model is as follows.

[0061] In S201, download the trained environment perception model.

[0062] According to some embodiments, a pre-trained deep convolutional neural network environment perception model is installed and initialized on a vehicle or mobile device. These models are trained on datasets generated using the aforementioned methods and are capable of effectively handling environment perception tasks under different lighting conditions and weather conditions. The environment perception model can automatically download the latest version from the cloud via an application on the in-vehicle system or mobile device and deploy it locally. This process involves a communication module to ensure fast and stable download speeds and timely model updates.

[0063] According to some embodiments, see Figure 4 The hardware structure of the adaptive environment change perception algorithm model update system includes a central processing module, a communication module, a storage unit, a power supply module, a human-computer interaction module, and a cloud training device.

[0064] The central processing module, as the core control unit of the system, is responsible for running environmental perception algorithms, performing anomaly detection and analysis, managing model loading and switching, and coordinating communication between various modules. The communication module enables high-speed, low-latency data transmission between the vehicle and cloud-based training equipment, supporting the uploading of perceived anomaly data and the downloading of updated models.

[0065] The storage unit is used to store multiple pre-trained scene-specific perception models (such as "nighttime and rainy days" and "daytime and foggy days"). The power module provides stable power support for the entire system, ensuring continuous operation during vehicle operation.

[0066] The human-machine interaction module displays the current perception status, model update progress, and abnormal alarm information to the driver or system administrator, and supports manual triggering of updates or confirmation operations. The cloud training device is deployed on a remote server cluster and has powerful computing capabilities. It is used to receive abnormal scene data uploaded from the vehicle, perform annotation, verification, model retraining and optimization, and then distribute the updated model to the vehicle.

[0067] When the vehicle detects an anomaly, the central processing module invokes the deployed environmental perception model to infer the real-time acquired images. If a significant deviation in the perception results is detected (such as a target false negative rate exceeding a threshold, a sharp drop in confidence, or inconsistencies in timing), it is determined to be a "perception anomaly." At this point, the central processing module automatically initiates the anomaly reporting process. Through the communication module, a data packet containing the anomaly frame image, sensor information (such as light intensity and weather type), model output results, and context timestamps is encrypted and uploaded to the cloud training device. The upload process can employ techniques such as fragmented transmission and compression encoding to improve efficiency.

[0068] After receiving abnormal data, the cloud-based training device will combine automated annotation tools with manual verification to generate high-quality labels, add the new data to the training set corresponding to the fine-grained scenario (such as "night or rainy day"), fine-tune the original model using incremental learning or transfer learning strategies, and conduct simulation tests and performance evaluations on the updated model.

[0069] The verified updated model is packaged into a deployable format and transmitted back to the vehicle via the communication module. Upon receiving it, the central processing module stores it in the storage unit, recording the version number and activation conditions. Based on the current environment identification results (e.g., "nighttime + rain"), the central processing module selects the corresponding updated model from the storage unit and loads and runs it. Simultaneously, the user can be notified via the human-machine interface module: "Model updated, new version being activated."

[0070] The human-computer interaction module continuously displays the model's running status, update logs, and performance metrics. If anomalies persist after an update, the above process can be triggered repeatedly, forming a closed-loop "perception-feedback-learning-optimization" mechanism.

[0071] In S203, when the perception deviation of the environmental perception model is greater than a preset threshold, the scene with the perception error is uploaded to the cloud.

[0072] According to the example embodiment, to improve the accuracy and adaptability of the perception algorithm, a monitoring mechanism is set up in practical applications to evaluate the model's performance. Once a significant difference is found between the output of the current environment perception model and the actual situation (e.g., object detection failure, classification error), and this difference exceeds a preset accuracy threshold, an exception handling process is triggered. At this time, the system automatically collects current environmental information (including but not limited to image data, sensor readings, etc.) and the model's prediction results, and efficiently uploads this data to the cloud via network signals. The data received by the cloud includes not only the original image but also corresponding contextual information (such as timestamps, GPS coordinates, etc.) to facilitate subsequent analysis.

[0073] According to some embodiments, when the triggering rules set on the vehicle are frequently triggered (e.g., perception errors exceed preset values), the perception algorithm malfunctions and transmits the perceived data back to the cloud. There, it is analyzed and verified using automated tools or manually. Abnormal or failure scenario data is collected, labeled, and used to train, validate, and iterate the model. A model that meets the requirements is then transmitted back to the vehicle, improving the perception algorithm's online adaptability.

[0074] In S205, the updated model is received from the cloud.

[0075] In some implementations, the received data in the cloud is used for further analysis and improvement of existing perception models. Detailed analysis of the uploaded datasets is performed using automated tools or manual review to identify the specific causes of perception biases. Existing models are then retrained or fine-tuned based on new datasets to enhance their performance in specific scenarios.

[0076] Furthermore, considering the limited resources of in-vehicle hardware, this invention suggests combining the additional computing power of mobile devices (such as smartphones) to optimize model running efficiency. For example, a dedicated application could be developed to enable the phone to participate in perception task processing as an auxiliary computing unit, or directly act as a temporary storage medium to cache data to be uploaded.

[0077] This embodiment deploys the environmental perception model through the above steps, which not only ensures the continuous learning capability of the environmental perception system and its rapid response to external changes, but also maximizes the use of available resources and improves the robustness of the overall system and user experience.

[0078] This method fully leverages the advantages of synergy between edge computing (vehicle-side) and cloud computing (cloud-side). The vehicle-side operates with lightweight, real-time capabilities, while the cloud provides powerful computing support for model iteration. Human-machine interaction enhances safety and controllability. This system not only enables online adaptive updates to the perception model but also constructs a continuously evolving intelligent driving perception system, effectively improving the robustness and safety of autonomous driving in complex and ever-changing environments.

[0079] Figure 5 A block diagram of a computing device according to an exemplary embodiment of the present invention is shown.

[0080] like Figure 5 As shown, the computing device 30 includes a processor 12 and a memory 14. The computing device 30 may also include a bus 22, a network interface card 16, and an I / O interface 18. The processor 12, memory 14, network interface card 16, and I / O interface 18 can communicate with each other via the bus 22.

[0081] The processor 12 may include one or more general-purpose CPUs (Central Processing Units), microprocessors, or application-specific integrated circuits, for executing relevant program instructions. According to some embodiments, the computing device 30 may also include a high-performance display adapter (GPU) 20 for accelerating the processor 12.

[0082] Memory 14 may include a machine system readable medium in the form of volatile memory, such as random access memory (RAM), read-only memory (ROM), and / or cache memory. Memory 14 is used to store one or more programs containing instructions, as well as data. Processor 12 may read the instructions stored in memory 14 to perform the methods described above according to embodiments of the present invention.

[0083] The computing device 30 can also communicate with one or more networks via the network interface card 16. The network interface card 16 can be a DPU smart network card.

[0084] Bus 22 can include address bus, data bus, control bus, etc. Bus 22 provides a path for exchanging information between components.

[0085] It should be noted that, in specific implementations, the computing device 30 may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the device described above may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0086] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), network storage devices, cloud storage devices, or any type of medium or device suitable for storing instructions and / or data.

[0087] This invention also provides a computer program product comprising a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments.

[0088] Those skilled in the art will clearly understand that the technical solutions of this application can be implemented using software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware capable of independently performing or cooperating with other components to perform a specific function, where the hardware may be, for example, a field-programmable gate array (FPGA), integrated circuit, etc.

[0089] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0090] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0092] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0093] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0094] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application.

[0095] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0096] The exemplary embodiments of this application have been specifically shown and described above. It should be understood that this application is not limited to the detailed structures, arrangements, or implementation methods described herein; rather, this application is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended provisions.

Claims

1. A method for training an adaptive environment change perception model for intelligent driving, the method comprising: include: Acquire images of the area surrounding the vehicle in a standard scenario; The content image is semantically segmented and labeled using a pre-trained convolutional encoder to obtain a multi-layer feature map; Based on various scene statistics and the multi-layer feature map, the content image is converted into a generated image of the target scene using style transfer operation and deconvolution decoder, thereby obtaining a multi-scene training dataset. Based on the training dataset of the aforementioned multiple scenarios, a deep convolutional neural network model for environmental perception is trained.

2. The method according to claim 1, characterized in that, Also includes: In each scenario of fine-grained classification, the pre-trained convolutional encoder is used to extract feature maps layer by layer from multiple images in each scenario. Calculate channel statistics for each layer of feature map; The scene statistics are obtained by averaging the channel statistics of each layer obtained from multiple images in each scene, and the scene statistics include the mean and standard deviation. The scene statistics are pre-stored by layer and channel.

3. The method according to claim 1, characterized in that, The style transfer operation is implemented using the AdaIN operator: ， ， ， Among them, F trans F is the normalized feature map output by the AdaIN operator after injecting the target scene style. c For the feature map of the content image, μ(F) c ) represents the mean of the content image feature map, σ(F) c ) represents the standard deviation of the content image feature map. The average value of the target scene style. Let w be the standard deviation of the target scene style, i represent the scene of the i-th fine-grained classification, k represent the number of scenes to be fused for the target scene, and w be the standard deviation of the target scene style. 1i w represents the scene weight of the mean of the i-th scene. 2i The scene weight represents the standard deviation of the i-th scene. Let be the mean of the i-th scene style. Let be the standard deviation of the i-th scene style.

4. The method according to claim 3, characterized in that, The method further includes: predefining scene weights that are independent of each layer of feature maps.

5. The method according to claim 4, characterized in that, The scene weights are preset according to the feature map hierarchy for nighttime scenes and weather scenes.

6. The method according to claim 1, characterized in that, The deep convolutional neural network models trained for environment perception include: For each sub-scene category, a corresponding environment perception model is trained.

7. The method according to claim 1, characterized in that, The deep convolutional neural network models trained for environment perception include: For all types of scenarios, train an environment perception model that adapts to changes in the scenario.

8. A method for deploying an environment-aware model: Download the environment perception model trained using the method according to any one of claims 1 to 7; When the perception deviation of the environmental perception model exceeds a preset threshold, the scene with the perception error will be uploaded to the cloud. Receive updated models from the cloud.

9. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7.

10. A computing device, characterized in that, include: processor; as well as A memory storing a computer program that, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-7.

Images