Image enhancement method and device for automatic driving, electronic equipment, and storage medium

Abstract

The application discloses an image enhancement method and device for automatic driving, an electronic device, and a storage medium. The method comprises the following steps: acquiring a first image collected based on an automatic driving system; acquiring a second image based on a preset image enhancement processing mode according to the first image; inputting the first image and the second image into a pre-trained image quality evaluation model respectively to obtain an image quality prediction result; inputting the image quality prediction result into a pre-trained reinforcement learning model to obtain a preset image enhancement processing mode adaptively matched with the second image; and processing the first image according to the preset image enhancement processing mode adaptively matched with the second image to obtain a target image. The application can effectively improve the image quality of automatic driving, and has low computational complexity and high real-time performance.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to an image enhancement method, apparatus, electronic device, and storage medium for autonomous driving. Background Technology

[0002] Autonomous driving systems require real-time perception and recognition of their surroundings, and images are one of the most important sources of information. However, image quality is often affected by lighting conditions. For example, in low light, rain, snow, or other extreme weather conditions, image contrast, clarity, and detail all decrease, thus impacting the performance and safety of autonomous driving systems. Therefore, it is necessary to improve the image quality for autonomous driving systems.

[0003] In related technologies, to improve the image quality for autonomous driving, traditional image enhancement techniques such as histogram equalization, gamma correction, and homomorphic filtering are employed, or neural network models based on deep learning, such as generative adversarial networks and residual networks, are used. However, these methods all have some limitations. Summary of the Invention

[0004] This application provides an image enhancement method, apparatus, electronic device, and storage medium for autonomous driving, to provide an environment-adaptive image enhancement solution.

[0005] The embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide an image enhancement method for autonomous driving, wherein the method includes:

[0007] Acquire the first image based on the autonomous driving system;

[0008] Based on the first image, obtain a second image obtained using a preset image enhancement processing method;

[0009] The first image and the second image are respectively input into a pre-trained image quality assessment model to obtain image quality prediction results;

[0010] The image quality prediction result is input into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image;

[0011] The first image is processed according to the preset image enhancement processing method of adaptive matching of the second image to obtain the target image.

[0012] In some embodiments, the first image includes an original color image, and processing the first image to obtain the target image includes:

[0013] The original color image is subjected to adaptive image enhancement processing to obtain the final enhanced image, which is then provided to the autonomous driving system.

[0014] In some embodiments, obtaining a second image based on a preset image enhancement processing method from the first image includes:

[0015] The first image is input into the decomposition network, which outputs the reflectance map and illumination map of the first image.

[0016] The illumination map of the first image is input into the enhancement network, and the enhanced illumination map is output.

[0017] The enhanced first image is obtained based on the reflectance map of the first image and the enhanced illumination map;

[0018] The enhanced first image is then subjected to multi-scale fusion processing to obtain the second image.

[0019] In some embodiments, obtaining a second image based on a preset image enhancement processing method, according to the first image, further includes:

[0020] The second image is obtained by applying different image enhancement methods to the first image. The different image enhancement methods include at least one of the following: histogram equalization, gamma correction, homomorphic filtering, generative adversarial network, and residual network.

[0021] In some embodiments, inputting the first image and the second image into a pre-trained image quality assessment model to obtain image quality prediction results includes:

[0022] The first image and the second image are respectively input into a pre-trained image quality assessment model to predict the image quality score and the degree of image quality improvement. The pre-trained image quality assessment model is based on a deep neural network and is trained by machine learning using multiple sets of data. Each set of data is based on a preset autonomous driving image quality assessment and enhancement dataset.

[0023] In some embodiments, inputting the image quality prediction result into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image includes:

[0024] Based on the image quality score and the degree of image quality improvement in the image quality prediction results, the reward value for each preset image enhancement method in the second image is calculated;

[0025] Based on the reward value, the optimal image enhancement processing method for adaptive matching of the second image is obtained.

[0026] In some embodiments, a multi-scale fusion-based post-processing strategy is employed before outputting the enhanced image to improve the detail and sharpness of the enhanced image.

[0027] Secondly, embodiments of this application also provide an image enhancement device for autonomous driving, wherein the device includes:

[0028] The first acquisition module is used to acquire the first image based on the autonomous driving system.

[0029] The second acquisition module is used to acquire a second image based on a preset image enhancement processing method, according to the first image;

[0030] The image quality assessment module is used to input the first image and the second image into a pre-trained image quality assessment model to obtain image quality prediction results.

[0031] The reinforcement learning module is used to input the image quality prediction result into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image;

[0032] The enhancement processing module is used to process the first image according to a preset image enhancement processing method that adaptively matches the second image, so as to obtain the target image.

[0033] Thirdly, embodiments of this application also provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the above-described method.

[0034] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the above-described method.

[0035] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: A first image is acquired based on data collected by an autonomous driving system, and then a second image is acquired based on a preset image enhancement processing method. The first image and the second image are then input into a pre-trained image quality assessment model to obtain image quality prediction results. The image quality prediction results are then input into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image. Finally, the first image is processed according to the preset image enhancement processing method for adaptive matching of the second image to obtain the target image. Because the second image has undergone image enhancement, it is more suitable for the current driving scenario compared to the first image, such as improving image quality under extreme weather conditions like low light, rain, and snow. Attached Figure Description

[0036] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0037] Figure 1 This is a schematic diagram of the image enhancement method for autonomous driving in an embodiment of this application;

[0038] Figure 2 This is a schematic diagram of the image enhancement device structure for autonomous driving in an embodiment of this application;

[0039] Figure 3 This is a schematic diagram illustrating the principle of obtaining the first-stage enhanced color image in the image enhancement method for autonomous driving in the embodiments of this application;

[0040] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] During their research, the inventors discovered that both traditional image enhancement algorithms and deep learning-based machine learning methods have various limitations, including:

[0043] a. Traditional image enhancement techniques often rely on manually set parameters and thresholds, making it difficult to adapt to different lighting conditions and scenes.

[0044] b. Deep learning models often require a large amount of labeled data for training, but there is currently a lack of specialized datasets for image quality enhancement in autonomous driving.

[0045] c. Deep learning models generally require high computational resources and time, making it difficult to meet the real-time requirements of autonomous driving systems.

[0046] To address the aforementioned shortcomings, embodiments of this application provide an image enhancement method for autonomous driving. This method evaluates the results of images enhanced using different methods through a pre-trained image quality assessment model, and then selects the target image using a pre-trained reinforcement learning model. This effectively improves the image quality for autonomous driving while exhibiting low computational complexity and high real-time performance. Low computational complexity is crucial for autonomous driving systems, as it reduces computational latency and response time. High real-time performance is critical for autonomous driving systems, ensuring their perception performance and the safety of perception results.

[0047] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0048] This application provides an image enhancement method for autonomous driving, such as... Figure 1 The diagram illustrates a flowchart of an image enhancement method for autonomous driving according to an embodiment of this application. The method includes at least the following steps S110 to S150:

[0049] Step S110: Acquire the first image based on the autonomous driving system.

[0050] The autonomous driving system can operate on the vehicle's intelligent driving controller and, once activated, can perform perception and localization functions. During this process, it acquires color images based on data collected by the autonomous driving system. Color images typically refer to RGB three-channel images.

[0051] "First image" refers to the original, unprocessed image in general. No specific limitation is made here.

[0052] Step S120: Based on the first image, obtain a second image obtained based on a preset image enhancement processing method.

[0053] The "first image" refers to the enhanced image obtained after applying a preset image enhancement method. The "second image" refers to the original image after image enhancement processing.

[0054] It is understood that the preset image enhancement processing method includes, but is not limited to, traditional processing methods or deep learning-based processing methods. In the embodiments of this application, no specific limitation is made, as long as it belongs to image enhancement processing.

[0055] Step S130: Input the first image and the second image into the pre-trained image quality assessment model to obtain the image quality prediction result.

[0056] The first image and the second image are input into a pre-trained image quality assessment model to obtain image quality prediction results. The pre-trained image quality assessment model outputs an image quality score and the degree of image quality improvement.

[0057] It's important to note that "image quality score" refers to the scoring / weighting result based on the evaluation model, and can generally be considered as the confidence level of the current image. Furthermore, "the degree of image quality improvement" compares the image obtained after image enhancement processing with the image before image enhancement processing. When an image is underexposed or overexposed, it will appear too dark or too bright, resulting in insufficient or excessive contrast and an inability to clearly display image details. Image enhancement processing can highlight feature information in the image, improving the subjective visual quality, which objectively means improving the image's contrast. Therefore, "the degree of image quality improvement" can be evaluated based on differences in evaluation indicators such as image contrast.

[0058] Based on the above "image quality score" and "degree of image quality improvement", the image quality prediction result can be obtained.

[0059] Step S140: Input the image quality prediction result into the pre-trained reinforcement learning model to obtain the preset image enhancement processing method for adaptive matching of the second image.

[0060] Based on the image quality prediction results obtained in the above steps, the results are input into a pre-trained reinforcement learning model. The pre-trained reinforcement learning model calculates the reward value for each image enhancement method based on the predicted quality score and the degree of improvement. The method with the optimal reward value is selected as the preset image enhancement processing method for the second image adaptive matching.

[0061] Step S150: Process the first image according to the preset image enhancement processing method of adaptive matching of the second image to obtain the target image.

[0062] The selected optimal preset image enhancement processing method is applied to process the color images acquired by the autonomous driving system to obtain the final enhanced image. This final enhanced image is then used by the autonomous driving system for localization and perception.

[0063] The above method employs a pre-trained image quality assessment model to evaluate the image quality prediction results after preset image enhancement processing. Furthermore, based on the scoring results of the image quality prediction results within a pre-trained reinforcement learning model, an adaptively matched preset image enhancement processing method is found. This avoids the limitations of traditional image enhancement methods or deep learning-based image enhancement methods, effectively improving the contrast, clarity, and detail of autonomous driving images under low light and extreme weather conditions, thereby enhancing the perception and recognition capabilities of autonomous driving systems. Moreover, since both the quality prediction and scoring processes are based on deep learning models, the entire process can adaptively select the most suitable image enhancement processing method for the current scene.

[0064] The above method, by employing multiple image enhancement processing techniques and using a deep learning model for quality prediction and scoring, can adapt to different lighting conditions and scenes, without the need for manual parameter setting and thresholds, and has strong generalization ability.

[0065] Using the above method, since the first image and the second image are respectively input into a pre-trained image quality assessment model to obtain image quality prediction results, and the image quality prediction results are input into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image, the deep learning model does not require a lot of complex calculations. It can maintain high computational efficiency and real-time performance while ensuring image quality, meeting the operational requirements of autonomous driving systems. In other words, it employs a deep neural network model and reinforcement learning algorithm to achieve an adaptive and intelligent image quality enhancement method for autonomous driving.

[0066] Unlike traditional image augmentation techniques, which struggle to adapt to varying lighting conditions and scenes, the method described above adaptively selects the most suitable image enhancement approach for the current scenario. Specifically, a pre-trained image quality assessment model predicts the quality of the image enhancement process, and a pre-trained reinforcement learning model outputs the optimal image quality enhancement method.

[0067] Unlike related technologies, which often require large amounts of labeled data for training deep learning models and suffer from high computational costs and time constraints, the method described above utilizes a dataset specifically designed for image quality enhancement in autonomous driving. This eliminates the need for extensive labeled data, reducing the cost of data acquisition and annotation. Furthermore, the deep learning model used in this method requires minimal computational resources and is relatively quick to execute, thus maintaining high computational efficiency and real-time performance while ensuring image quality, meeting the operational requirements of autonomous driving systems.

[0068] In one embodiment of this application, the first image includes an original color image, and the process of processing the first image to obtain a target image includes: performing adaptive image enhancement processing on the original color image to obtain a final enhanced image for provision to an autonomous driving system.

[0069] Adaptive image enhancement processing is applied to the original color image. The resulting enhanced image minimizes the impact of extreme weather conditions such as low light and rain / snow in terms of both contrast and resolution, and improves the image's contrast, clarity, and detail to provide a better solution for autonomous driving systems.

[0070] In one embodiment of this application, obtaining a second image based on a preset image enhancement processing method from the first image includes: inputting the first image into a decomposition network to output a reflectance map and an illumination map of the first image; inputting the illumination map of the first image into an enhancement network to output an enhanced illumination map; obtaining an enhanced first image based on the reflectance map of the first image and the enhanced illumination map; and performing multi-scale fusion post-processing on the enhanced first image to obtain the second image.

[0071] like Figure 3 As shown, the color images collected by the autonomous driving system are input into the decomposition network. The decomposition network consists of multiple convolutional layers and residual blocks, and its output is a reflectance map and an illumination map.

[0072] The illumination map is then fed into an enhancement network, which consists of multiple convolutional and upsampling layers, and its output is the enhanced illumination map. Finally, the reflectance map and the enhanced illumination map are multiplied to obtain the enhanced color image.

[0073] It is understood that the above decomposition network or enhancement network can choose DLA34, but this is not intended to specifically limit it.

[0074] Finally, the enhanced color image undergoes multi-scale fusion, which involves weighted averaging of the original and enhanced images at different scales to obtain the first-stage enhanced color image, which is obtained by weighted averaging of the original and enhanced images. A post-processing strategy based on multi-scale fusion is employed before outputting the enhanced image to improve its detail and sharpness.

[0075] The above method differs from the standalone image enhancement methods used in related technologies and is more suitable for extreme weather conditions such as low light, rain, and snow.

[0076] In one embodiment of this application, the step of obtaining a second image based on a preset image enhancement processing method according to the first image further includes: processing the first image using different image enhancement methods to obtain the second image, wherein the different image enhancement methods include at least one of the following: histogram equalization, gamma correction, homomorphic filtering, generative adversarial network, and residual network.

[0077] The different image enhancement methods include both traditional image enhancement processing methods and image enhancement processing methods based on deep learning models.

[0078] It should be noted that histogram equalization, gamma correction, homomorphic filtering, generative adversarial networks, residual networks, and other processing methods may include at least one individual processing method or multiple combined processing methods, and are not specifically limited in the embodiments of this application.

[0079] In one embodiment of this application, the step of inputting the first image and the second image into a pre-trained image quality assessment model to obtain image quality prediction results includes: inputting the first image and the second image into the pre-trained image quality assessment model to predict the image quality score and the degree of image quality improvement. The pre-trained image quality assessment model is based on a deep neural network trained by machine learning using multiple sets of data. Each set of data is obtained based on a preset autonomous driving image quality assessment and enhancement dataset.

[0080] In practice, a pre-trained deep neural network model is used to analyze the input image, predict the image quality score, and apply different image enhancement methods, including but not limited to histogram equalization, gamma correction, homomorphic filtering, generative adversarial networks, residual networks, and the image enhancement methods mentioned in the first stage, to determine the degree of quality improvement of the image.

[0081] The evaluation results output by the model include the quality score after image enhancement processing and the degree of improvement after image enhancement processing. Specifically, it may include, but is not limited to, image contrast, sharpness, image features, etc. In the embodiments of this application, no specific limitation is made. Those skilled in the art can select according to the actual use scenario.

[0082] In addition, during the training process of the above model, two loss functions, histogram of oriented gradients and autocorrelation loss, were introduced to ensure the quality indicators of the enhanced image, such as contrast, sharpness, detail, and orientation consistency.

[0083] The embodiments of this application do not specifically limit the specific structure of the network described above. Those skilled in the art can choose according to the actual situation, as long as it can achieve quality scoring.

[0084] In one embodiment of this application, the step of inputting the image quality prediction result into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image includes: calculating the reward value of each preset image enhancement method in the second image based on the image quality score and the degree of image quality improvement in the image quality prediction result; and obtaining the optimal image enhancement processing method for adaptive matching of the second image based on the reward value.

[0085] By utilizing a reinforcement learning algorithm, a reward value for each image enhancement method is calculated based on the predicted quality score and the degree of improvement, and the optimal image enhancement method is selected.

[0086] Furthermore, various reward functions are introduced into the aforementioned reinforcement learning algorithm to ensure the quality indicators of the enhanced image, such as contrast, sharpness, detail, and orientation consistency. The embodiments of this application do not specifically limit the reinforcement learning algorithm, as long as it can achieve the same functionality as described above.

[0087] In embodiments of this application, an image processing method for autonomous driving is also provided, wherein the above-described image enhancement method is employed.

[0088] Acquire the first image based on the autonomous driving system;

[0089] Based on the first image, obtain a second image obtained using a preset image enhancement processing method;

[0090] The first image and the second image are respectively input into a pre-trained image quality assessment model to obtain image quality prediction results;

[0091] The image quality prediction result is input into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image;

[0092] Based on the preset image enhancement processing method of adaptive matching of the second image, the first image is processed to obtain the target image, and the vehicle positioning perception is further performed on the target image.

[0093] This application embodiment also provides an image enhancement device 200 for autonomous driving, such as... Figure 2 As shown, a schematic diagram of the structure of an image enhancement device for autonomous driving in an embodiment of this application is provided. The image enhancement device 200 for autonomous driving includes at least: a first acquisition module 210, a second acquisition module 220, an image quality evaluation module 230, a reinforcement learning module 240, and an enhancement processing module 250, wherein:

[0094] In one embodiment of this application, the first acquisition module 210 is specifically used to: acquire a first image based on the autonomous driving system.

[0095] The autonomous driving system can operate on the vehicle's intelligent driving controller and, once activated, can perform perception and localization functions. During this process, it acquires color images based on data collected by the autonomous driving system. Color images typically refer to RGB three-channel images.

[0096] "First image" refers to the original, unprocessed image in general. No specific limitation is made here.

[0097] In one embodiment of this application, the second acquisition module 220 is specifically used to: acquire a second image based on a preset image enhancement processing method according to the first image.

[0098] The "first image" refers to the enhanced image obtained after applying a preset image enhancement method. The "second image" refers to the original image after image enhancement processing.

[0099] It is understood that the preset image enhancement processing method includes, but is not limited to, traditional processing methods or deep learning-based processing methods. In the embodiments of this application, no specific limitation is made, as long as it belongs to image enhancement processing.

[0100] In one embodiment of this application, the image quality assessment module 230 is specifically used to: input the first image and the second image into a pre-trained image quality assessment model to obtain image quality prediction results.

[0101] The first image and the second image are input into a pre-trained image quality assessment model to obtain image quality prediction results. The pre-trained image quality assessment model outputs an image quality score and the degree of image quality improvement.

[0102] It's important to note that "image quality score" refers to the scoring / weighting result based on the evaluation model, and can generally be considered as the confidence level of the current image. Furthermore, "the degree of image quality improvement" compares the image obtained after image enhancement processing with the image before image enhancement processing. When an image is underexposed or overexposed, it will appear too dark or too bright, resulting in insufficient or excessive contrast and an inability to clearly display image details. Image enhancement processing can highlight feature information in the image, improving the subjective visual quality, which objectively means improving the image's contrast. Therefore, "the degree of image quality improvement" can be evaluated based on differences in evaluation indicators such as image contrast.

[0103] Based on the above "image quality score" and "degree of image quality improvement", the image quality prediction result can be obtained.

[0104] In one embodiment of this application, the reinforcement learning module 240 is specifically used to: input the image quality prediction result into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image.

[0105] Based on the image quality prediction results obtained in the above steps, the results are input into a pre-trained reinforcement learning model. The pre-trained reinforcement learning model calculates the reward value for each image enhancement method based on the predicted quality score and the degree of improvement. The method with the optimal reward value is selected as the preset image enhancement processing method for the second image adaptive matching.

[0106] In one embodiment of this application, the enhancement processing module 250 is specifically used to: process the first image according to a preset image enhancement processing method of adaptive matching of the second image to obtain a target image.

[0107] The selected optimal preset image enhancement processing method is applied to process the color images acquired by the autonomous driving system to obtain the final enhanced image. This final enhanced image is then used by the autonomous driving system for localization and perception.

[0108] It is understood that the above-described image enhancement device for autonomous driving can implement all the steps of the image enhancement method for autonomous driving provided in the foregoing embodiments. The relevant explanations of the image enhancement method for autonomous driving are applicable to the image enhancement device for autonomous driving, and will not be repeated here.

[0109] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0110] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0111] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0112] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, logically forming an image enhancement device for autonomous driving. The processor executes the program stored in memory and specifically performs the following operations:

[0113] Acquire the first image based on the autonomous driving system;

[0114] Based on the first image, obtain a second image obtained using a preset image enhancement processing method;

[0115] The first image and the second image are respectively input into a pre-trained image quality assessment model to obtain image quality prediction results;

[0116] The image quality prediction result is input into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image;

[0117] The first image is processed according to the preset image enhancement processing method of adaptive matching of the second image to obtain the target image.

[0118] The above is as stated in this application. Figure 1The method for image enhancement devices used in autonomous driving disclosed in the illustrated embodiments can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0119] The electronic device can also perform Figure 1 A method for image enhancement devices used in autonomous driving is described, and the image enhancement devices used in autonomous driving are implemented in... Figure 1 The functions of the embodiments shown are not described in detail here.

[0120] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the image enhancement device for autonomous driving in the illustrated embodiment is specifically used to perform:

[0121] Acquire the first image based on the autonomous driving system;

[0122] Based on the first image, obtain a second image obtained using a preset image enhancement processing method;

[0123] The first image and the second image are respectively input into a pre-trained image quality assessment model to obtain image quality prediction results;

[0124] The image quality prediction result is input into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image;

[0125] The first image is processed according to the preset image enhancement processing method of adaptive matching of the second image to obtain the target image.

[0126] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0128] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0129] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0130] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0131] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0132] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0133] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0134] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An image enhancement method for autonomous driving, wherein, The method includes: Acquire the first image based on the autonomous driving system; Based on the first image, obtain a second image obtained using a preset image enhancement processing method; The step of obtaining a second image based on a preset image enhancement processing method from the first image includes: inputting the first image into a decomposition network to output a reflectance map and an illumination map of the first image; inputting the illumination map of the first image into an enhancement network to output an enhanced illumination map; obtaining an enhanced first image based on the reflectance map of the first image and the enhanced illumination map; and performing multi-scale fusion post-processing on the enhanced first image to obtain the second image. The first image and the second image are respectively input into a pre-trained image quality assessment model to obtain image quality prediction results; The image quality prediction result is input into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image; The step of inputting the image quality prediction result into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image includes: calculating the reward value of each preset image enhancement method in the second image based on the image quality score and the degree of image quality improvement in the image quality prediction result; and obtaining the optimal image enhancement processing method for adaptive matching of the second image based on the reward value. The first image is processed according to the preset image enhancement processing method of adaptive matching of the second image to obtain the target image.

2. The method as described in claim 1, wherein, The first image includes an original color image, and the process of processing the first image to obtain the target image includes: The original color image is subjected to adaptive image enhancement processing to obtain the final enhanced image, which is then provided to the autonomous driving system.

3. The method as described in claim 2, wherein, The step of obtaining a second image based on a preset image enhancement processing method, according to the first image, further includes: The second image is obtained by applying different image enhancement methods to the first image. The different image enhancement methods include at least one of the following: histogram equalization, gamma correction, homomorphic filtering, generative adversarial network, and residual network.

4. The method as described in claim 1, wherein, The step of inputting the first image and the second image into a pre-trained image quality assessment model to obtain image quality prediction results includes: The first image and the second image are respectively input into a pre-trained image quality assessment model to predict the image quality score and the degree of image quality improvement. The pre-trained image quality assessment model is based on a deep neural network trained by machine learning using multiple sets of data. Each set of data is based on a preset autonomous driving image quality assessment and enhancement dataset.

5. An image processing method for autonomous driving, wherein, The image enhancement method described in any one of claims 1 to 4 is employed.

6. An image enhancement device for autonomous driving, wherein, The device includes: The first acquisition module is used to acquire the first image based on the autonomous driving system. The second acquisition module is used to acquire a second image based on a preset image enhancement processing method, according to the first image; The step of obtaining a second image based on a preset image enhancement processing method from the first image includes: inputting the first image into a decomposition network to output a reflectance map and an illumination map of the first image; inputting the illumination map of the first image into an enhancement network to output an enhanced illumination map; obtaining an enhanced first image based on the reflectance map of the first image and the enhanced illumination map; and performing multi-scale fusion post-processing on the enhanced first image to obtain the second image. The image quality assessment module is used to input the first image and the second image into a pre-trained image quality assessment model to obtain image quality prediction results. The reinforcement learning module is used to input the image quality prediction result into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image; The step of inputting the image quality prediction result into a pre-trained reinforcement learning model to obtain a preset image enhancement processing method for adaptive matching of the second image includes: calculating the reward value of each preset image enhancement method in the second image based on the image quality score and the degree of image quality improvement in the image quality prediction result; and obtaining the optimal image enhancement processing method for adaptive matching of the second image based on the reward value. The enhancement processing module is used to process the first image according to a preset image enhancement processing method that adaptively matches the second image, so as to obtain the target image.

7. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 4.

8. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the method of any one of claims 1 to 4.

Images