Electronic rearview mirror severe weather image visual enhancement method and device, equipment and storage medium

Abstract

The invention discloses an electronic rearview mirror severe weather image visual enhancement method and device, equipment and a storage medium, and relates to the technical field of electronic rearview mirrors, and the method comprises the steps: obtaining an original input image; processing an original input image through a hyper-parameter estimation network in the end-to-end adaptive image enhancement model to obtain hyper-parameters of the image enhancement module; after the image enhancement module is adjusted based on the hyper-parameters, the original input image is subjected to image enhancement processing through the adjusted image enhancement module to obtain the target image, and enhancement processing of the electronic rearview mirror image under various severe weather conditions is achieved. The visual effect in a full severe weather scene is improved, and better observation experience is brought to a driver.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic rearview mirrors, and in particular to a method, device, equipment and storage medium for enhancing the visual quality of electronic rearview mirror images in bad weather. Background Art

[0002] With the continuous development of emerging technologies in the automotive industry, electronic rearview mirrors have become a new direction to replace traditional rearview mirrors in cars. With the electrification of rearview mirrors, the visual display module is handed over to the electronic system, making it possible to apply various existing advanced technologies in its image processing direction.

[0003] Although electronic rearview mirrors are different from traditional rearview mirrors in structure, appearance, and usage, their imaging depends on the imaging capabilities of the camera. Their visual effects are not necessarily better than those of traditional rearview mirrors in low light, strong light, at night, and in adverse weather conditions such as fog, dust, rain, and snow, and they cannot bring drivers a stronger desire to use and a better user experience. At present, some solutions are specialized in image processing on rainy or foggy days, or simply perform visual image enhancement based on color space. There are also methods for image optimization and reconstruction through image generation. These methods have certain adaptability in their corresponding scenarios, but also have strong scene dependence. In addition, many solutions have problems such as complex processes and excessive calculations.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present invention is to provide a method, device, equipment and storage medium for visual enhancement of electronic rearview mirror images in severe weather, aiming to solve the technical problem of low adaptability of enhancement processing of electronic rearview mirror images.

[0006] To achieve the above object, the present invention provides a method for enhancing the visual quality of an electronic rearview mirror image in bad weather, the method comprising the following steps:

[0007] Get the original input image;

[0008] Processing the original input image through a hyperparameter estimation network in an end-to-end adaptive image enhancement model to obtain hyperparameters of an image enhancement module;

[0009] After the image enhancement module is adjusted based on the hyperparameter, the original input image is subjected to image enhancement processing by the adjusted image enhancement module to obtain a target image.

[0010] In one embodiment, before the step of processing the original input image through the hyperparameter estimation network in the end-to-end adaptive image enhancement model to obtain the hyperparameters of the image enhancement module, the step further includes:

[0011] Create an electronic rearview mirror scene dataset;

[0012] Performing migration and fine-tuning training on the neural network model based on the electronic rearview mirror scene data set to obtain a target model;

[0013] The target model is subjected to module stripping processing to obtain an end-adaptive image enhancement model.

[0014] In one embodiment, the step of performing module stripping processing on the target model to obtain an end-adaptive image enhancement model includes:

[0015] The hyperparameter estimation network and the image enhancement module of the target model are stripped to obtain an end-to-end adaptive image enhancement model based on the hyperparameter estimation network and the image enhancement module.

[0016] In one embodiment, the step of performing image enhancement processing on the original input image by the adjusted image enhancement module to obtain a target image includes:

[0017] The original input image is subjected to denoising, white balance, gamma correction, hue adjustment, contrast adjustment, and sharpening processing in sequence by the adjusted image enhancement module to obtain an enhanced image;

[0018] The enhanced image and the original input image are fused to obtain a target image.

[0019] In one embodiment, the step of fusing the enhanced image with the original input image to obtain a target image includes:

[0020] Reducing the image pixel values ​​of the enhanced image to obtain a restored image;

[0021] Resampling the original input image to obtain a resampled original input image;

[0022] The restored image and the resampled original input image are subjected to image fusion processing to obtain a target image.

[0023] In one embodiment, after the step of obtaining the original input image, the method further includes:

[0024] Preprocessing the original input image to obtain a normalized input image;

[0025] The step of processing the original input image through a hyperparameter estimation network in the end-to-end adaptive image enhancement model to obtain hyperparameters of the image enhancement module also includes:

[0026] The normalized input image is processed by a hyperparameter estimation network in an end-to-end adaptive image enhancement model to obtain hyperparameters of an image enhancement module.

[0027] In one embodiment, before the step of performing migration fine-tuning training on the neural network model based on the electronic rearview mirror scene data set to obtain the target model, the step further includes:

[0028] Obtain historical scene datasets;

[0029] The initial neural network model is trained using the historical scene data set to obtain a neural network model.

[0030] In addition, to achieve the above-mentioned purpose, the present invention also proposes a device for enhancing the visual quality of an electronic rearview mirror in bad weather, the device comprising:

[0031] An image acquisition module, used to acquire an original input image;

[0032] A data processing module, used to process the original input image through a hyperparameter estimation network to obtain hyperparameters of an image enhancement module;

[0033] The image enhancement module is used to adjust the image enhancement module based on the hyperparameters, and then perform image enhancement processing on the original input image through the adjusted image enhancement module to obtain a target image.

[0034] In addition, to achieve the above-mentioned objectives, the present invention also proposes an electronic rearview mirror severe weather image visual enhancement device, the device comprising: a memory, a processor, and an electronic rearview mirror severe weather image visual enhancement program stored in the memory and executable on the processor, the electronic rearview mirror severe weather image visual enhancement program being configured to implement the steps of the electronic rearview mirror severe weather image visual enhancement method as described above.

[0035] In addition, to achieve the above-mentioned objectives, the present invention also proposes a storage medium, on which is stored a program for visually enhancing electronic rearview mirror images in severe weather conditions. When the program for visually enhancing electronic rearview mirror images in severe weather conditions is executed by a processor, the steps of the method for visually enhancing electronic rearview mirror images in severe weather conditions as described above are implemented.

[0036] One or more technical solutions proposed in this application have at least the following technical effects:

[0037] The original input image is obtained; the original input image is processed by the hyperparameter estimation network in the end-to-end adaptive image enhancement model to obtain the hyperparameters of the image enhancement module; after adjusting the image enhancement module based on the hyperparameters, the original input image is enhanced by the adjusted image enhancement module to obtain the target image, thereby achieving enhanced processing of the electronic rearview mirror image under various severe weather conditions, improving the visual effects in all severe weather scenarios, and bringing better observation experience to the driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0040] Figure 1 A flowchart diagram of Embodiment 1 of the method for visually enhancing an image in bad weather in an electronic rearview mirror of the present application;

[0041] Figure 2 The electronic rearview mirror image enhancement flow chart provided for the electronic rearview mirror bad weather image visual enhancement method embodiment 1 of the present application;

[0042] Figure 3 A flow chart of Embodiment 2 of the method for visually enhancing an image in bad weather in an electronic rearview mirror of the present application;

[0043] Figure 4 A module migration fine-tuning training flow chart provided for Embodiment 2 of the method for visual enhancement of electronic rearview mirror images in severe weather conditions in this application;

[0044] Figure 5 A flowchart diagram of Embodiment 3 of the method for visually enhancing an image in bad weather in an electronic rearview mirror of the present application;

[0045] Figure 6 This is a schematic diagram of the module structure of the device for enhancing the visual image of an electronic rearview mirror in severe weather according to an embodiment of the present application;

[0046] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the method for visual enhancement of electronic rearview mirror images in severe weather in an embodiment of the present application.

[0047] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0049] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0050] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an electronic rearview mirror bad weather image visual enhancement device, etc. The following takes the electronic rearview mirror bad weather image visual enhancement device as an example to illustrate this embodiment and the following embodiments.

[0051] Based on this, the embodiment of the present application provides a method for enhancing the visual quality of an electronic rearview mirror image in bad weather, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the method for visually enhancing an electronic rearview mirror image in severe weather conditions of the present application.

[0052] In this embodiment, the electronic rearview mirror bad weather image visual enhancement method includes steps S10 to S30:

[0053] Step S10, obtaining an original input image;

[0054] It should be noted that the original input image is the image that needs to be enhanced visually.

[0055] The original input image can be acquired by an image acquisition device around the vehicle.

[0056] Furthermore, after step S10, the method further includes:

[0057] Preprocess the original input image to obtain a normalized input image;

[0058] It should be noted that the original input image needs to be preprocessed in order to provide normalized input data to the neural network model.

[0059] It should be understood that normalization is to scale the pixel values ​​of the image to a specific range, which can be [0, 1] or [-1, 1], which helps the model converge faster.

[0060] Step S20, processing the original input image through a hyperparameter estimation network in the end-to-end adaptive image enhancement model to obtain hyperparameters of the image enhancement module;

[0061] It should be noted that the end-to-end adaptive image enhancement model is an optimized and improved CNNPP module (i.e., a parameter estimator based on a convolutional neural network) combined with the DIP module (i.e., a differentiable image processing module) and transplanted into the image enhancement application of the electronic rearview mirror to cope with the problem of poor visual effects of traditional rearview mirrors in special severe weather conditions, forming an end-to-end adaptive image enhancement model, namely, the IAEM module.

[0062] It should be understood that the model uses the GhostConv module to replace part of the original Conv module. The GhostConv module can greatly reduce the computational complexity of the convolution operation by adding a convolution to halve the number of feature channels. At the same time, the addition of convolutional connection branches further enriches the feature source of the model, so the feature extraction capability is stronger than the original CNNPP module. In addition, when layer5 replaces GhostConv, an SE channel attention mechanism module is used, so that the hyperparameter estimation network can further focus on the features of certain specific channels when estimating hyperparameters, thereby guiding the model to obtain better parameter estimation effects.

[0063] Hyperparameters are data used to adjust the image enhancement function in the image enhancement module.

[0064] In some embodiments, the step of processing the original input image through the hyperparameter estimation network in the end-to-end adaptive image enhancement model to obtain the hyperparameters of the image enhancement module further includes:

[0065] The normalized input image is processed by the hyperparameter estimation network in the end-to-end adaptive image enhancement model to obtain the hyperparameters of the image enhancement module.

[0066] Step S30, after adjusting the image enhancement module based on the hyperparameters, the original input image is enhanced by the adjusted image enhancement module to obtain a target image.

[0067] In the specific implementation, the hyperparameters are directly passed to the six image enhancement modules of dehazing / denoising, white balance, gamma correction, hue adjustment, contrast adjustment, and sharpening within the image enhancement module to control the intensity of image processing by these modules. Then, the original image is enhanced by each module and then combined with the original image resampling fusion to restore it to the enhanced RGB visual image, i.e., the target image.

[0068] like Figure 2 As shown, Figure 2The electronic rearview mirror image enhancement flow chart first needs to preprocess the original input image in order to provide the normalized input data to the neural network model. After the inference of the CNNPP-GSE parameter estimation network, 15 hyperparameters suitable for the current image enhancement module can be obtained. These hyperparameters are directly passed to the six image enhancement modules of defogging / denoising, white balance, gamma correction, hue adjustment, contrast adjustment, and sharpening in the IAEM module to control the intensity of these modules on image processing. The defogging / denoising module is a module that uses the atmospheric scattering model to restore the image with limited vision caused by the air light scattering phenomenon in the image, which can remove the influence of fog, dust, rain and snow to a certain extent; the white balance, gamma correction, contrast and hue adjustment modules are traditional image processing filter modules with pixel-level filtering capabilities. Through parameter adjustment, the visual effects of various aspects such as image color brightness can be enhanced; the sharpening module sharpens and enhances the image by introducing image sharpening calculation, which can effectively highlight the object details in the image and strengthen the sense of object boundaries. After the IAEM module completes the processing, the output result is rescaled to the RGB image data range of 0-255 through image restoration. In order to make the final image display effect more in line with the sensory requirements of the actual scene, the original input image is resampled once and then pixel-wise fused with the restored enhanced image to obtain the final output image. At this point, the enhancement operation of the electronic rearview mirror image is completed.

[0069] This embodiment provides a method for visual enhancement of electronic rearview mirror images in severe weather conditions. The method estimates the hyperparameters of each image enhancement module through a joint training method using an optimized deep convolutional network, and then uses these parameters and corresponding modules to perform input image enhancement processing. The method mainly includes three parts: hyperparameter dynamic estimation model optimization, model migration fine-tuning training, and electronic rearview mirror image adaptive enhancement application.

[0070] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , step S20 includes steps S201 to S203:

[0071] Step S201, creating an electronic rearview mirror scene data set;

[0072] It should be noted that by collecting various driving scene images through electronic rearview mirrors, we can create an electronic rearview mirror scene dataset corresponding to downstream tasks. The production of this dataset is no different from the public dataset, and does not require a very large amount of data to perform subsequent model migration and fine-tuning training in the electronic rearview mirror scenario, and finally obtain the model we need.

[0073] Step S202, performing migration fine-tuning training on the neural network model based on the electronic rearview mirror scene data set to obtain a target model;

[0074] like Figure 4 As shown, Figure 4 The training flow chart for module migration and fine-tuning is firstly to train the improved IA-YOLO model for joint detection tasks through a large number of public intelligent driving and parking scene data sets, and then to fine-tune the model through the created electronic rearview mirror scene data set, and then to peel off the obtained model, and finally to obtain the required IAEM module. The neural network model can be an improved IA-YOLO model, and the target model is a neural network model including a hyperparameter estimation network and an image enhancement module.

[0075] Furthermore, before step S202, the method further includes:

[0076] Obtain historical scene datasets;

[0077] The initial neural network model is trained using the historical scene data set to obtain a neural network model.

[0078] It should be noted that first, the model structure was optimized based on the CNNPP module, which enhanced the feature extraction capability of the initial neural network model and accelerated the model reasoning speed.

[0079] In the specific implementation, a large-scale public historical scene dataset of self-driving, parking and other scenarios is used to train the improved CNNPP module and the YOLO algorithm until the model converges to the ideal downstream task detection level.

[0080] As shown in Table 1, Table 1 is a comparison table of hyperparameter estimation network optimization improvements:

[0081] Table 1

[0082]

[0083] The improved model uses the GhostConv module to replace part of the original Conv module. The GhostConv module can greatly reduce the computational complexity of the convolution operation by adding a convolution to halve the number of feature channels. At the same time, the addition of the convolution connection branch further enriches the feature source of the model, so the feature extraction capability is stronger than the original CNNPP module. In addition, in layer5, the fifth layer, an SE channel attention mechanism module is used to replace GhostConv, so that the model can further focus on the features of certain specific channels when estimating hyperparameters, thereby guiding the model to obtain better parameter estimation effects.

[0084] It should be noted that the improved CNNPP-GSE module (i.e., hyperparameter estimation network) has increased internal network branches and depth, which enhances the feature extraction capability. The GhostConv module reduces the amount of convolution calculations and speeds up the model reasoning. In conjunction with the channel attention mechanism, better hyperparameter estimation capabilities can be obtained.

[0085] It should be understood that in order for the model to acquire these capabilities, it needs to be trained. If the traditional data annotation supervised training method is used, it is necessary to design feature enhancement parameters for each image. This is an intuitive but unrealistic approach, and the workload is huge and the interpretability is poor. Therefore, this strategy directly uses the IA-YOLO training mode to complete the training, and replaces the Yolov3 in the original text with the current newer yolov8. The algorithm itself can be trained for downstream tasks such as target detection and target segmentation, and the algorithm output results depend on the visual features of the input image. Therefore, the quality of the algorithm output results also directly depends on the visual effect of the input image. This means that the algorithm can be used as a medium to replace human annotation supervised image enhancement training. Therefore, a large number of existing public data sets can be used to jointly train the model. These data sets contain pictures under different weather conditions and lighting conditions in driving scenarios. After training, a model that can perform data enhancement in general scenarios is obtained.

[0086] Step S203: performing module stripping processing on the target model to obtain an end-adaptive image enhancement model.

[0087] In the specific implementation, the IAEM module is separated from the joint task training model, and an end-to-end model consisting of hyperparameter estimation and image enhancement modules is obtained separately.

[0088] Furthermore, the step S203 includes:

[0089] The hyperparameter estimation network and image enhancement module of the target model are stripped off to obtain an end-to-end adaptive image enhancement model based on the hyperparameter estimation network and the image enhancement module.

[0090] It should be noted that the stripping process is to separate certain parts of the model so that these parts can be processed or optimized independently to improve the flexibility and efficiency of the model.

[0091] This embodiment provides a method for visual enhancement of electronic rearview mirror images in severe weather conditions, which uses the IAEM neural network module to achieve image enhancement of the imaging visual effect of the electronic rearview mirror under severe weather conditions. Drawing on the successful experience of the IA-YOLO algorithm, the CNNPP module is optimized and improved and combined with the DIP module to be transplanted into the image enhancement application of the electronic rearview mirror to address the problem of poor visual effects of traditional rearview mirrors in special severe weather conditions. An end-to-end adaptive image enhancement model is formed, and a data set preparation and model migration fine-tuning training method for electronic rearview mirror image scenes are proposed.

[0092] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction, and will not be described in detail later. Figure 5 , step S30, the electronic rearview mirror bad weather image visual enhancement method further includes steps S301-S302:

[0093] Step S301, performing denoising, white balance, gamma correction, hue adjustment, contrast adjustment, and sharpening processing on the original input image in sequence through the adjusted image enhancement module to obtain an enhanced image;

[0094] It should be noted that the dehaze / denoising module is a module that uses the atmospheric scattering model to restore images with limited vision due to the air light scattering phenomenon in the image. It can remove the influence of fog, dust, rain and snow to a certain extent; the white balance, gamma correction, contrast and tone adjustment modules are traditional image processing filter modules with pixel-level filtering capabilities. Through parameter adjustment, it can achieve visual effect enhancement in various aspects such as image color brightness; the sharpening module sharpens and enhances the image by introducing image sharpening calculation, which can effectively highlight the detail information of objects in the image and enhance the sense of boundaries of objects.

[0095] Step S302: fusing the enhanced image with the original input image to obtain a target image.

[0096] It should be noted that the fusion processing is to make the final image display effect more in line with the sensory requirements of the actual scene.

[0097] In a specific implementation, the original input image is resampled once and then pixel-wise fused with the restored enhanced image to obtain the target image.

[0098] It should be understood that pixel-level fusion is to combine the pixel values ​​of two or more images according to a certain rule to generate a new image; the fusion rule can be a simple average, weighted average, or a more complex algorithm, such as Laplace pyramid-based fusion, deep learning-based fusion, etc., where the purpose of fusion is to combine the advantages of different images, such as improving resolution, enhancing details, fusing data from different sensors, etc.

[0099] Furthermore, the step S302 includes:

[0100] Reduce the image pixel value of the enhanced image to obtain a restored image;

[0101] Resampling the original input image to obtain a resampled original input image;

[0102] The restored image and the resampled original input image are fused to obtain the target image.

[0103] It should be noted that the image pixel value of the reduced enhanced image is to rescale the output result to the RGB image data range of 0-255 through image restoration after the image denoising, white balance, gamma correction, tone adjustment, contrast adjustment, and sharpening processing is completed.

[0104] It should be understood that resampling is the process of changing the resolution of an image, which can be upsampling (increasing the resolution) or downsampling (reducing the resolution); the resampled image is then enhanced to improve its visual effect.

[0105] This embodiment provides a method for visual enhancement of electronic rearview mirror images in severe weather conditions. Through the enhancement module, the electronic rearview mirror images in specific scenes and special weather conditions are adaptively enhanced by a combination of various methods such as defogging / denoising, brightness adjustment, contrast adjustment, and detail sharpening. Enhanced processing of the electronic rearview mirror images under various severe weather conditions is achieved, and the visual effects in all severe weather scenes are improved to bring better observation experience to the driver.

[0106] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the method for visual enhancement of severe weather images of the electronic rearview mirror of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0107] This application also provides an electronic rearview mirror bad weather image visual enhancement device, please refer to Figure 6 , the electronic rearview mirror bad weather image visual enhancement device comprises:

[0108] The image acquisition module 10 is used to acquire an original input image.

[0109] The data processing module 20 is used to process the original input image through a hyperparameter estimation network to obtain hyperparameters of the image enhancement module.

[0110] The image enhancement module 30 is used to adjust the image enhancement module based on the hyperparameters, and then perform image enhancement processing on the original input image through the adjusted image enhancement module to obtain a target image.

[0111] The electronic rearview mirror bad weather image visual enhancement device provided by the present application adopts the electronic rearview mirror bad weather image visual enhancement method in the above embodiment, which can solve the technical problem of low adaptability of the enhancement processing of the electronic rearview mirror image. Compared with the prior art, the beneficial effects of the electronic rearview mirror bad weather image visual enhancement device provided by the present application are the same as the beneficial effects of the electronic rearview mirror bad weather image visual enhancement method provided by the above embodiment, and the other technical features of the electronic rearview mirror bad weather image visual enhancement device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0112] In one embodiment, the data processing module 20 is also used to create an electronic rearview mirror scene data set; perform migration and fine-tuning training on the neural network model based on the electronic rearview mirror scene data set to obtain a target model; and perform module stripping processing on the target model to obtain an end-adaptive image enhancement model.

[0113] In one embodiment, the data processing module 20 is also used to strip off the hyperparameter estimation network and the image enhancement module of the target model to obtain an end-to-end adaptive image enhancement model based on the hyperparameter estimation network and the image enhancement module.

[0114] In one embodiment, the image enhancement module 30 is further used to perform denoising, white balance, gamma correction, hue adjustment, contrast adjustment, and sharpening processing on the original input image in sequence through the adjusted image enhancement module to obtain an enhanced image; and to fuse the enhanced image and the original input image to obtain a target image.

[0115] In one embodiment, the image enhancement module 30 is also used to reduce the image pixel value of the enhanced image to obtain a restored image; resample the original input image to obtain a resampled original input image; and perform image fusion processing on the restored image and the resampled original input image to obtain a target image.

[0116] In one embodiment, the image acquisition module 10 is also used to preprocess the original input image to obtain a normalized input image; the step of processing the original input image through a hyperparameter estimation network in an end-to-end adaptive image enhancement model to obtain hyperparameters of the image enhancement module also includes: processing the normalized input image through a hyperparameter estimation network in an end-to-end adaptive image enhancement model to obtain hyperparameters of the image enhancement module.

[0117] In one embodiment, the data processing module 20 is further used to obtain a historical scene data set; and train the initial neural network model using the historical scene data set to obtain a neural network model.

[0118] The present application provides an electronic rearview mirror severe weather image visual enhancement device, the electronic rearview mirror severe weather image visual enhancement device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the electronic rearview mirror severe weather image visual enhancement method in the above-mentioned embodiment one.

[0119] Reference below Figure 7 , which shows a schematic diagram of the structure of the electronic rearview mirror bad weather image visual enhancement device suitable for implementing the embodiment of the present application. The electronic rearview mirror bad weather image visual enhancement device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic rearview mirror severe weather image visual enhancement device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0120] like Figure 7As shown, the electronic rearview mirror bad weather image visual enhancement device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the electronic rearview mirror bad weather image visual enhancement device are also stored. The processing device 1001, ROM 1002 and RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the electronic rearview mirror severe weather image visual enhancement device to communicate wirelessly or wired with other devices to exchange data. Although the electronic rearview mirror severe weather image visual enhancement device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0121] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0122] The electronic rearview mirror bad weather image visual enhancement device provided by the present application adopts the electronic rearview mirror bad weather image visual enhancement method in the above embodiment, which can solve the technical problem of low adaptability of the enhancement processing of the electronic rearview mirror image. Compared with the prior art, the beneficial effects of the electronic rearview mirror bad weather image visual enhancement device provided by the present application are the same as the beneficial effects of the electronic rearview mirror bad weather image visual enhancement method provided by the above embodiment, and the other technical features of the electronic rearview mirror bad weather image visual enhancement device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0123] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0124] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0125] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the method for visual enhancement of electronic rearview mirror severe weather images in the above-mentioned embodiment.

[0126] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0127] The computer-readable storage medium may be included in the electronic rearview mirror severe weather image visual enhancement device; or it may exist independently without being assembled into the electronic rearview mirror severe weather image visual enhancement device.

[0128] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic rearview mirror severe weather image visual enhancement device, the electronic rearview mirror severe weather image visual enhancement device: obtains the original input image; processes the original input image through the hyperparameter estimation network in the end-to-end adaptive image enhancement model to obtain the hyperparameters of the image enhancement module; after adjusting the image enhancement module based on the hyperparameters, the original input image is enhanced by the adjusted image enhancement module to obtain the target image.

[0129] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0132] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned electronic rearview mirror bad weather image visual enhancement method, which can solve the technical problem of low adaptability of electronic rearview mirror image enhancement processing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the electronic rearview mirror bad weather image visual enhancement method provided by the above-mentioned embodiment, and will not be repeated here.

[0133] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for visual enhancement of electronic rearview mirror severe weather images.

[0134] The computer program product provided by the present application can solve the technical problem of low adaptability of the enhancement processing of the electronic rearview mirror image. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the electronic rearview mirror bad weather image visual enhancement method provided by the above embodiment, and will not be repeated here.

[0135] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for enhancing the visual quality of an electronic rearview mirror image in bad weather, characterized in that: The electronic rearview mirror bad weather image visual enhancement method comprises the following steps: Get the original input image; Processing the original input image through a hyperparameter estimation network in an end-to-end adaptive image enhancement model to obtain hyperparameters of an image enhancement module; After the image enhancement module is adjusted based on the hyperparameter, the original input image is subjected to image enhancement processing by the adjusted image enhancement module to obtain a target image.

2. The method for enhancing the visual performance of an electronic rearview mirror image in bad weather as claimed in claim 1, characterized in that: Before the step of processing the original input image by the hyperparameter estimation network in the end-to-end adaptive image enhancement model to obtain the hyperparameters of the image enhancement module, the method further includes: Create an electronic rearview mirror scene dataset; Performing migration and fine-tuning training on the neural network model based on the electronic rearview mirror scene data set to obtain a target model; The target model is subjected to module stripping processing to obtain an end-adaptive image enhancement model.

3. The method for enhancing the visual quality of an electronic rearview mirror image in bad weather as claimed in claim 2, characterized in that: The step of performing module stripping processing on the target model to obtain an end-adaptive image enhancement model includes: The hyperparameter estimation network and the image enhancement module of the target model are stripped to obtain an end-to-end adaptive image enhancement model based on the hyperparameter estimation network and the image enhancement module.

4. The method for enhancing the visual performance of an electronic rearview mirror in bad weather conditions as claimed in claim 1, characterized in that: The step of performing image enhancement processing on the original input image by the adjusted image enhancement module to obtain a target image comprises: The original input image is subjected to denoising, white balance, gamma correction, hue adjustment, contrast adjustment, and sharpening processing in sequence by the adjusted image enhancement module to obtain an enhanced image; The enhanced image and the original input image are fused to obtain a target image.

5. The method for enhancing the visual quality of an electronic rearview mirror image in bad weather as claimed in claim 4, characterized in that: The step of fusing the enhanced image with the original input image to obtain a target image comprises: Reducing the image pixel values ​​of the enhanced image to obtain a restored image; Resampling the original input image to obtain a resampled original input image; The restored image and the resampled original input image are subjected to image fusion processing to obtain a target image.

6. The method for enhancing the visual quality of an electronic rearview mirror image in bad weather as claimed in claim 1, characterized in that: After the step of obtaining the original input image, the method further includes: Preprocessing the original input image to obtain a normalized input image; The step of processing the original input image through a hyperparameter estimation network in the end-to-end adaptive image enhancement model to obtain hyperparameters of the image enhancement module also includes: The normalized input image is processed by a hyperparameter estimation network in an end-to-end adaptive image enhancement model to obtain hyperparameters of an image enhancement module.

7. The method for enhancing the visual quality of an electronic rearview mirror image in bad weather as claimed in claim 2, characterized in that: Before the step of performing migration fine-tuning training on the neural network model based on the electronic rearview mirror scene data set to obtain the target model, the step further includes: Obtain historical scene datasets; The initial neural network model is trained using the historical scene data set to obtain a neural network model.

8. An electronic rearview mirror bad weather image visual enhancement device, characterized in that: The device comprises: An image acquisition module, used to acquire an original input image; A data processing module, used to process the original input image through a hyperparameter estimation network to obtain hyperparameters of an image enhancement module; The image enhancement module is used to adjust the image enhancement module based on the hyperparameters, and then perform image enhancement processing on the original input image through the adjusted image enhancement module to obtain a target image.

9. An electronic rearview mirror bad weather image visual enhancement device, characterized in that: The device includes: a memory, a processor, and an electronic rearview mirror severe weather image visual enhancement program stored in the memory and executable on the processor, wherein the electronic rearview mirror severe weather image visual enhancement program is configured to implement the steps of the electronic rearview mirror severe weather image visual enhancement method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores an electronic rearview mirror severe weather image visual enhancement program, and when the electronic rearview mirror severe weather image visual enhancement program is executed by the processor, the steps of the electronic rearview mirror severe weather image visual enhancement method as described in any one of claims 1 to 7 are implemented.

Images