Night scene image enhancement method, device and storage medium

Abstract

The application discloses a night scene image enhancement method and device and a storage medium. The night scene image enhancement method comprises the following steps: acquiring a night scene image to be enhanced; performing image enhancement processing on the night scene image to be enhanced by using a CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm to obtain a first image; and performing image enhancement processing on the first image by using a Zero-DCE image enhancement network to obtain a target image. The night scene image enhancement method combines the CLAHE algorithm and the Zero-DCE image enhancement deep learning network, can suppress noise while enhancing the definition and contrast of the image, and is suitable for complex scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a night scene image enhancement method, device and storage medium. BACKGROUND

[0002] In a production environment, video images are a necessary monitoring element for production and safety. However, due to low light or backlight environment, especially in the case of insufficient light, the video images captured often have underexposure phenomenon, resulting in a decrease in image resolution.

[0003] To this end, the related art proposes a night scene image enhancement algorithm, including: 1) a histogram equalization (Histogram equalization) method, which simply uses the statistical properties of the overall image and usually cannot achieve ideal results for complex scenes; 2) an enhancement algorithm based on deep learning theory, which usually only uses a single channel for light optimization, and the color cannot be well restored, and in the case of complex light, overexposure phenomenon is also prone to occur, and a specific inference platform is required, and the pre-training work task is relatively large. In addition, the above method also tends to amplify noise while increasing image brightness, affecting image quality. SUMMARY

[0004] The present application aims to at least solve one of the technical problems in the related art. To this end, one object of the present application is to propose a night scene image enhancement method, device and storage medium, which combines the limited contrast adaptive histogram equalization (CLAHE) algorithm and the Zero-DCE image enhancement deep learning network, and can suppress noise while enhancing image clarity and contrast, and is suitable for complex scenes. The present application also integrates the two into an embedded platform, so that the embedded platform has learning evolution and intelligent growth functions.

[0005] To achieve the above object, the first embodiment of the present application proposes a night scene image enhancement method, which comprises: acquiring a night scene image to be enhanced; performing image enhancement processing on the night scene image to be enhanced by a limited contrast adaptive histogram equalization (CLAHE) algorithm to obtain a first image; and performing image enhancement processing on the first image by a Zero-DCE image enhancement network to obtain a target image.

[0006] In addition, the night scene image enhancement method according to the above embodiments of the present application can also have the following additional technical features:

[0007] According to one embodiment of the present application, the method further comprises: establishing a night scene image enhancement database; converting a.pth format file of a Zero-DCE image enhancement network based on a pytorch framework into a prototxt file and a.caffemodel format file based on a caffe framework; building a caffe framework and importing the prototxt file and the.caffemodel format file into the caffe framework to obtain a Zero-DCE image enhancement network based on the caffe framework; training the Zero-DCE image enhancement network based on the caffe framework using the database, and transplanting the trained Zero-DCE image enhancement network based on the caffe framework to an embedded platform; and wherein the trained Zero-DCE image enhancement network based on the caffe framework on the embedded platform is used to perform image enhancement processing on the first image.

[0008] According to one embodiment of the present application, the method further comprises: creating a CLAHE algorithm using a function in an OpenCV library and transplanting the created CLAHE algorithm to the embedded platform; and wherein the CLAHE algorithm on the embedded platform is used to perform image enhancement processing on the night scene image to be enhanced.

[0009] According to one embodiment of the present application, the.pth format file includes pytorch parameter names and weights stored in a dictionary, and the conversion of the.pth format file of the Zero-DCE image enhancement network based on the pytorch framework into the prototxt file and the.caffemodel format file based on the caffe framework comprises: establishing a parameter name mapping from the caffe framework to the pytorch framework, and storing the parameter name mapping in a dictionary or a txt to obtain the prototxt file; and based on the parameter name mapping, assigning the weights in the.pth format file to corresponding caffe parameters to obtain the.caffemodel format file.

[0010] According to one embodiment of the present application, the.pth format file of the Zero-DCE image enhancement network based on the pytorch framework is converted into the prototxt file and the.caffemodel format file based on the caffe framework through a pytorch2caffe tool.

[0011] According to one embodiment of the present application, after the prototxt file and the.caffemodel format file are imported into the caffe framework, it is further tested whether the prototxt file and the.caffemodel format file are normally run in the caffe framework, and if so, the Zero-DCE image enhancement network based on the caffe framework is obtained.

[0012] According to one embodiment of the present application, the image enhancement processing of the first image by the Zero-DCE image enhancement network to obtain a target image comprises: inputting the first image into the Zero-DCE image enhancement network to output a set of parameters of light enhancement curves; and applying the parameters of the light enhancement curves to all pixels of the RGB channel of the first image to obtain the target image.

[0013] According to one embodiment of the present application, the light enhancement curve is represented by the following formula:

[0014] LE(x)=LE n-1 (x)+A n (x)LE n-1 (x)(1-LE n-1 (x))

[0015] wherein, A n (x) represents a matrix composed of α corresponding to each pixel value in the first image, α∈[-1,1], which is a trainable parameter of the light enhancement curve, used to adjust the series of the light enhancement curve and the exposure, x is the pixel coordinate of the first image, LE n (x) is the enhanced image of the first image in the n-1th iteration.

[0016] According to one embodiment of the present application, the Zero-DCE image enhancement network is composed of 7 convolutional layers with symmetrical structure, the convolution kernel of the first 6 layers is 3x3x32, the step is 1, followed by a ReLU activation function, and the convolution channel of the last layer is 24, used for the parameter matrix of 8 iteration rounds, followed by a Tanh activation function.

[0017] The night scene image enhancement method of the embodiment of the present application firstly acquires a to-be-enhanced night scene image, performs a limited contrast self-adaptive histogram equalization (CLAHE) algorithm processing on the to-be-enhanced night scene image, then performs a deep learning processing on the to-be-enhanced night scene image by using a Zero-DCE image enhancement network, and obtains a target image. The night scene image enhancement method of the present application combines the limited contrast self-adaptive histogram equalization (CLAHE) algorithm and the Zero-DCE image enhancement deep learning network, can suppress noise while enhancing the definition and contrast of the image, and is suitable for complex scenes. The present application also integrates the two into an embedded platform, so that the embedded platform has the functions of learning evolution and intelligent growth.

[0018] To achieve the above-mentioned purpose, the second aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the night scene image enhancement method as described above.

[0019] To achieve the above-mentioned purpose, the third aspect of the present application provides a night scene image enhancement device, which comprises: an acquisition module configured to acquire a to-be-enhanced night scene image; a first processing module configured to perform an image enhancement processing on the to-be-enhanced night scene image by using a limited contrast self-adaptive histogram equalization (CLAHE) algorithm, and obtain a first image; and a second processing module configured to perform an image enhancement processing on the first image by using a Zero-DCE image enhancement network, and obtain a target image.

[0020] According to an embodiment of the present application, the device further comprises: a building module configured to build a night scene image enhancement database; a converting module configured to convert a.pth format file of the Zero-DCE image enhancement network based on a pytorch framework into a prototxt file and a.caffemodel format file based on a caffe framework; a building module configured to build a caffe framework; an importing module configured to import the prototxt file and the.caffemodel format file into the caffe framework, and obtain a Zero-DCE image enhancement network based on the caffe framework; a training module configured to train the Zero-DCE image enhancement network based on the caffe framework using the database; and a first transplanting module configured to transplant the trained Zero-DCE image enhancement network based on the caffe framework to an embedded platform. The second processing module is specifically configured to perform an image enhancement processing on the first image by using the trained Zero-DCE image enhancement network based on the caffe framework on the embedded platform.

[0021] According to one embodiment of the present application, the device further comprises a creating module configured to create a CLAHE algorithm by using a function in an OpenCV library; and a second transplanting module configured to transplant the created CLAHE algorithm to the embedded platform; wherein the first processing module is specifically configured to perform image enhancement processing on the to-be-enhanced night scene image by using the CLAHE algorithm on the embedded platform.

[0022] Additional aspects and advantages of the present application will be given in part in the following description, become apparent from the following description, or be understood through practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a flowchart of a night scene image enhancement method according to one embodiment of the present application;

[0024] Figure 2 is a flowchart of creating a CLAHE algorithm according to one embodiment of the present application;

[0025] Figure 3 is a flowchart of converting a pytorch framework to a caffe framework according to one embodiment of the present application;

[0026] Figure 4 is a flowchart of converting a.pth format file to a prototxt file and a.caffemodel format file according to one embodiment of the present application;

[0027] Figure 5 is a flowchart of performing image enhancement processing on a first image by a Zero-DCE image enhancement network according to one embodiment of the present application;

[0028] Figure 6 is a schematic diagram of a night scene image enhancement device according to one embodiment of the present application;

[0029] Figure 7 is a schematic diagram of a night scene image enhancement device according to one specific embodiment of the present application. DETAILED DESCRIPTION

[0030] The embodiments of the present application are described in detail below with reference to the accompanying drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and should not be understood as limiting the present application.

[0031] The night scene image enhancement method, device and storage medium according to the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0032] Figure 1is a flow chart of a night scene image enhancement method according to an embodiment of the present application.

[0033] In an embodiment of the present application, as shown in Figure 1 the night scene image enhancement method comprises:

[0034] S1, obtaining a night scene image to be enhanced.

[0035] Specifically, the night scene image enhancement method provided by the present application combines a CLAHE (Contrast Limited Adaptive Histogram Equalization, contrast limited adaptive histogram equalization) algorithm and a Zero-DCE image enhancement network to process the night scene image, and integrates the CLAHE algorithm and the Zero-DCE image enhancement network into an embedded platform. The CLAHE algorithm is used as a pre-processing of image processing, and the Zero-DCE image enhancement network is used as a main part of image processing. The embedded platform obtains the night scene image to be enhanced on the platform, first implements the CLAHE algorithm on the night scene image to be enhanced, enhances the contrast of the night scene image to be enhanced, and then inputs the night scene image to be enhanced into the Zero-DCE image enhancement network to optimize the illumination of the night scene image to be enhanced.

[0036] S2, performing image enhancement processing on the night scene image to be enhanced by using a CLAHE algorithm to obtain a first image.

[0037] Specifically, the night scene image usually has a large dark area or overexposure. When analyzing the light and dark characteristics of the night scene image, the brightness interval is qualitatively divided into dark tone, shadow, middle tone, light tone, highlight and the like, and the boundaries of each region can be flexibly controlled according to application characteristics. When analyzing the color characteristics of the night scene image, the present application uses a CLAHE algorithm to perform image enhancement processing on the night scene image to be enhanced. The CLAHE algorithm performs histogram statistics on the R / G, B color channels of the night scene image to be enhanced. The histogram mean value is calculated. The brightness distribution of the night scene image to be enhanced is adjusted according to the histogram mean value.

[0038] Further specifically, CLAHE is a classic histogram equalization algorithm, and almost all image processing software supports the CLAHE algorithm. The CLAHE algorithm of the present application needs to be integrated with the Zero-DCE image enhancement network into an embedded platform. Therefore, after the CLAHE algorithm is created, the created CLAHE algorithm is transplanted to the embedded platform.

[0039] In an embodiment of the present application, as shown in Figure 2 the night scene image enhancement method further comprises:

[0040] S201, a CLAHE algorithm is created by using a function in an OpenCV library, and the created CLAHE algorithm is transplanted to an embedded platform.

[0041] S202, the CLAHE algorithm on the embedded platform is used to perform image enhancement processing on the night scene image to be enhanced.

[0042] Specifically, image processing software such as OpenCV, ImageJ, Matlab, etc. can support the CLAHE algorithm. The CLAHE algorithm is created by using a function in an OpenCV library. The basic steps of the CLAHE algorithm are as follows: first, the night scene image to be enhanced is divided into a plurality of sub-blocks according to parameters, the histogram of each sub-block is equalized, the histogram and the cumulative histogram are calculated, the original gray scale and the equalized gray scale mapping table of each block are obtained, each gray scale level of the histogram of each sub-block is limited and cropped within a certain threshold range, each point of the night scene image to be enhanced is traversed, the gray scale value of each point is combined with the position and the surrounding associated sub-block to perform gray scale interpolation processing to obtain the gray scale value of the corresponding output image of each point. The brightness of the night scene image to be enhanced is adjusted according to the gray scale value of the corresponding output image of each point. The main function of the CLAHE algorithm is to enhance the contrast of the image while suppressing noise.

[0043] Further specifically, after the CLAHE algorithm is created by using a function in an OpenCV library, the created CLAHE algorithm is transplanted to an embedded platform. The CLAHE algorithm is used to perform image enhancement processing on the night scene image to be enhanced, and the image enhancement processing is performed on the embedded platform. After the embedded platform obtains the night scene image to be enhanced, the CLAHE algorithm is first performed on the night scene image to be enhanced to obtain a first image, and then the first image is input into a Zero-DCE image enhancement network, and the Zero-DCE image enhancement network outputs a target image.

[0044] S3, the first image is processed by the Zero-DCE image enhancement network to obtain a target image.

[0045] Specifically, the Zero-DCE image enhancement network is a deep learning model based on the pytorch framework, and in order to integrate the Zero-DCE image enhancement network and the CLAHE algorithm into an embedded platform, the deep learning model needs to be converted into an embedded platform support model, and the embedded platform support model is under the caffe (Convolutional Architecture for Fast Feature Embedding, convolutional architecture for fast feature embedding) framework, so the Zero-DCE image enhancement network based on the pytorch framework is converted into the Zero-DCE image enhancement network based on the caffe framework.

[0046] In one embodiment of the present application, as shown in Figure 3 The night scene image enhancement method further comprises:

[0047] S101, a night scene image enhancement database is established.

[0048] S102, convert the.pth format file of the Zero-DCE image enhancement network based on the pytorch framework into the prototxt file and the.caffemodel format file based on the caffe framework.

[0049] S103, build a caffe framework, and import the prototxt file and the.caffemodel format file into the caffe framework to obtain the Zero-DCE image enhancement network based on the caffe framework.

[0050] S104, train the Zero-DCE image enhancement network based on the caffe framework using the database, and transplant the trained Zero-DCE image enhancement network based on the caffe framework to an embedded platform.

[0051] S105, wherein the trained Zero-DCE image enhancement network based on the caffe framework on the embedded platform is used for image enhancement processing of the first image.

[0052] Specifically, after converting the Zero-DCE image enhancement network based on the pytorch framework into the Zero-DCE image enhancement network based on the caffe framework, the Zero-DCE image enhancement network based on the caffe framework needs to be trained, so first a night scene image enhancement database is established for training the Zero-DCE image enhancement network based on the caffe framework. The night scene image needs to be preprocessed between the night scene image enhancement databases. The night scene image data is pre-stored in the database, which can accelerate the training rhythm.

[0053] Further specifically, the Zero-DCE image enhancement network under the pytorch framework is converted into the Zero-DCE image enhancement network under the caffe framework, first, the Zero-DCE image enhancement network npy file, i.e., the.pth format file under the pytorch framework is converted into the prototxt file and the.caffemodel format file based on the caffe framework, the architecture of the caffe is written according to the network architecture of the pytorch to obtain the prototxt file, the parameter weight of the pytorch framework is assigned to the corresponding caffe parameter to obtain the.caffemodel file; then the caffe framework is built, the prototxt file and the.caffemodel format file are imported into the caffe framework to obtain the Zero-DCE image enhancement network based on the caffe framework, and then the Zero-DCE image enhancement network of the caffe framework is trained. The first image processed by the CLAHE algorithm is processed by using the trained Zero-DCE image enhancement network to obtain the target image required for tracking.

[0054] In an embodiment of the present application, as shown in Figure 4 The.pth format file includes the pytorch parameter name and weight stored in a dictionary, the.pth format file of the Zero-DCE image enhancement network based on the pytorch framework is converted into the prototxt file and the.caffemodel format file based on the caffe framework, including:

[0055] In S1021, the parameter name mapping from the caffe framework to the pytorch framework is established, and the parameter name mapping is stored in a dictionary or a txt file to obtain the prototxt file.

[0056] In S1022, based on the parameter name mapping, the weight in the.pth format file is assigned to the corresponding caffe parameter to obtain the.caffemodel format file.

[0057] Specifically, when the deep learning model is converted into the model supported by the embedded platform, the parameter name and weight of the pytorch framework are saved and stored in a dictionary, for example, in the.pth format file, the pytorch code of the Zero-DCE is converted into the prototxt file corresponding to the caffe by using a conversion tool. The parameter name mapping from the caffe framework to the pytorch framework is established, and the parameter name mapping is stored in a dictionary or a txt file, based on the parameter name mapping, the weight in the.pth format file is assigned to the corresponding caffe parameter to obtain the.caffemodel format file.

[0058] In an embodiment of the present application, the.pth format file of the Zero-DCE image enhancement network based on the pytorch framework is converted into the prototxt file and the.caffemodel format file based on the caffe framework by the pytorch2caffe tool.

[0059] Specifically, the conversion tool between the pytorch framework and the caffe framework can directly use the open source library pytorch2caffe tool for conversion. The present application uses the open source library pytorch2caffe as the conversion tool, which can be directly downloaded and used at the website https: / / github.com / woodsgao / pytorch2caffe. And the.pth format file of the Zero-DCE image enhancement network based on the pytorch framework is converted into the prototxt file and the.caffemodel format file based on the caffe framework by the pytorch2caffe tool.

[0060] Further specifically, after obtaining the prototxt file and the.caffemodel format file based on the caffe framework, the prototxt file and the.caffemodel format file are imported into the caffe framework, and it is also necessary to test whether the prototxt file and the.caffemodel format file normally run in the caffe framework.

[0061] In an embodiment of the present application, after the prototxt file and the.caffemodel format file are imported into the caffe framework, it is also necessary to test whether the prototxt file and the.caffemodel format file normally run in the caffe framework, and if so, the Zero-DCE image enhancement network based on the caffe framework is obtained.

[0062] Specifically, the prototxt file and the.caffemodel format file are imported into the caffe framework, and it is tested whether they normally run. It is observed whether the specified output file is output after the running is completed. If there is no error, it means that the output is normal, indicating that the prototxt file and the.caffemodel format file can normally run in the caffe framework, and the output file is determined by the model.

[0063] Further specifically, after the conversion of the Zero-DCE image enhancement network between the pytorch framework and the caffe framework is completed, the Zero-DCE image enhancement network based on the caffe framework is transplanted to the embedded platform, and the first image processed by the CLAHE algorithm is subjected to image enhancement processing.

[0064] In one embodiment of the present application, as shown in Figure 5 The first image is subjected to image enhancement processing by a Zero-DCE image enhancement network to obtain a target image, including:

[0065] S31, input the first image into the Zero-DCE image enhancement network, and output a set of parameters of light enhancement curves.

[0066] Specifically, the first image processed by the CLAHE algorithm is input into the Zero-DCE image enhancement network, and a set of parameters of light enhancement curves is output after convolution, pooling and other processing of the Zero-DCE image enhancement network.

[0067] In one embodiment of the present application, the Zero-DCE image enhancement network is composed of 7 convolutional layers with symmetrical structure, the convolution kernel of the first 6 layers is 3x3x32, the step is 1, followed by a ReLU activation function, and the last layer convolution channel is 24, which is used for the parameter matrix of 8 iteration rounds, followed by a Tanh activation function.

[0068] Specifically, the Zero-DCE image enhancement network of the present application is composed of 7 convolutional layers with symmetrical structure, the convolution kernel of the first 6 layers is 3x3x32, the step is 1, the convolution channel is 32, a ReLU activation function is connected after each convolutional layer of the first 6 layers, the convolution kernel of the last layer is 3x3x24, the convolution channel is 24, which is used for the parameter matrix of 8 iteration rounds, followed by a Tanh activation function. After the first image is processed by the Zero-DCE image enhancement network, a set of parameters of light enhancement curves is output. The parameters of the light enhancement curve include the parameters of the R / G, B three channels of the first image.

[0069] In one embodiment of the present application, the light enhancement curve is represented by the following formula:

[0070] LE(x)=LE n-1 (x)+A n (x)LE n-1 (x)(1-LE n-1 (x))

[0071] Wherein, A n (x) represents a matrix composed of each pixel value in the first image, and a∈[-1,1] is a trainable parameter of the light enhancement curve, which is used to adjust the order of the light enhancement curve and the exposure, x is the pixel coordinate of the first image, and LE n (x) is the enhanced image of the first iteration of the first image.

[0072] Specifically, the Zero-DCE image enhancement network designs an image-specific light enhancement curve that can approximate pixel-level and high-order curves by iteratively applying itself. This image-specific light enhancement curve can effectively perform mapping in a wider dynamic range.

[0073] Further specifically, the mapping of the light enhancement curve of the image needs to follow: 1) each pixel value will be normalized to [0, 1] to prevent information loss. 2) the curve must be monotonic to prevent the difference between adjacent pixels from disappearing. 3) the form of the curve should be as simple as possible and differentiable. According to the above three conditions, the light enhancement curve as shown in the above formula is set. x is the pixel coordinate of the first image, and alpha is a trainable parameter of the curve, which is used to adjust the order of the curve and the exposure. Each pixel value is between [0, 1], and each operation is used at the pixel level. The curve is applied to the input RGB channel respectively, which can better maintain the inherent color and avoid overfitting.

[0074] S32, apply the parameters of the light enhancement curve to all pixels of the RGB channel of the first image to obtain a target image.

[0075] Specifically, in order to adjust the image in a wider dynamic range, we change the global alpha to an alpha for each pixel, and denote the matrix composed of each pixel value corresponding to alpha as A n (x). As shown in the above formula, the light enhancement curve is applied to all pixels of the RGB channel of the first image by using the light enhancement curve parameters obtained by the Zero-DCE image enhancement network to obtain a target image. The clarity and contrast of the target image are improved compared with the original image, and there is no image noise.

[0076] The night scene image enhancement method of the embodiment of the application first acquires a to-be-enhanced night scene image, performs a limited contrast self-adaptive histogram equalization (CLAHE) algorithm processing on the to-be-enhanced night scene image, then performs deep learning processing on the to-be-enhanced night scene image by using a Zero-DCE image enhancement network, and obtains a target image. The night scene image enhancement method combines the limited contrast self-adaptive histogram equalization (CLAHE) algorithm and the Zero-DCE image enhancement deep learning network, can enhance the clarity and contrast of an image while suppressing noise, is suitable for complex scenes, and integrates the two into an embedded platform, so that the embedded platform has the functions of learning evolution and intelligent growth.

[0077] The application further provides a computer readable storage medium.

[0078] In an embodiment of the application, a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement the night scene image enhancement method as described above.

[0079] The application further provides a night scene image enhancement device.

[0080] In one embodiment of the application, as shown in the figure, Figure 6 The night scene image enhancement device 100 comprises: an acquisition module 101 configured to acquire a night scene image to be enhanced; a first processing module 102 configured to perform image enhancement processing on the night scene image to be enhanced by using a CLAHE algorithm to limit contrast, to obtain a first image; and a second processing module 103 configured to perform image enhancement processing on the first image by using a Zero-DCE image enhancement network, to obtain a target image.

[0081] In one embodiment of the application, as shown in the figure, Figure 7 The night scene image enhancement device 100 further comprises: an establishment module 104 configured to establish a night scene image enhancement database; a conversion module 105 configured to convert a.pth format file of the Zero-DCE image enhancement network based on a pytorch framework into a prototxt file and a.caffemodel format file based on a caffe framework; a building module 106 configured to build the caffe framework; an import module 107 configured to import the prototxt file and the.caffemodel format file into the caffe framework, to obtain the Zero-DCE image enhancement network based on the caffe framework; a training module 108 configured to train the Zero-DCE image enhancement network based on the caffe framework by using the database; a first transplantation module 109 configured to transplant the trained Zero-DCE image enhancement network based on the caffe framework to an embedded platform; and the second processing module 103 is specifically configured to perform image enhancement processing on the first image by using the trained Zero-DCE image enhancement network based on the caffe framework on the embedded platform.

[0082] In one embodiment of the application, as shown in the figure, Figure 7 The night scene image enhancement device 100 further comprises: a creation module 110 configured to create the CLAHE algorithm by using functions in an OpenCV library; and a second transplantation module 111 configured to transplant the created CLAHE algorithm to an embedded platform; and the first processing module 102 is specifically configured to perform image enhancement processing on the night scene image to be enhanced by using the CLAHE algorithm on the embedded platform.

[0083] It should be noted that other specific embodiments of the night scene image enhancement device 100 in the application can refer to the specific embodiments of the night scene image enhancement method in the application.

[0084] The night scene image enhancement method, device and storage medium provided by the embodiment of the present application first acquire a to-be-enhanced night scene image, perform a CLAHE algorithm processing on the to-be-enhanced night scene image, and then perform a deep learning processing on the to-be-enhanced night scene image by using a Zero-DCE image enhancement network to obtain a target image. The night scene image enhancement method provided by the present application combines the CLAHE algorithm and the Zero-DCE image enhancement deep learning network, can suppress noise while enhancing the definition and contrast of the image, and is suitable for complex scenes. The present application also integrates the two into an embedded platform, so that the embedded platform has the functions of learning evolution and intelligent growth.

[0085] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logic functions, which can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus or device, such as a computer-based system, a system including a processor or other system that can fetch the instructions from an instruction execution system, apparatus or device and execute the instructions, or in conjunction with these instruction execution systems, apparatus or devices. For the purpose of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device, or in conjunction with these instruction execution systems, apparatus or devices. More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connections having one or more wires (electrical devices), portable computer disks (magnetic devices), random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memories), fiber optic devices, and portable compact disc read-only memories (CDROMs). In addition, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by electronic conversion, interpretation or processing, if necessary, in other suitable ways, and then stored in a computer memory.

[0086] It should be understood that parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the above-described embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if realized by hardware, and as in another embodiment, any one or a combination of the following technologies known in the art can be used: discrete logic circuit with logic gates for implementing logic functions on data signals, application specific integrated circuit with suitable combination logic gates, programmable gate array (PGA), field programmable gate array (FPGA), etc.

[0087] In the description of the application, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the application. In the description of the application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0088] In the description of the application, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application.

[0089] In addition, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0090] In the present application, unless otherwise specifically defined and limited, the terms "mounting", "connection", "connection", "fixing" and the like should be broadly understood, for example, it can be fixed connection, or detachable connection, or integral; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through intermediate medium, it can be the internal communication of two elements or the interaction relationship of two elements, unless otherwise specifically limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0091] In the present application, unless otherwise explicitly specified and limited, a first feature is "on" or "under" a second feature can mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, the first feature is "over", "above" and "on top of" the second feature can mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is horizontally higher than the second feature. The first feature is "under", "below" and "underneath" the second feature can mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is horizontally lower than the second feature.

[0092] Although the embodiments of the present application have been shown and described above, it is to be understood that the above-described embodiments are exemplary only, and are not to be construed as limiting the present application, and that changes, modifications, substitutions and variations can be made thereto by those skilled in the art without departing from the scope of the present application.

Claims

1. A night scene image enhancement method characterized by, The method comprises: obtaining a night scene image to be enhanced; performing image enhancement processing on the night scene image to be enhanced through a CLAHE algorithm for limiting contrast, to obtain a first image; performing image enhancement processing on the first image through a Zero-DCE image enhancement network, to obtain a target image; The method further comprises: establishing a night scene image enhancement database; converting a.pth format file of the Zero-DCE image enhancement network based on the pytorch framework into a prototxt file and a.caffemodel format file based on the caffe framework; building a caffe framework, and importing the prototxt file and the.caffemodel format file into the caffe framework, to obtain a Zero-DCE image enhancement network based on the caffe framework; training the Zero-DCE image enhancement network based on the caffe framework using the database, and transplanting the trained Zero-DCE image enhancement network based on the caffe framework to an embedded platform; wherein the trained Zero-DCE image enhancement network based on the caffe framework on the embedded platform is used to perform image enhancement processing on the first image; The method further comprises: using a function in an OpenCV library to create a CLAHE algorithm, and transplanting the created CLAHE algorithm to the embedded platform; wherein the CLAHE algorithm on the embedded platform is used to perform image enhancement processing on the night scene image to be enhanced.

2. The night view image enhancing method according to claim 1, characterized by, The.pth format file comprises pytorch parameter names and weights stored in a dictionary, and the conversion of the.pth format file of the Zero-DCE image enhancement network based on the pytorch framework into the prototxt file and the.caffemodel format file based on the caffe framework comprises: mapping parameter names from the caffe framework to the pytorch framework, and storing the parameter name mapping in a dictionary or a txt, to obtain the prototxt file; based on the parameter name mapping, assigning weights in the.pth format file to corresponding caffe parameters, to obtain the.caffemodel format file.

3. The night-view image enhancing method according to claim 1 or 2, characterized by, The.pth format file of the Zero-DCE image enhancement network based on the pytorch framework is converted into the prototxt file and the.caffemodel format file based on the caffe framework through a pytorch2caffe tool.

4. The night view image enhancing method according to claim 1, characterized by, After the prototxt file and the.caffemodel format file are imported into the caffe framework, it is further tested whether the prototxt file and the.caffemodel format file normally run in the caffe framework, and if so, the Zero-DCE image enhancement network based on the caffe framework is obtained.

5. The night view image enhancing method according to claim 1, characterized by, The image enhancement processing on the first image through the Zero-DCE image enhancement network comprises: inputting the first image into the Zero-DCE image enhancement network to output parameters of a light enhancement curve; applying the parameters of the light enhancement curve to all pixels of an RGB channel of the first image to obtain the target image.

6. The night view image enhancing method according to claim 1, characterized by, The light enhancement curve is expressed by the following formula: wherein, represents a matrix of the alpha composition corresponding to each pixel value in the first image, a e [ 1,1] is a trainable parameter of the light enhancement curve, for adjusting the order of the light enhancement curve and the exposure, x is a pixel coordinate of the first image, is the enhanced image of the first image in the n-1th iteration.

7. The night-view image enhancing method of claim 1, wherein, The Zero-DCE image enhancement network is composed of 7 convolutional layers with symmetrical structure, the convolutional kernel of the first 6 layers is 3x3x32, the step is 1, followed by a ReLU activation function, the convolutional channel of the last layer is 24, used for an 8-iteration round of parameter matrix, followed by a Tanh activation function.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the night scene image enhancement method according to any one of claims 1-7.

9. A night view image enhancement apparatus characterized by comprising: The device comprises: an acquisition module configured to acquire a night scene image to be enhanced; a first processing module configured to perform image enhancement processing on the night scene image to be enhanced through a CLAHE algorithm to obtain a first image; a second processing module configured to perform image enhancement processing on the first image through a Zero-DCE image enhancement network to obtain a target image; a building module configured to build a night scene image enhancement database; a conversion module configured to convert a.pth format file of the Zero-DCE image enhancement network based on a pytorch framework into a prototxt file and a.caffemodel format file based on a caffe framework; a building module configured to build a caffe framework; an import module configured to import the prototxt file and the.caffemodel format file into the caffe framework to obtain a Zero-DCE image enhancement network based on the caffe framework; a training module configured to train the Zero-DCE image enhancement network based on the caffe framework using the database; a first transplantation module configured to transplant the trained Zero-DCE image enhancement network based on the caffe framework to an embedded platform; a creating module configured to create a CLAHE algorithm using functions in an OpenCV library; a second transplantation module configured to transplant the created CLAHE algorithm to the embedded platform; The second processing module is specifically configured to perform image enhancement processing on the first image using the trained Zero-DCE image enhancement network based on the caffe framework on the embedded platform; and the first processing module is specifically configured to perform image enhancement processing on the night scene image to be enhanced using the CLAHE algorithm on the embedded platform.

Images