Image processing method and device, equipment, medium, product and vehicle

Abstract

The invention relates to an image processing method and device, equipment, a medium, a product and a vehicle, and the method comprises the steps: carrying out the image enhancement of a target image, and obtaining an enhanced image; wherein the target image is an image shot by the target device for the current environment, and the quality of the shot image can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, medium, product and vehicle. Background Art

[0002] Advanced Driving Assistance System (ADAS) is an important component of autonomous driving. The system can use a variety of sensors installed on the vehicle to perform calculations and analyses based on the data collected by the sensors while the car is driving, to provide assistance to the driver, prompting the driver to be aware of possible dangers, and effectively increasing the comfort and safety of car driving.

[0003] However, the driving assistance system sometimes has a high error rate in the collected data due to certain interference factors. For example, in low-light and dim environments such as at night and in tunnels, the camera has difficulty capturing sufficient light, resulting in reduced image quality and making it difficult to identify objects in the image. Summary of the Invention

[0004] The embodiments of the present application provide an image processing method that can improve the quality of captured images to at least partially solve the above-mentioned technical problems.

[0005] In order to achieve the above-mentioned object, according to a first aspect of the present application, there is provided an image processing method, comprising:

[0006] Perform image enhancement on the target image to obtain an enhanced image;

[0007] The target image is an image captured by the target device for the current environment.

[0008] Optionally, performing image enhancement on the target image to obtain an enhanced image includes:

[0009] When the current environment meets the preset low-light environment conditions, the target image is enhanced based on the attention mechanism through the low-light enhancement network to obtain the enhanced image.

[0010] Optionally, before performing image enhancement on the target image based on the attention mechanism through the low-light enhancement network to obtain the enhanced image, the method further includes:

[0011] Performing multi-scale illumination enhancement processing on the target image to obtain an enhanced target image;

[0012] The low-light enhancement network is used to enhance the target image based on the attention mechanism to obtain an enhanced image, including:

[0013] The enhanced target image is enhanced by using a low-light enhancement network and an attention mechanism to obtain an enhanced image.

[0014] Optionally, performing multi-scale illumination enhancement processing on the target image to obtain an enhanced target image includes:

[0015] Using a preset Gaussian filter function, filtering the target image based on at least two preset filter radii to obtain at least two image filtering results;

[0016] Based on the image filtering result, an enhanced target image is obtained.

[0017] Optionally, the target image includes an image under at least one color channel, and the filtering process is performed on the target image based on at least two preset filtering radii using a preset Gaussian filter function to obtain at least two image filtering results, including:

[0018] The image in each color channel is filtered based on at least two preset filter radii using a preset Gaussian filter function to obtain at least two channel filtering results for each color channel.

[0019] Optionally, obtaining an enhanced target image based on the image filtering result includes:

[0020] Performing weighted processing on at least two channel filtering results of each color channel to obtain a channel image of each color channel;

[0021] Based on the channel image corresponding to each color channel, the enhanced target image is obtained.

[0022] Optionally, performing image enhancement on the target image based on an attention mechanism through a low-light enhancement network to obtain an enhanced image includes:

[0023] Sampling the target image through the low-light enhancement network to obtain a first image feature;

[0024] Performing feature extraction on the first image feature based on the attention mechanism to obtain a first attention feature;

[0025] An enhanced image is obtained based on the first attention feature.

[0026] Optionally, sampling the target image through the low-light enhancement network to obtain a first image feature includes:

[0027] Through the low-light enhancement network, a first sampling process is performed on the target image based on a first preset sampling multiple to obtain a first image feature.

[0028] Optionally, obtaining an enhanced image based on the first attention feature includes:

[0029] Performing feature size restoration based on the first preset sampling multiple and the first attention feature to obtain a second image feature that matches the size of the target image;

[0030] An enhanced image is generated based on the second image feature.

[0031] Optionally, the performing feature size restoration based on the first preset sampling multiple and the first attention feature to obtain a second image feature that matches the size of the target image includes:

[0032] Based on a second preset sampling multiple, performing a second sampling process on the first attention feature to obtain a third image feature;

[0033] The feature size of the third image feature is restored based on the first preset sampling multiple and the second preset sampling multiple to obtain a second image feature that matches the size of the target image.

[0034] Optionally, the first image features include at least one first sampling feature obtained during the first sampling process, and before performing feature size restoration on the third image feature based on the first preset sampling multiple and the second preset sampling multiple to obtain a second image feature matching the size of the target image, the method further includes:

[0035] Determining a target sampling feature that matches the third image feature size from the first sampling features;

[0036] The target sampling feature is fused with the third image feature to obtain an updated third image feature.

[0037] Optionally, fusing the target sampling feature with the third image feature to obtain an updated third image feature includes:

[0038] Splicing the target sampling feature with the third image feature to obtain a first splicing feature;

[0039] Feature extraction is performed based on the first splicing feature to obtain an updated third image feature.

[0040] Optionally, the performing feature size restoration on the third image feature based on the first preset sampling multiple and the second preset sampling multiple to obtain a second image feature matching the size of the target image includes:

[0041] performing a first sampling process on the third image feature based on the second preset sampling multiple to obtain a fourth image feature;

[0042] Based on the first preset sampling multiple and the fourth image feature, a second sampling process is performed to obtain a sampling feature whose size matches the size of the target image as the second image feature.

[0043] Optionally, before performing a second sampling process based on the first preset sampling multiple and the fourth image feature to obtain a sampling feature whose size matches the size of the target image as the second image feature, the method further includes:

[0044] Splicing the first attention feature and the fourth image feature to obtain a second splicing feature;

[0045] Feature extraction is performed on the second splicing feature to obtain an updated fourth image feature.

[0046] Optionally, the performing feature extraction on the second splicing feature to obtain an updated fourth image feature includes:

[0047] Based on the attention mechanism, feature extraction is performed on the second splicing feature to obtain a second attention feature as the fourth image feature.

[0048] Optionally, the first sampling process includes one of upsampling and downsampling, the second sampling process includes one of upsampling and downsampling, and the first sampling process and the second sampling process are different.

[0049] Optionally, the training step of the low-light enhancement network includes:

[0050] Obtaining a training sample image and a label image corresponding to the training sample image;

[0051] Performing image enhancement on the training sample image through the low-light enhancement network to obtain a result image;

[0052] Based on the result image and the label image, parameters of the low-light enhancement network are updated.

[0053] Optionally, obtaining a training sample image includes:

[0054] Obtaining an original sample image and a meteorological image corresponding to at least one meteorological scene;

[0055] fusing the meteorological image with the original sample image to obtain an extended sample image;

[0056] A training sample image is obtained based on the original sample image and the expanded sample image.

[0057] Optionally, updating parameters of the low-light enhancement network based on the result image and the label image includes:

[0058] Calculate the difference between the result image and the label image using a preset loss function to obtain a loss value;

[0059] Based on the loss value, parameters of the low-light enhancement network are updated.

[0060] Optionally, the calculating the difference between the result image and the label image using a preset loss function to obtain a loss value includes:

[0061] Calculating the difference between the result image and the label image using a preset loss function to obtain deviation information, structural information difference, and pixel information difference between the result image and the label image;

[0062] The deviation information, the structural information difference and the pixel information difference are weighted by a preset loss function to obtain the loss value.

[0063] Optionally, it also includes:

[0064] Performing color space conversion on the target image to obtain brightness change information corresponding to the target image;

[0065] Based on the brightness change information corresponding to the target image, it is determined whether the environment meets the preset low-light environment condition.

[0066] Optionally, determining whether the environment meets a preset low-light environment condition based on brightness change information corresponding to the target image includes:

[0067] Determining a brightness probability distribution function corresponding to the target image based on brightness change information corresponding to the target image;

[0068] Inputting a preset brightness check value into the brightness probability distribution function to obtain a target brightness probability corresponding to the target image;

[0069] Based on the target brightness probability, it is determined whether the environment meets a preset low-light environment condition.

[0070] Optionally, determining whether the environment meets a preset low-light environment condition based on the target brightness probability includes:

[0071] When the target brightness probability is greater than or equal to a preset probability threshold, determining that the environment meets the preset low-light environment condition;

[0072] When the target brightness probability is less than a preset probability threshold, it is determined that the environment does not meet the preset low-light environment condition.

[0073] According to a second aspect of the present application, there is provided an image processing apparatus, comprising:

[0074] An image enhancement module is used to enhance the target image to obtain an enhanced image;

[0075] The target image is an image captured by the target device for the current environment.

[0076] Optionally, the image enhancement module is specifically configured to:

[0077] When the current environment meets the preset low-light environment conditions, the target image is enhanced based on the attention mechanism through the low-light enhancement network to obtain the enhanced image.

[0078] Optionally, the image enhancement module is specifically configured to:

[0079] Performing multi-scale illumination enhancement processing on the target image to obtain an enhanced target image;

[0080] The enhanced target image is enhanced by using a low-light enhancement network and an attention mechanism to obtain an enhanced image.

[0081] Optionally, the image processing module is specifically configured to:

[0082] Using a preset Gaussian filter function, filtering the target image based on at least two preset filter radii to obtain at least two image filtering results;

[0083] Based on the image filtering result, an enhanced target image is obtained.

[0084] Optionally, the target image includes an image under at least one color channel, and the image processing module is specifically configured to:

[0085] The image in each color channel is filtered based on at least two preset filter radii using a preset Gaussian filter function to obtain at least two channel filtering results for each color channel.

[0086] Optionally, the image processing module is specifically configured to:

[0087] Performing weighted processing on at least two channel filtering results of each color channel to obtain a channel image of each color channel;

[0088] Based on the channel image corresponding to each color channel, the enhanced target image is obtained.

[0089] Optionally, the image enhancement module is specifically configured to:

[0090] Sampling the target image through the low-light enhancement network to obtain a first image feature;

[0091] Performing feature extraction on the first image feature based on the attention mechanism to obtain a first attention feature;

[0092] An enhanced image is obtained based on the first attention feature.

[0093] Optionally, the image enhancement module is specifically configured to:

[0094] Through the low-light enhancement network, a first sampling process is performed on the target image based on a first preset sampling multiple to obtain a first image feature.

[0095] Optionally, the image enhancement module is specifically configured to:

[0096] Performing feature size restoration based on the first preset sampling multiple and the first attention feature to obtain a second image feature that matches the size of the target image;

[0097] An enhanced image is generated based on the second image feature.

[0098] Optionally, the image enhancement module is specifically configured to:

[0099] Based on a second preset sampling multiple, performing a second sampling process on the first attention feature to obtain a third image feature;

[0100] The feature size of the third image feature is restored based on the first preset sampling multiple and the second preset sampling multiple to obtain a second image feature that matches the size of the target image.

[0101] Optionally, the first image feature includes at least one first sampling feature obtained during the first sampling process, and the image processing apparatus may include a first feature updating module, wherein the first feature updating module is specifically configured to:

[0102] Determining a target sampling feature that matches the third image feature size from the first sampling features;

[0103] The target sampling feature is fused with the third image feature to obtain an updated third image feature.

[0104] Optionally, the first feature updating module is specifically configured to:

[0105] Splicing the target sampling feature with the third image feature to obtain a first splicing feature;

[0106] Feature extraction is performed based on the first splicing feature to obtain an updated third image feature.

[0107] Optionally, the image enhancement module is specifically configured to:

[0108] performing a first sampling process on the third image feature based on the second preset sampling multiple to obtain a fourth image feature;

[0109] Based on the first preset sampling multiple and the fourth image feature, a second sampling process is performed to obtain a sampling feature whose size matches the size of the target image as the second image feature.

[0110] Optionally, the image processing apparatus may include a second feature updating module, and the second feature updating module is specifically configured to:

[0111] Splicing the first attention feature and the fourth image feature to obtain a second splicing feature;

[0112] Feature extraction is performed on the second splicing feature to obtain an updated fourth image feature.

[0113] Optionally, the second feature updating module is specifically configured to:

[0114] Based on the attention mechanism, feature extraction is performed on the second splicing feature to obtain a second attention feature as the fourth image feature.

[0115] Optionally, the first sampling process includes one of upsampling and downsampling, the second sampling process includes one of upsampling and downsampling, and the first sampling process and the second sampling process are different.

[0116] Optionally, the image processing apparatus may include a training module, wherein the training module is specifically configured to:

[0117] Obtaining a training sample image and a label image corresponding to the training sample image;

[0118] Performing image enhancement on the training sample image through the low-light enhancement network to obtain a result image;

[0119] Based on the result image and the label image, parameters of the low-light enhancement network are updated.

[0120] Optionally, the training module is specifically used to:

[0121] Obtaining an original sample image and a meteorological image corresponding to at least one meteorological scene;

[0122] fusing the meteorological image with the original sample image to obtain an extended sample image;

[0123] A training sample image is obtained based on the original sample image and the expanded sample image.

[0124] Optionally, the training module is specifically used to:

[0125] Calculate the difference between the result image and the label image using a preset loss function to obtain a loss value;

[0126] Based on the loss value, parameters of the low-light enhancement network are updated.

[0127] Optionally, the training module is specifically used to:

[0128] Calculating the difference between the result image and the label image using a preset loss function to obtain deviation information, structural information difference, and pixel information difference between the result image and the label image;

[0129] The deviation information, the structural information difference and the pixel information difference are weighted by a preset loss function to obtain the loss value.

[0130] Optionally, the image processing apparatus may include a condition judgment module, and the condition judgment module is specifically configured to:

[0131] Performing color space conversion on the target image to obtain brightness change information corresponding to the target image;

[0132] Based on the brightness change information corresponding to the target image, it is determined whether the environment meets the preset low-light environment condition.

[0133] Optionally, the condition judgment module is specifically used to:

[0134] Determining a brightness probability distribution function corresponding to the target image based on brightness change information corresponding to the target image;

[0135] Inputting a preset brightness check value into the brightness probability distribution function to obtain a target brightness probability corresponding to the target image;

[0136] Based on the target brightness probability, it is determined whether the environment meets a preset low-light environment condition.

[0137] Optionally, the condition judgment module is specifically used to:

[0138] When the target brightness probability is greater than or equal to a preset probability threshold, determining that the environment meets the preset low-light environment condition;

[0139] When the target brightness probability is less than a preset probability threshold, it is determined that the environment does not meet the preset low-light environment condition.

[0140] According to a third aspect of the present application, an electronic device is provided, comprising one or more processors and a memory, wherein the memory stores a computer program. When the computer program is executed by the processor, the processor executes any one of the image processing methods provided in the embodiments of the present application.

[0141] According to a fourth aspect of the present application, a computer-readable storage medium is provided, comprising a computer program. When the computer program runs on a controller, the computer program is used to enable the controller to execute any one of the image processing methods provided in the embodiments of the present application.

[0142] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program or instructions, which implement any image processing method provided in the embodiments of the present application when the computer program or instructions are executed by a processor.

[0143] According to a sixth aspect of the present application, a vehicle is provided, comprising an electronic device.

[0144] In the image processing method of the embodiment of the present application, an enhanced image is obtained by performing image enhancement on the target image; wherein the target image is an image captured by the target device for the current environment, and thus the quality of the captured image is improved by performing image enhancement on the target image.

[0145] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0146] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0147] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same drawing numbers represent the same parts in the following description.

[0148] Figure 1 is a flowchart of an image processing method provided in an exemplary embodiment of the present disclosure;

[0149] Figure 2 is a schematic structural diagram of an image processing apparatus provided in an exemplary embodiment of the present disclosure;

[0150] Figure 3 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0151] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0152] The present invention provides an image processing method, device, and system. The image processing device can be integrated into electronic devices such as terminal devices and / or cloud servers. For example, the terminal device can be a vehicle-mounted control terminal, an onboard data processing system, an onboard communication system, a drone controller (such as a handle), a smartphone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smartwatch, and other devices installed in the vehicle.

[0153] In addition, the term "a plurality of" in the embodiments of the present application refers to two or more than two. The terms "first" and "second" in the embodiments of the present application are used to distinguish descriptions and should not be understood to imply relative importance.

[0154] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0155] See also Figure 1 , Figure 1 This is a flowchart of an image processing method provided in one embodiment of the present application. For ease of description, a terminal device is used in the present embodiment of the present application, that is, the image processing method may include:

[0156] S101. Perform image enhancement on a target image to obtain an enhanced image; wherein the target image is an image captured by a target device in a current environment.

[0157] Among them, the above-mentioned target device can be a device that can capture images, and the target device can be set in any scene required by the user. For example, the target device can be a vehicle-mounted camera, which can be set on a vehicle to capture road conditions while the vehicle is driving, or to capture surrounding images when the vehicle is stopped, etc. The specific setting can be made according to needs and is not limited here.

[0158] The vehicle position information may be obtained by detecting the vehicle through positioning by a first positioning device, and the position information of a certain device on the vehicle may be obtained by detecting a certain device on the vehicle through the first positioning device.

[0159] In this embodiment, by performing image enhancement on the target image, the target image can be enhanced, and the key information in the target image can be enhanced. Regardless of whether subsequent processing such as image recognition or target detection is performed based on the enhanced image, the corresponding detection accuracy can be improved.

[0160] It is understood that by performing image enhancement on the target image, the reliability and safety of the autonomous driving system can also be improved. For example, when driving at night or in adverse weather conditions, the accuracy of the visual system directly affects the safety performance of the vehicle. This embodiment can enhance the autonomous driving system's ability to perceive the surrounding environment and reduce the risk of accidents caused by limited vision. Furthermore, it can help improve the all-weather performance of the autonomous driving system. For example, in daytime conditions with ample sunlight, the system may perform relatively well. However, at night or in adverse weather, this embodiment can maintain high image quality under various lighting conditions, ensuring the system's reliable operation in all weather conditions.

[0161] In some embodiments, the image enhancement of the target image to obtain the enhanced image may include: when the current environment meets the preset low-light environment conditions, the target image is enhanced based on the attention mechanism through a low-light enhancement network to obtain the enhanced image.

[0162] It can be understood that by performing low-light enhancement on target images currently in low-light environments, the problem of inaccurate image recognition in dim environments is resolved, effectively improving the performance of the autonomous driving system at night or in adverse weather conditions. Furthermore, the brightness and clarity of the image can be increased, thereby enhancing the details in the image. In dim environments, conventional cameras struggle to capture sufficient light, resulting in reduced image quality and making object recognition difficult. By enhancing light in this embodiment, the environment surrounding the vehicle is more clearly visible, facilitating more accurate identification of key elements such as road signs, pedestrians, and vehicles.

[0163] Specifically, the above-mentioned low-light enhancement network can construct a network model through a residual network to achieve data exchange between different scales, and finally output an enhanced image of the same size as the target image.

[0164] Specifically, the network structure of the above-mentioned low-light enhancement network can be of various types, such as V-type network structure, W-type network structure, M-type network structure, etc., among which downsampling and upsampling processing can be performed in sequence based on the network structure of the low-light enhancement network. The specific setting can be made according to the needs and is not limited here.

[0165] In some embodiments, the image enhancement of the target image based on the attention mechanism through the low-light enhancement network to obtain the enhanced image can include: first, sampling the target image through the low-light enhancement network to obtain a first image feature; then, feature extraction of the first image feature based on the attention mechanism to obtain a first attention feature; finally, obtaining the enhanced image based on the first attention feature.

[0166] The above sampling may be upsampling, downsampling or other sampling methods.

[0167] Exemplarily, the first image feature can be set as feature M1, and the context Transformer module in the low-light enhancement network can be used to extract deep features of feature M1 using the attention mechanism to obtain feature M1', which is the first attention feature.

[0168] Optionally, in this example, the Swin Transformer can be used to locally utilize a multi-head attention mechanism, reducing the number of computational parameters through global parameter sharing and image segmentation, while establishing long-range dependencies between data. The Swin Transformer is a neural network module in the low-light enhancement network.

[0169] Specifically, the above-mentioned process of extracting features of the first image features based on the attention mechanism may be to divide the first image features into multiple data sets, and then use the attention mechanism to find the association between different data sets. Since these data sets are divided according to spatial positions, the association between different data sets can be based on the association of spatial distribution features between the data sets.

[0170] Exemplarily, M1 can be cut into sixteen data sets of equal size, assuming they are numbered 1, 2, 3...16 respectively. Then, the association between the data set numbered 1 and the data set numbered 2 is compared, the association between the data set numbered 1 and the data set numbered 3 is compared, and the association between the data set numbered 1 and the data set numbered 4 is compared, until the relationship between all pairwise relationships of all numbered data sets is compared. Among them, the same weight parameter is used when comparing the data in each two data sets. Data with strong associations are enhanced, and data with weak associations are suppressed. This can be automatically completed by the model according to the weights.

[0171] Specifically, if the above-mentioned sampling is a first sampling process, sampling the target image through the low-light enhancement network to obtain a first image feature may include: performing a first sampling process on the target image based on a first preset sampling multiple through the low-light enhancement network to obtain a first image feature.

[0172] Among them, the above-mentioned first sampling processing can be a sampling method such as upsampling and downsampling, which can be set specifically according to needs and is not limited here. For example, if the above-mentioned first sampling processing is downsampling, the size can be continuously reduced through the convolution module in the low-light enhancement network until it is reduced to a first preset sampling multiple, and the corresponding first image features are obtained to increase the depth level features.

[0173] For example, if the target image is set to image M0 and the first sampling process is downsampling, the image M0 can be continuously downsampled through the convolution module in the low-light enhancement network to continuously reduce the size of the image M0 until the size of the image M0 is reduced by k1 times, and the extracted feature M1 is obtained, which is the first image feature.

[0174] Specifically, the process of performing the first sampling processing on the target image based on the first preset sampling multiple can generally use a residual neural network ResNet18 or ResNet50 for feature extraction. For example, using the residual neural network ResNet18 or ResNet50 can reduce the size of the target image by 32 times.

[0175] Among them, the residual neural networks ResNet18 and ResNet50 have similar overall structures, but the different number of repeatable modules leads to different depths of the convolutional networks. At the same time, compared with ResNet34, ResNet50 can improve the network feature extraction capability by ensuring that the number of parameters is not increased in the bottleneck mode. Therefore, in this embodiment, ResNet18 or ResNet50 can be selected for feature extraction during the first sampling process.

[0176] Furthermore, obtaining an enhanced image based on the first attention feature may include: first, restoring the feature size based on the first preset sampling multiple and the first attention feature to obtain a second image feature that matches the size of the target image; and then, generating an enhanced image based on the second image feature.

[0177] It can be understood that if the low-light enhancement network is a V-shaped structure, then after obtaining the first attention mechanism, the size of the first attention feature can be directly restored to a second image feature that matches the size of the target image, so as to obtain an enhanced image based on the second image feature, that is, the enhanced image is obtained by restoring after one sampling; and if the low-light enhancement network is another type of network structure, such as a W-type network structure, then the first attention feature needs to be reprocessed, so that after the processing of the first attention feature conforms to the corresponding network structure, the processed feature is restored to its size, so as to restore it to a second image feature that matches the size of the target image, so as to obtain an enhanced image based on the second image feature, that is, the enhanced image is obtained by restoring after at least two samplings.

[0178] In some embodiments, if the low-light enhancement network is other non-V-type network structures, such as a W-type network structure, then due to the presence of multiple sampling, it is necessary to enter another preset sampling multiple to process the first attention feature, that is, the second preset sampling multiple. The second preset sampling multiple can be set according to demand and is not limited here. For example, the second preset sampling multiple can be set to 2 times, and there is no relationship between the second preset sampling multiple and the first preset sampling multiple.

[0179] Specifically, the feature size restoration based on the first preset sampling multiple and the first attention feature to obtain a second image feature that matches the size of the target image may include: performing a second sampling process on the first attention feature based on the second preset sampling multiple to obtain a third image feature; and performing feature size restoration on the third image feature based on the first preset sampling multiple and the second preset sampling multiple to obtain a second image feature that matches the size of the target image.

[0180] Exemplarily, the first attention feature can be set as feature M1', and feature M1' can be restored to a limited extent, that is, feature M1' is upsampled to obtain feature M2 based on a second preset sampling multiple of k2. Feature M2 is the third image feature, and the restoration process is to restore the small-scale feature M1' to the large-scale feature M2.

[0181] In this embodiment, a deconvolution module may be used to implement the second sampling process. At the same time, after each deconvolution step, several scale-invariant convolution operations will be performed to ensure data stability.

[0182] Specifically, because the first sampling process does not directly reduce the target image to a size matching the first preset sampling factor, but instead involves multiple reductions until the target image is reduced to a size matching the first preset sampling factor, and each reduction generates a first sampling feature of the corresponding size, until at least one first sampling feature is obtained at the first preset sampling factor, the first image features include the at least one first sampling feature obtained during the first sampling process. Furthermore, the second sampling process is identical to the first sampling process, requiring multiple sampling steps.

[0183] Accordingly, before restoring the feature size of the third image feature based on the first preset sampling multiple and the second preset sampling multiple to obtain the second image feature that matches the size of the target image, it may also include: first, determining a target sampling feature that matches the size of the third image feature from the first sampling feature, and then fusing the target sampling feature with the third image feature to obtain an updated third image feature.

[0184] It can be understood that since multiple first sampling features corresponding to different sizes will be generated during the first sampling processing, the sampling features that match the third image feature size can be fused with the third image features to enable the fused data to contain information of different levels of abstraction, and the two types of information with different levels of abstraction can be fused to enhance the richness of the third image features.

[0185] Exemplarily, the third image feature may be set as feature M2, and the feature M2 is fused with the target sampling feature to obtain feature M2', which is the updated third image feature.

[0186] Furthermore, the fusing of the target sampling feature with the third image feature to obtain an updated third image feature may include: first, splicing the target sampling feature with the third image feature to obtain a first splicing feature, and then performing feature extraction based on the first splicing feature to obtain an updated third image feature.

[0187] Specifically, the feature size restoration of the third image feature based on the first preset sampling multiple and the second preset sampling multiple to obtain a second image feature that matches the size of the target image includes: performing a first sampling process on the third image feature based on the second preset sampling multiple to obtain a fourth image feature; and performing a second sampling process based on the first preset sampling multiple and the fourth image feature to obtain a sampling feature whose size matches the size of the target image as the second image feature.

[0188] The size matching with the target image size may be achieved by maintaining consistency in the sampling multiple. For example, if sampling is performed at a first preset sampling multiple, the size restoration must be performed based on the first preset sampling multiple.

[0189] Exemplarily, the third image feature can be set as feature M2', the first preset sampling multiple can be set as k1, the second preset sampling multiple can be set as k2, and the first sampling process can be performed on feature M2' based on k2 to obtain feature M3, which is the fourth image feature. The second sampling process can then be performed on M3 based on k1 to obtain a sampling feature whose size matches the size of the target image.

[0190] Specifically, in order to further improve the feature richness of the fourth image feature and improve the quality of image enhancement, the second sampling process is performed based on the first preset sampling multiple and the fourth image feature to obtain a sampling feature whose size matches the size of the target image. Before it is used as the second image feature, it may also include: first, splicing the first attention feature with the fourth image feature to obtain a second splicing feature; then, performing feature extraction on the second splicing feature to obtain an updated fourth image feature, so that the updated fourth image feature can include an understanding of the original first attention feature of the target image, and can better identify the impact of light on the image.

[0191] Exemplarily, the feature M3 is concatenated with the feature M1′ to obtain the feature M3′, which is the updated fourth image feature.

[0192] It's understandable that this embodiment uses deconvolution to restore data to an image. To ensure that image details are preserved while enhancing low-light quality during the restoration process, each intermediate variable generated is fused with the features generated in the previous step, so that the newly generated data contains both an abstract understanding of the image and an understanding of its texture and edges. Each step is similar to ResNet18, but the downsampling in each step is replaced by upsampling through the deconvolution module.

[0193] Specifically, the extracting features from the second splicing feature to obtain the updated fourth image feature includes: extracting features from the second splicing feature based on an attention mechanism to obtain a second attention feature as the fourth image feature.

[0194] Specifically, the first sampling process includes one of upsampling and downsampling, the second sampling process includes one of upsampling and downsampling, and the first sampling process and the second sampling process are different.

[0195] In some embodiments, after obtaining the target image taken by the target device for the current environment, the target image can also be detected to determine whether the current environment in which the target device is located is a low-light environment, so as to decide subsequent processing based on the judgment result of whether the environment is a low-light environment.

[0196] Specifically, the terminal device can perform color space conversion on the target image to obtain brightness change information corresponding to the target image, and determine whether the environment meets the preset low-light environment conditions based on the brightness change information corresponding to the target image. For example, the ambient brightness corresponding to the target image can be compared with a preset threshold. If the ambient brightness is lower than or equal to the preset brightness threshold, it means that the environment meets the preset low-light environment conditions. Alternatively, the probability information of the ambient brightness corresponding to the target object can be compared with the preset probability threshold. If the probability information is lower than or equal to the preset probability threshold, it means that the environment meets the preset low-light environment conditions, etc. The specific settings can be made according to needs and are not limited here.

[0197] Specifically, the above-mentioned color space conversion of the target image may include: the terminal device may convert the color space of the target image from the RGB color space to the HSV color space. Accordingly, after converting to the HSV color space, a brightness histogram can be obtained by counting the brightness components in the HSV color space. The brightness histogram can indicate the brightness change information of the target image. The brightness histogram includes the brightness component of each pixel in the image. If it is desired to determine whether the entire image is in a low-light environment, it is necessary to determine the image brightness level, that is, the brightness change information of the target image can be counted using a histogram. In addition, since the target image needs to be processed later, after determining whether the current environment of the target device is a low-light environment, the color space of the target image needs to be converted from the HSV color space back to the RGB color space.

[0198] It should be noted that in the HSV color space, H represents hue, which can be expressed as the difference between the color with the lowest brightness and the color with the lowest brightness. S represents saturation, which is the degree of color variation in a single image. V represents brightness. The brightness represented by V refers to the human eye's visual perception of changes in light and dark. It can store the color with the highest brightness among the three primary colors of RGB. Therefore, the brightness change of the corresponding target image can be judged by the high or low value of V.

[0199] It is understandable that although the RGB color space and the HSV color space represent different data storage methods, they can be converted to each other. The conversion formula between the two is as follows:

[0200] V=max

[0201]

[0202] Among them, the above max=max(R, G, B) is used to indicate the maximum value in RGB; the above min=min(R, G, B) is used to indicate the minimum value in RGB; if the above H is less than 0, then H=H+360.

[0203] It should be noted that since only V is used in the process of determining whether the current environment of the target device is a low-light environment, and the calculation of V does not involve the calculation of saturation and hue, at the same time, after determining whether the current environment of the target device is a low-light environment, the color space of the target image needs to be converted from the HSV color space back to the RGB color space. Therefore, V can be calculated separately to simplify the amount of calculation.

[0204] Optionally, in this embodiment, the image data in the RGB color space may be retained, so that there is no need to convert the image from the HSV color space back to the RGB color space later.

[0205] For example, assuming the input image is M, the histogram can be defined as:

[0206]

[0207] Among them, P(k) is the probability of the k-th brightness level appearing in the input image, n is the total number of image pixels, and n k Indicates the number of pixels of the kth level brightness. In this embodiment, I represents the brightness interval. Since the brightness is determined by the highest value of the RGB component in the RGB color space, I=256, which means that the brightness is divided into 256 levels in total.

[0208] Specifically, determining whether the environment meets the preset low-light environment conditions based on the brightness change information corresponding to the target image may include: first, determining the brightness probability distribution function corresponding to the target image based on the brightness change information corresponding to the target image; then, inputting a preset brightness check value into the brightness probability distribution function to obtain the target brightness probability corresponding to the target image; finally, determining whether the environment meets the preset low-light environment conditions based on the target brightness probability.

[0209] In this embodiment, based on the brightness variation information collected through the brightness histogram, the probability distribution of the brightness in the image can be clarified to determine the brightness probability distribution function.

[0210] For example, the probability distribution function is expressed as follows:

[0211]

[0212] Wherein, the above F(X) is a probability distribution function.

[0213] Specifically, determining whether the environment meets the preset low-light environment conditions based on the target brightness probability may include: comparing the target brightness probability with a preset probability threshold to obtain a corresponding comparison result, that is, when the target brightness probability is greater than or equal to the preset probability threshold, determining that the environment meets the preset low-light environment conditions; when the target brightness probability is less than the preset probability threshold, determining that the environment does not meet the preset low-light environment conditions.

[0214] It can be understood that the above-mentioned preset probability threshold can represent a significance level.

[0215] It can be understood that by determining whether the preset brightness check value is less than δ when the preset probability threshold is γ, if it is less than, it indicates that the environment meets the preset low-light environment conditions. If it is greater than or equal to, it indicates that the environment is relatively bright and the image information itself can meet the subsequent detection tasks under natural lighting, and no further processing is required. Where γ is a number between 0 and 1, and δ represents the desired brightness.

[0216] For example, after determining the probability distribution function of brightness, a preset probability threshold γ and a preset brightness check value δ are determined based on experience. If F(δ) is less than γ, it can be directly used in subsequent calculations. If F(δ) is greater than or equal to γ, it can be considered that the environment meets the preset low-light environment conditions, that is, the image brightness is low, resulting in unclear key information, and image enhancement operations are required.

[0217] The preset probability threshold and the preset brightness check value δ can be optimized and set based on the experimental results in historical experiments to determine the specific values of the preset probability threshold and the preset brightness check value.

[0218] In some embodiments, before performing image enhancement on the target image based on the attention mechanism through the low-light enhancement network to obtain the enhanced image, it can also include: performing multi-scale illumination enhancement processing on the target image to obtain the enhanced target image, so as to perform multi-scale illumination enhancement processing on the target image before processing the target image using the attention mechanism to improve the image quality of the target image.

[0219] Accordingly, the method of performing image enhancement on the target image based on the attention mechanism through the low-light enhancement network to obtain an enhanced image may include: performing image enhancement on the enhanced target image based on the attention mechanism through the low-light enhancement network to obtain an enhanced image.

[0220] Specifically, a multi-scale illumination enhancement process may be performed on the target image by using a filter function to obtain an enhanced target image, wherein the filter function may include a Gaussian filter function.

[0221] In some embodiments, performing multi-scale illumination enhancement processing on the target image to obtain an enhanced target image may include: first, filtering the target image based on at least two preset filtering radii using a preset Gaussian filtering function to obtain at least two image filtering results, and then obtaining an enhanced target image based on the image filtering results.

[0222] In this embodiment, a plurality of filter radii of different scales are selected to implement multi-scale illumination enhancement processing on the target image to obtain an enhanced target image.

[0223] It should be noted that the setting of the preset filter radius can be set according to actual conditions and is not limited here. For example, 0.01, 0.02, or 0.03 times the preset image size can be selected as the filter radius of the Gaussian filter function.

[0224] Optionally, the filtering process can be implemented using a multi-scale MSR algorithm (Multi-Scale Retinex). The MSR algorithm is a multi-scale SSR (Singal Scale Retinex) algorithm. The SSR algorithm is a single-scale Retinex operation, where the retinex is a compound word, retina and cortex. The MSR algorithm is a multi-scale Retinex operation, so filtering can be performed by modifying the scale based on the SSR algorithm.

[0225] The SSR algorithm calculates the reflectance component of the target image, which carries image detail information, using the ambient light's luminance component and the target image's color information. The color information is directly obtained from the image, while the luminance component is obtained by Gaussian blurring the image. Furthermore, the Gaussian blur function in the SSR algorithm convolves the target image with a normal distribution function, where λ represents the normalized scale and c represents the Gaussian surround scale.

[0226] Specifically, the target image includes an image under at least one color channel, and the filtering process is performed on the target image based on at least two preset filtering radii using a preset Gaussian filtering function to obtain at least two image filtering results. This may include: filtering the image under each color channel based on at least two preset filtering radii using a preset Gaussian filtering function to obtain at least two channel filtering results for each color channel.

[0227] The image under at least one color channel may be an image of a meteorological image superimposed with a meteorological scene.

[0228] Furthermore, obtaining an enhanced target image based on the image filtering results may include: first, performing weighted processing on at least two channel filtering results of each color channel to obtain a channel image of each color channel, and then obtaining an enhanced target image based on the channel image corresponding to each color channel.

[0229] For example, the mathematical expression of the above SSR algorithm is as follows:

[0230] S(x, y) = L(x, y) × R(x, y)

[0231] Among them, S(x,y) represents the color information of the received target image, L(x,y) represents the brightness component of the ambient light, R(x,y) represents the reflection component of the target object carrying image detail information, and x and y represent the position of the pixel in the image.

[0232] Then, taking the logarithms of both ends of the mathematical expression for the filtering process and performing the transformation yields:

[0233] logR(x,y)=losS(x,y)-logL(x,y)

[0234] Among them, L(x,y) can be obtained by Gaussian blurring the image information S(x,y). Therefore, the formula can be further processed as shown below:

[0235] logR(x,y)=logS(x,y)-logL(x,y)=logS(x,y)-log[F(x,y)*S(x,y)]

[0236] Among them, the Gaussian blur function F(x,y) in the SSR algorithm is:

[0237]

[0238] Where c represents the Gaussian surround scale, and λ is the normalized scale, ensuring that ∫∫F(x, y)dxdy = 1. The corresponding multi-scale SSR algorithm is expressed as selecting different Gaussian blur radii c, calculating multiple reflection information components, and performing weighted summation to obtain the following formula:

[0239]

[0240] Among them, S i (x,y) represents the color information of the original image, F n (x, y) represents a Gaussian filter function, i represents different color channels, N represents the number of scales, that is, the number of the above-mentioned filter radii, and ω represents the weight of different scales. In this embodiment, 1 / N is selected.

[0241] In this example, three-layer filtering can be used. Therefore, the above N=3, ω=1 / 3, n=1, 2, 3 are respectively substituted into the function calculation to obtain three results, and then the three results are added together to obtain R_MSRi.

[0242] As can be seen from the above, by performing image enhancement on the target image, an enhanced image is obtained; wherein, the target image is an image captured by the target device for the current environment, and thus by performing image enhancement on the target image, the quality of the captured image is improved.

[0243] An embodiment of the present application provides a training step for a low-light enhancement network, which may include: obtaining a training sample image and a label image corresponding to the training sample image; performing image enhancement on the training sample image through the low-light enhancement network to obtain a result image; and updating the parameters of the low-light enhancement network based on the result image and the label image.

[0244] Among them, the processing process of performing image enhancement on the training sample image through the low-light enhancement network to obtain the result image can refer to the specific processing process of the step of performing image enhancement on the target image through the low-light enhancement network to obtain the enhanced image in the above embodiment.

[0245] Specifically, the obtaining of training sample images includes: obtaining an original sample image and a meteorological image corresponding to at least one meteorological scene; obtaining an extended sample image based on the fusion of the meteorological image and the original sample image; and obtaining a training sample image based on the original sample image and the extended sample image.

[0246] As you can understand, to further expand the data sample size, this example uses data fusion to increase the sample size. For example, the meteorological image can be a smoke image, which can be pure smoke data (without background) synthesized by software. These smoke images are combined with existing images to form a new dataset. In actual applications, other data such as raindrops can also be added to increase the number of training samples.

[0247] The pure smoke image mentioned above refers to a smoke image generated by Blender that contains transparency information and lacks a background. When the pure smoke data is synthesized with the existing image, an image approximating a vehicle driving in a foggy scene can be obtained. The data before synthesis does not contain images with similar features. The core purpose of this step is to increase the number of samples and expand the data capacity.

[0248] In some embodiments, the parameter updating of the low-light enhancement network based on the result image and the label image includes: first, calculating the difference between the result image and the label image through a preset loss function to obtain a loss value; then, based on the loss value, updating the parameters of the low-light enhancement network.

[0249] Specifically, the difference calculation between the result image and the label image is performed through a preset loss function to obtain a loss value, which may include: first, the difference calculation between the result image and the label image is performed through a preset loss function to obtain the deviation information, structural information difference and pixel information difference between the result image and the label image; then, the deviation information, the structural information difference and the pixel information difference are weighted by a preset loss function to obtain the loss value.

[0250] For example, the loss function is expressed as follows:

[0251] L loss =λ1×L1+λ SSIM ×L SSM +λ per ×L per

[0252] Among them, λ1,λ SSIM ,λ per They all represent the weights of different loss functions.

[0253] Among them, L1 calculates the minimum absolute deviation, that is, the above deviation information, whose purpose is to minimize the sum of the absolute value differences between the target value and the estimated value. L1 can be expressed as:

[0254]

[0255] Among them, y represents the true image, x represents the predicted image, and N represents the total number of samples.

[0256] Among them, L SSIM The difference in structural information between the generated image and the real image is calculated, that is, the above structural information difference. This loss is intended to improve the visual effect of the output truth. L SSIM It can be expressed as:

[0257]

[0258] Where μx, μy represent the pixel averages of the generated image and the real image respectively, σxy represents the covariance between x and y, and σy 2 and σy 2 They represent the variance of the generated image and the real image respectively, N represents the total number of samples, and C1 and C2 are constants.

[0259] Among them, the perceptual loss L per In order to better restore the image details, that is, the above pixel information difference, by pulling the distance of the L2 paradigm into the feature level, L per It can be expressed as:

[0260]

[0261] Where x and y represent the generated image and the real image, respectively. φ(·) represents the i-th feature map of the image after the pre-training model. C represents the number of channels. H and W represent the height and width of the feature map.

[0262] To facilitate better implementation of the image processing method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the above image processing method. The meanings of the terms herein are the same as those in the above image processing method, and the specific implementation details can be referred to the description in the method embodiment.

[0263] For example, Figure 2 As shown, the image processing device may include:

[0264] The image enhancement module 201 is used to perform image enhancement on the target image to obtain an enhanced image;

[0265] The target image is an image captured by the target device for the current environment.

[0266] Optionally, the image enhancement module 201 is specifically configured to:

[0267] When the current environment meets the preset low-light environment conditions, the target image is enhanced based on the attention mechanism through the low-light enhancement network to obtain the enhanced image.

[0268] Optionally, the image enhancement module 201 is specifically configured to:

[0269] Performing multi-scale illumination enhancement processing on the target image to obtain an enhanced target image;

[0270] The enhanced target image is enhanced by using a low-light enhancement network and an attention mechanism to obtain an enhanced image.

[0271] Optionally, the image processing module is specifically configured to:

[0272] Using a preset Gaussian filter function, filtering the target image based on at least two preset filter radii to obtain at least two image filtering results;

[0273] Based on the image filtering result, an enhanced target image is obtained.

[0274] Optionally, the target image includes an image under at least one color channel, and the image processing module is specifically configured to:

[0275] The image in each color channel is filtered based on at least two preset filter radii using a preset Gaussian filter function to obtain at least two channel filtering results for each color channel.

[0276] Optionally, the image processing module is specifically configured to:

[0277] Performing weighted processing on at least two channel filtering results of each color channel to obtain a channel image of each color channel;

[0278] Based on the channel image corresponding to each color channel, the enhanced target image is obtained.

[0279] Optionally, the image enhancement module 201 is specifically configured to:

[0280] Sampling the target image through the low-light enhancement network to obtain a first image feature;

[0281] Performing feature extraction on the first image feature based on the attention mechanism to obtain a first attention feature;

[0282] An enhanced image is obtained based on the first attention feature.

[0283] Optionally, the image enhancement module 201 is specifically configured to:

[0284] Through the low-light enhancement network, a first sampling process is performed on the target image based on a first preset sampling multiple to obtain a first image feature.

[0285] Optionally, the image enhancement module 201 is specifically configured to:

[0286] Performing feature size restoration based on the first preset sampling multiple and the first attention feature to obtain a second image feature that matches the size of the target image;

[0287] An enhanced image is generated based on the second image feature.

[0288] Optionally, the image enhancement module 201 is specifically configured to:

[0289] Based on a second preset sampling multiple, performing a second sampling process on the first attention feature to obtain a third image feature;

[0290] The feature size of the third image feature is restored based on the first preset sampling multiple and the second preset sampling multiple to obtain a second image feature that matches the size of the target image.

[0291] Optionally, the first image feature includes at least one first sampling feature obtained during the first sampling process, and the image processing apparatus may include a first feature updating module, wherein the first feature updating module is specifically configured to:

[0292] Determining a target sampling feature that matches the third image feature size from the first sampling features;

[0293] The target sampling feature is fused with the third image feature to obtain an updated third image feature.

[0294] Optionally, the first feature updating module is specifically configured to:

[0295] Splicing the target sampling feature with the third image feature to obtain a first splicing feature;

[0296] Feature extraction is performed based on the first splicing feature to obtain an updated third image feature.

[0297] Optionally, the image enhancement module 201 is specifically configured to:

[0298] performing a first sampling process on the third image feature based on the second preset sampling multiple to obtain a fourth image feature;

[0299] Based on the first preset sampling multiple and the fourth image feature, a second sampling process is performed to obtain a sampling feature whose size matches the size of the target image as the second image feature.

[0300] Optionally, the image processing apparatus may include a second feature updating module, and the second feature updating module is specifically configured to:

[0301] Splicing the first attention feature and the fourth image feature to obtain a second splicing feature;

[0302] Feature extraction is performed on the second splicing feature to obtain an updated fourth image feature.

[0303] Optionally, the second feature updating module is specifically configured to:

[0304] Based on the attention mechanism, feature extraction is performed on the second splicing feature to obtain a second attention feature as the fourth image feature.

[0305] Optionally, the first sampling process includes one of upsampling and downsampling, the second sampling process includes one of upsampling and downsampling, and the first sampling process and the second sampling process are different.

[0306] Optionally, the image processing apparatus may include a training module, wherein the training module is specifically configured to:

[0307] Obtaining a training sample image and a label image corresponding to the training sample image;

[0308] Performing image enhancement on the training sample image through the low-light enhancement network to obtain a result image;

[0309] Based on the result image and the label image, parameters of the low-light enhancement network are updated.

[0310] Optionally, the training module is specifically used to:

[0311] Obtaining an original sample image and a meteorological image corresponding to at least one meteorological scene;

[0312] fusing the meteorological image with the original sample image to obtain an extended sample image;

[0313] A training sample image is obtained based on the original sample image and the expanded sample image.

[0314] Optionally, the training module is specifically used to:

[0315] Calculate the difference between the result image and the label image using a preset loss function to obtain a loss value;

[0316] Based on the loss value, parameters of the low-light enhancement network are updated.

[0317] Optionally, the training module is specifically used to:

[0318] Calculating the difference between the result image and the label image using a preset loss function to obtain deviation information, structural information difference, and pixel information difference between the result image and the label image;

[0319] The deviation information, the structural information difference and the pixel information difference are weighted by a preset loss function to obtain the loss value.

[0320] Optionally, the image processing apparatus may include a condition judgment module, and the condition judgment module is specifically configured to:

[0321] Performing color space conversion on the target image to obtain brightness change information corresponding to the target image;

[0322] Based on the brightness change information corresponding to the target image, it is determined whether the environment meets the preset low-light environment condition.

[0323] Optionally, the condition judgment module is specifically used to:

[0324] Determining a brightness probability distribution function corresponding to the target image based on brightness change information corresponding to the target image;

[0325] Inputting a preset brightness check value into the brightness probability distribution function to obtain a target brightness probability corresponding to the target image;

[0326] Based on the target brightness probability, it is determined whether the environment meets a preset low-light environment condition.

[0327] Optionally, the condition judgment module is specifically used to:

[0328] When the target brightness probability is greater than or equal to a preset probability threshold, determining that the environment meets the preset low-light environment condition;

[0329] When the target brightness probability is less than a preset probability threshold, it is determined that the environment does not meet the preset low-light environment condition.

[0330] The image processing device proposed in this application obtains an enhanced image by performing image enhancement on the target image; wherein the target image is an image captured by the target device for the current environment, thereby improving the quality of the captured image by performing image enhancement on the target image.

[0331] During specific implementation, the above modules can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. The specific implementation methods and corresponding beneficial effects of the above modules can be found in the previous method embodiments and will not be repeated here.

[0332] The present application also provides an electronic device, such as Figure 3 , which shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0333] The electronic device may include one or more processors 301 of processing cores, one or more computer-readable storage media memories 302, a power supply 303, an input unit 304 and other components. Those skilled in the art will appreciate that Figure 3 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0334] The processor 301 is the control center of the electronic device. It connects all parts of the electronic device using various interfaces and circuits. It executes the various functions of the electronic device and processes data by running or executing computer programs and / or modules stored in the memory 302 and accessing data stored in the memory 302. Optionally, the processor 301 may include one or more processing cores. Preferably, the processor 301 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 301.

[0335] The memory 302 can be used to store computer programs and modules. The processor 301 executes various functional applications and data processing by running the computer programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0336] The electronic device also includes a power supply 303 for supplying power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 303 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0337] The electronic device may further include an input unit 304, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0338] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the electronic device will load the executable files corresponding to one or more computer program processes into the memory 302 according to the following instructions, and the processor 301 will run the computer programs stored in the memory 302 to implement various functions, such as:

[0339] Perform image enhancement on the target image to obtain an enhanced image;

[0340] The target image is an image captured by the target device for the current environment.

[0341] The specific implementation methods and corresponding beneficial effects of the above operations can be found in the detailed description of the image processing method above, which will not be elaborated here.

[0342] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0343] To this end, an embodiment of the present application provides a computer-readable storage medium storing a computer program that can be loaded by a processor to execute the steps of any of the image processing methods provided in the embodiments of the present application. For example, the computer program can execute the following steps:

[0344] Perform image enhancement on the target image to obtain an enhanced image;

[0345] The target image is an image captured by the target device for the current environment.

[0346] The specific implementation methods and corresponding beneficial effects of the above operations can be found in the previous embodiments and will not be described in detail here.

[0347] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0348] Since the computer program stored in the computer-readable storage medium can execute the steps of any image processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any image processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0349] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above-described image processing method.

[0350] According to one aspect of the present application, a vehicle is provided, which includes the above-mentioned electronic device.

[0351] The above is a detailed introduction to an image processing method, device, equipment, medium, product and vehicle provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Performing image enhancement on the target image to obtain an enhanced image; The target image is an image captured by the target device for the current environment.

2. The image processing method according to claim 1, wherein: The step of performing image enhancement on the target image to obtain an enhanced image includes: When the current environment meets the preset low-light environment conditions, the target image is enhanced based on the attention mechanism through the low-light enhancement network to obtain the enhanced image.

3. The image processing method according to claim 2, wherein: Before the target image is enhanced by the low-light enhancement network based on the attention mechanism to obtain the enhanced image, the following steps are also included: Performing multi-scale illumination enhancement processing on the target image to obtain an enhanced target image; The low-light enhancement network is used to enhance the target image based on the attention mechanism to obtain an enhanced image, including: The enhanced target image is enhanced by using a low-light enhancement network and an attention mechanism to obtain an enhanced image.

4. The image processing method according to claim 3, wherein: The performing multi-scale illumination enhancement processing on the target image to obtain an enhanced target image includes: Using a preset Gaussian filter function, filtering the target image based on at least two preset filter radii to obtain at least two image filtering results; Based on the image filtering result, an enhanced target image is obtained.

5. The image processing method according to claim 4, characterized in that The target image includes an image under at least one color channel, and the filtering process is performed on the target image based on at least two preset filtering radii using a preset Gaussian filtering function to obtain at least two image filtering results, including: The image in each color channel is filtered based on at least two preset filter radii using a preset Gaussian filter function to obtain at least two channel filtering results for each color channel.

6. The image processing method according to claim 5, characterized in that Obtaining an enhanced target image based on the image filtering result includes: Performing weighted processing on at least two channel filtering results of each color channel to obtain a channel image of each color channel; Based on the channel image corresponding to each color channel, the enhanced target image is obtained.

7. The image processing method according to claim 2, wherein: The low-light enhancement network is used to enhance the target image based on the attention mechanism to obtain an enhanced image, including: Sampling the target image through the low-light enhancement network to obtain a first image feature; Performing feature extraction on the first image feature based on the attention mechanism to obtain a first attention feature; An enhanced image is obtained based on the first attention feature.

8. The image processing method according to claim 7, wherein: The sampling of the target image by the low-light enhancement network to obtain a first image feature includes: Through the low-light enhancement network, a first sampling process is performed on the target image based on a first preset sampling multiple to obtain a first image feature.

9. The image processing method according to claim 8, characterized in that: The obtaining an enhanced image based on the first attention feature includes: Performing feature size restoration based on the first preset sampling multiple and the first attention feature to obtain a second image feature that matches the size of the target image; An enhanced image is generated based on the second image feature.

10. The image processing method according to claim 9, wherein: The performing feature size restoration based on the first preset sampling multiple and the first attention feature to obtain a second image feature that matches the size of the target image includes: Based on a second preset sampling multiple, performing a second sampling process on the first attention feature to obtain a third image feature; The feature size of the third image feature is restored based on the first preset sampling multiple and the second preset sampling multiple to obtain a second image feature that matches the size of the target image.

11. The image processing method according to claim 10, wherein: The first image features include at least one first sampling feature obtained in the first sampling process. Before restoring the feature size of the third image feature based on the first preset sampling multiple and the second preset sampling multiple to obtain a second image feature matching the size of the target image, the method further includes: Determining a target sampling feature that matches the third image feature size from the first sampling features; The target sampling feature is fused with the third image feature to obtain an updated third image feature.

12. The image processing method according to claim 11, wherein: The fusing the target sampling feature with the third image feature to obtain an updated third image feature includes: Splicing the target sampling feature with the third image feature to obtain a first splicing feature; Feature extraction is performed based on the first splicing feature to obtain an updated third image feature.

13. The image processing method according to claim 10, wherein: The performing feature size restoration on the third image feature based on the first preset sampling multiple and the second preset sampling multiple to obtain a second image feature matching the size of the target image includes: performing a first sampling process on the third image feature based on the second preset sampling multiple to obtain a fourth image feature; Based on the first preset sampling multiple and the fourth image feature, a second sampling process is performed to obtain a sampling feature whose size matches the size of the target image as the second image feature.

14. The image processing method according to claim 13, wherein: Before performing a second sampling process based on the first preset sampling multiple and the fourth image feature to obtain a sampling feature whose size matches the size of the target image as the second image feature, the method further includes: Splicing the first attention feature and the fourth image feature to obtain a second splicing feature; Feature extraction is performed on the second splicing feature to obtain an updated fourth image feature.

15. The image processing method according to claim 14, characterized in that: The extracting the second stitching feature to obtain an updated fourth image feature includes: Based on the attention mechanism, feature extraction is performed on the second splicing feature to obtain a second attention feature as the fourth image feature.

16. The image processing method according to claim 10, wherein: The first sampling process includes one of upsampling and downsampling, the second sampling process includes one of upsampling and downsampling, and the first sampling process and the second sampling process are different.

17. The image processing method according to claim 2, wherein: The training steps of the low-light enhancement network include: Obtaining a training sample image and a label image corresponding to the training sample image; Performing image enhancement on the training sample image through the low-light enhancement network to obtain a result image; Based on the result image and the label image, parameters of the low-light enhancement network are updated.

18. The image processing method according to claim 17, wherein: The obtaining of the training sample image comprises: Obtaining an original sample image and a meteorological image corresponding to at least one meteorological scene; fusing the meteorological image with the original sample image to obtain an extended sample image; A training sample image is obtained based on the original sample image and the expanded sample image.

19. The image processing method according to claim 17, wherein: The updating of parameters of the low-light enhancement network based on the result image and the label image includes: Calculate the difference between the result image and the label image using a preset loss function to obtain a loss value; Based on the loss value, parameters of the low-light enhancement network are updated.

20. The image processing method according to claim 19, wherein: The difference between the result image and the label image is calculated by using a preset loss function to obtain a loss value, including: Calculating the difference between the result image and the label image using a preset loss function to obtain deviation information, structural information difference, and pixel information difference between the result image and the label image; The deviation information, the structural information difference and the pixel information difference are weighted by a preset loss function to obtain the loss value.

21. The image processing method according to claim 2, wherein: Also includes: Performing color space conversion on the target image to obtain brightness change information corresponding to the target image; Based on the brightness change information corresponding to the target image, it is determined whether the environment meets the preset low-light environment condition.

22. The image processing method according to claim 21, characterized in that: The determining whether the environment meets the preset low-light environment condition based on the brightness change information corresponding to the target image includes: Determining a brightness probability distribution function corresponding to the target image based on brightness change information corresponding to the target image; Inputting a preset brightness check value into the brightness probability distribution function to obtain a target brightness probability corresponding to the target image; Based on the target brightness probability, it is determined whether the environment meets a preset low-light environment condition.

23. The image processing method according to claim 22, characterized in that: The determining, based on the target brightness probability, whether the environment meets a preset low-light environment condition includes: When the target brightness probability is greater than or equal to a preset probability threshold, determining that the environment meets the preset low-light environment condition; When the target brightness probability is less than a preset probability threshold, it is determined that the environment does not meet the preset low-light environment condition.

24. An image processing device, characterized in that The device comprises: An image enhancement module is used to enhance the target image to obtain an enhanced image; The target image is an image captured by the target device for the current environment.

25. An electronic device, characterized in that: The image processing apparatus comprises one or more processors and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the image processing method according to any one of claims 1 to 23.

26. A storage medium, characterized in that The invention comprises a computer program, which is used to cause the controller to perform the steps of the image processing method according to any one of claims 1 to 23 when the computer program is run on the controller.

27. A computer program product, characterized in that The method comprises a computer program or instructions, which implements the steps of the image processing method according to any one of claims 1 to 23 when the computer program or instructions are executed by a processor.

28. A vehicle, characterized in that: The vehicle includes the electronic device according to claim 25.

Images