Image enhancement method and device, electronic equipment and storage medium

Abstract

The invention relates to an image enhancement method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an original color image, and carrying out the color space conversion of the original color image, and obtaining a converted image; calculating a gradient matrix and an enhanced information matrix corresponding to the converted image, and determining an enhanced energy matrix corresponding to the converted image based on the gradient matrix and the enhanced information matrix; performing Gaussian filtering on the converted image to obtain a filtering graph, and obtaining a mask energy matrix based on the converted image and the filtering graph; based on the mask energy matrix and the enhanced energy matrix, performing energy enhancement on an edge region and a detail region in the converted image to obtain a target intermediate image; and performing color space conversion on the target intermediate image to obtain an energy-enhanced color image. Therefore, the definition of the original color image can be effectively improved, and the driving safety is further improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image enhancement method, device, electronic device and storage medium. Background Art

[0002] ‌With the development of electric and intelligent vehicles, high-level assisted driving and unmanned driving are receiving more and more attention. In the process of high-level assisted driving and unmanned driving, it is necessary to collect environmental information around the vehicle body. However, when using the on-board camera to collect images around the car, due to factors such as lighting and weather, the collected images may not be clear enough, resulting in the vehicle being unable to accurately perceive the surrounding environment information, thus affecting driving safety. Therefore, how to improve the clarity of the images collected by the on-board camera has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides an image enhancement method, device, electronic device and storage medium to solve the problem in the prior art that the images of the surroundings of a car collected by a vehicle-mounted camera have low clarity due to factors such as lighting and weather.

[0004] In a first aspect, the present application provides an image enhancement method, the method comprising: Acquire an original color image, and perform color space conversion on the original color image to obtain a converted image; Calculating a gradient matrix and an enhancement information matrix corresponding to the converted image, and determining an enhancement energy matrix corresponding to the converted image based on the gradient matrix and the enhancement information matrix; Performing Gaussian filtering on the converted image to obtain a filter image, and obtaining a mask energy matrix based on the converted image and the filter image; Based on the mask energy matrix and the enhancement energy matrix, energy enhancement is performed on edge regions and detail regions in the converted image to obtain a target intermediate image; The target intermediate image is subjected to color space conversion to obtain an energy-enhanced color image.

[0005] Optionally, the calculating a gradient matrix and an enhancement information matrix corresponding to the converted image includes: Calculating the horizontal gradient information and the vertical gradient information corresponding to each pixel in the converted image respectively, and obtaining a gradient matrix corresponding to the converted image based on the horizontal gradient information and the vertical gradient information corresponding to each pixel, wherein each element in the gradient matrix is ​​used to represent the comprehensive gradient information corresponding to each pixel, and the comprehensive gradient information corresponding to each pixel is comprehensively determined based on the horizontal gradient information and the vertical gradient information corresponding to each pixel; The neighborhood average pixel value corresponding to each pixel point in the converted image is calculated respectively, and the neighborhood average pixel value corresponding to each pixel point is compared with the pixel value corresponding to each pixel point, and according to the comparison result of each pixel point, an enhancement information matrix corresponding to the converted image is obtained, wherein each element in the enhancement information matrix is ​​used to characterize the enhancement information of each pixel point, and the enhancement information of each pixel point is determined according to the comparison result of each pixel point.

[0006] Optionally, the enhancement information of each pixel is expressed by the following formula: ; in, represents the enhancement information of the pixel at the i-th row and the j-th column in the converted image, represents the pixel value of the pixel at the i-th row and j-th column in the converted image, represents the weight coefficient for enhancing information strength, represents the neighborhood of the pixel point at the i-th row and j-th column, the size of the neighborhood is s×s, represents any pixel point in the neighborhood, Represents the sum of the pixel values ​​of each pixel in the neighborhood.

[0007] Optionally, determining an enhancement energy matrix corresponding to the converted image based on the gradient matrix and the enhancement information matrix includes: Based on the enhanced information matrix, determining the pulse value corresponding to each neuron in a preset neural network at different times, wherein the neurons in the preset neural network correspond one-to-one to the pixel points in the converted image; Obtaining the pulse value threshold corresponding to each neuron in the preset neural network at different times, and determining the ignition state corresponding to each neuron at different times based on the pulse value and threshold corresponding to each neuron at different times, wherein the ignition state includes ignition success and ignition failure, wherein the condition for satisfying the ignition success is that the pulse value is greater than or equal to the threshold corresponding to the same time, and the condition for satisfying the ignition failure is that the pulse value is less than the threshold corresponding to the same time; The enhanced energy matrix is ​​obtained based on the firing state of each neuron at different times and the enhanced energy of each pixel at different times, wherein the enhanced energy of each pixel at different times is calculated based on the gradient matrix.

[0008] Optionally, determining the pulse value corresponding to each neuron in the preset neural network at different times based on the enhanced information matrix includes: Determining enhancement information corresponding to each pixel in the converted image based on the enhancement information matrix; Obtaining the pulse value corresponding to each neuron in the preset neural network at the previous moment, the pulse value required for each neuron to be fired each time, the weight coefficient between each neuron and the neighboring neurons, the pulse value received by each neuron from the neighboring neurons, and the pixel value of each pixel in the filter image; Determine whether the pulse value generated by each neuron in the preset neural network after ignition is greater than a preset threshold; In the case where there are first-type neurons in the preset neural network whose pulse values ​​after ignition are greater than the preset threshold, based on the enhancement information of the pixel points corresponding to each of the first-type neurons, the pulse values ​​corresponding to each of the first-type neurons at the previous moment, the pulse values ​​required for each ignition of each of the first-type neurons, the weight coefficients between each of the first-type neurons and neighboring neurons, the pulse values ​​received by each of the first-type neurons from the neighboring neurons, and the pixel values ​​of the pixel points corresponding to each of the first-type neurons in the filter graph, the pulse values ​​corresponding to each of the first-type neurons at the current moment are determined; In the case where there are second-type neurons in the preset neural network, the pulse value generated after ignition is less than or equal to the preset threshold, the pulse value corresponding to the second-type neurons at the current moment is determined based on the enhancement information of the pixel points corresponding to each of the second-type neurons, the pulse value corresponding to each of the second-type neurons at the previous moment, the weight coefficient between each of the second-type neurons and the neighboring neurons, the pulse value received by each of the second-type neurons from the neighboring neurons, and the pixel value of the pixel points corresponding to the second-type neurons in each of the filter graphs.

[0009] Optionally, obtaining the enhanced energy matrix based on the firing state of each neuron at different times and the enhanced energy of each pixel at different times includes: Based on the gradient matrix, determining the target gradient direction corresponding to each pixel point in the converted image; Based on the target gradient direction corresponding to each pixel point, determine the weighted sum of the pixel values ​​and weight values ​​corresponding to multiple neighboring pixel points of each pixel point in the target gradient direction of the respective neighborhood, wherein the weight value corresponding to each neighboring pixel point corresponds one-to-one to the value of the convolution operator corresponding to the target gradient direction; In the case where there are third-type neurons whose ignition state is successfully ignited in the preset neural network, a weighted sum of pixel points corresponding to each of the third-type neurons is determined as a weight coefficient of the pixel points corresponding to each of the third-type neurons, and based on the weight coefficient of the pixel points corresponding to each of the third-type neurons and the enhancement information of the pixel points corresponding to each of the third-type neurons, the enhancement energy of the pixel points corresponding to each of the third-type neurons is determined; In the case where there are fourth-type neurons whose ignition state is ignition failure in the preset neural network, the enhancement energy of the pixel points corresponding to each of the fourth-type neurons is set to zero; The enhanced energy matrix is ​​obtained according to the enhanced energies corresponding to the pixel points corresponding to the third type of neurons and the enhanced energies corresponding to the pixel points corresponding to the fourth type of neurons.

[0010] Optionally, the step of performing energy enhancement on edge regions and detail regions in the converted image based on the mask energy matrix and the enhancement energy matrix to obtain a target intermediate image comprises: Calculating the product of the mask energy matrix and the enhanced energy matrix, and obtaining an initial intermediate image according to the calculation result, wherein the initial intermediate image only includes images of edge areas and detail areas after energy enhancement in the converted image; The initial intermediate image and the converted image are superimposed to obtain the target intermediate image, wherein the target intermediate image includes images of edge areas and detail areas after energy enhancement in the converted image, and images of flat areas without energy enhancement.

[0011] In a second aspect, the present application further provides an image enhancement device, the device comprising: A first conversion module is used to obtain an original color image and perform color space conversion on the original color image to obtain a converted image; A determination module, used to calculate a gradient matrix and an enhancement information matrix corresponding to the converted image, and determine an enhancement energy matrix corresponding to the converted image based on the gradient matrix and the enhancement information matrix; A filtering module, configured to perform Gaussian filtering on the converted image to obtain a filter image, and obtain a mask energy matrix based on the converted image and the filter image; An energy enhancement module, used for performing energy enhancement on edge areas and detail areas in the converted image based on the mask energy matrix and the enhancement energy matrix to obtain a target intermediate image; The second conversion module is used to perform color space conversion on the target intermediate image to obtain an energy-enhanced color image.

[0012] In a third aspect, the present application further provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement the image enhancement method described in any one of the first aspects when executing the program stored in the memory.

[0013] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the image enhancement method described in any one of the first aspects.

[0014] The above technical solution provided by the present application has the following advantages over the prior art: the method provided by the present application obtains the original color image and performs color space conversion on the original color image to obtain the converted image; calculates the gradient matrix and enhancement information matrix corresponding to the converted image, and determines the enhancement energy matrix corresponding to the converted image based on the gradient matrix and the enhancement information matrix; performs Gaussian filtering on the converted image to obtain a filter image, and obtains a mask energy matrix based on the converted image and the filter image; performs energy enhancement on the edge area and detail area in the converted image based on the mask energy matrix and the enhancement energy matrix to obtain a target intermediate image; performs color space conversion on the target intermediate image to obtain an energy-enhanced color image. In the above manner, the gradient matrix and enhancement information matrix of the converted image obtained after the color space conversion of the original color image can be used to obtain the enhancement energy matrix, and then based on the mask energy matrix and the enhancement energy matrix of the converted image, the edge area and detail area in the converted image can be energy enhanced, so that the clarity of the original color image can be effectively improved, thereby improving driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0018] Figure 1 A schematic diagram of a flow chart of an image enhancement method provided in an embodiment of the present application; Figure 2 A schematic diagram of an improved dynamic threshold neural P system provided in an embodiment of the present application; Figure 3 A schematic diagram of a neuron in an improved dynamic threshold neural P system provided in an embodiment of the present application; Figure 4 A schematic diagram of an unsharp masking process provided in an embodiment of the present application; Figure 5 A schematic diagram of a principle for obtaining enhanced information provided in an embodiment of the present application; Figure 6 A flowchart of another image enhancement method provided in an embodiment of the present application; Figure 7 A schematic diagram of the structure of an image enhancement device provided in an embodiment of the present application; Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0020] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0021] In order to solve the problem in the prior art that the images of the car's surroundings collected by the on-board camera have low clarity due to factors such as lighting and weather, the present application provides an image enhancement method, device, electronic device and storage medium that can improve the clarity of the image.

[0022] See also Figure 1 , Figure 1 The following is a flow chart of an image enhancement method provided in an embodiment of the present application. Figure 1 As shown, the image enhancement method may include the following steps: Step S101: Acquire an original color image, and perform color space conversion on the original color image to obtain a converted image.

[0023] Specifically, the original color image may refer to an RGB image captured by a vehicle-mounted camera, and the converted image refers to an HSV image obtained by performing color space conversion on the original color image.

[0024] In this step, the color space conversion is performed because HSV images are more in line with the intuitiveness of human color recognition and are convenient for color feature extraction and processing, while direct processing of RGB images may cause color difference problems and lead to color distortion. In contrast, HSV images divide colors into three independent parts: hue, saturation, and value. Hue represents the basic attributes of color, saturation represents the purity or vividness of color, and value represents the brightness of color. This decomposition makes the color representation of HSV space more intuitive, easy to understand and operate, especially when it is necessary to process color-related features, such as color segmentation and extraction, HSV space provides greater convenience and accuracy.

[0025] Step S102: Calculate the gradient matrix and the enhancement information matrix corresponding to the converted image, and determine the enhancement energy matrix corresponding to the converted image based on the gradient matrix and the enhancement information matrix.

[0026] After obtaining the converted image (ie, the HSV image), the gradient of the converted image may be calculated, such as by using any first-order derivative operator or second-order derivative operator to calculate the gradient of the converted image to obtain a gradient matrix corresponding to the converted image.

[0027] Moreover, after obtaining the converted image (i.e., HSV image), the enhanced information of the converted image can also be calculated. Specifically, the pixel value corresponding to each pixel point in the converted image can be enhanced by using the average pixel value of the neighborhood corresponding to each pixel point and the pixel value corresponding to each pixel point. After calculating the gradient matrix and the enhanced information matrix corresponding to the converted image, the enhanced energy matrix corresponding to the converted image can be determined based on the improved dynamic threshold neural P system (Dynamic Threshold Neural PSystems, referred to as EDTNP), the gradient matrix and the enhanced information matrix. Specifically, the improved dynamic threshold neural P system can obtain the enhanced energy input of each neuron and its neighborhood according to the edge or detail direction obtained by each neuron in its neighborhood, and add this input to the neuron and its neighborhood. In terms of topology, the improved dynamic threshold neural P system is a directed graph with local neighborhood connections in a plane. Figure 2 A schematic diagram of an improved dynamic threshold neural P system provided in an embodiment of the present application, Figure 2The EDTNP system with X as its input matrix is ​​shown, O is the enhanced information matrix of the non-flat area (referring to the edge area and the detail area), and S is an enhanced energy matrix of its external output. represents the element in the i-th row and j-th column of the input matrix X. represents the i-th row and j-th column element in the enhanced information matrix O, represents the element in the i-th row and j-th column of the enhanced energy matrix S, Represents the neuron in the i-th row and j-th column in the EDTNP system П. Figure 3 Schematic diagram showing neurons of the EDTNP system, neurons Includes a data unit (used to indicate the state of the neuron), a threshold , a pulse rule , External Input , enhanced information input in non-flat areas , gradient information input and external output . Pulse Rule The ignition conditions are It indicates that, is the state at time t, It's a neuron The threshold at time t. Figure 3 As can be seen from the dotted arrow in the figure, according to the gradient matrix V, the gradient information input can be calculated (used to indicate the gradient direction), and then input according to the gradient information Neurons can be identified The local weight coefficient , then we can use the neuron The local weight coefficient , Enhance information input , pulse value and dynamic threshold , together determine the neurons Enhanced energy It should be noted that in the EDTNP system, when , the neuron is activated, a process called firing. Meet the ignition conditions ,in When the neuron When the ignition condition is not met, ignition fails.

[0028] Step S103: Perform Gaussian filtering on the converted image to obtain a filter image, and obtain a mask energy matrix based on the converted image and the filter image.

[0029] Using the unsharp mask algorithm, the converted image is Gaussian filtered to obtain a filter image, and the converted image and the filter image are subtracted point by point to obtain a sharp template, that is, a mask energy matrix. The unsharp mask here refers to the original image being pre-processed with a fuzzy unsharp mask (equivalent to filtering with a low-pass filter), and performing a point-by-point difference operation with the original image to obtain a sharp template, and then multiplying the sharp template by a weight coefficient and adding it to the original image, thereby enhancing the high-frequency components in the image and enhancing the image contour, such as Figure 4 It can be expressed by the following formula: ; in, represents the original input image, represents the unsharp blurred image after blur preprocessing, is the sharpening template, k is the weight coefficient, when k>0, is the sharpening effect; when k<0, It is a passivation effect. It should be noted that after the original image is processed by low-pass filtering, the high-frequency components are suppressed, making the image blurry. Subtracting the original image from the blurred image can make The low-frequency components in the image are greatly reduced, while the high-frequency components are better preserved. Therefore, when the high-frequency components in an image are amplified and compared with the original image When superimposed together, its high-frequency components are enhanced while the low-frequency components are not greatly affected. Step S104: Based on the mask energy matrix and the enhancement energy matrix, energy enhancement is performed on the edge area and detail area in the converted image to obtain a target intermediate image.

[0030] The enhanced energy matrix is ​​used as the weight coefficient in the above unsharp mask algorithm, multiplied by the mask energy matrix, and then added to the converted image to obtain the target intermediate image. It should be noted that the target intermediate image is the HSV image after energy enhancement.

[0031] Step S105: Perform color space conversion on the target intermediate image to obtain an energy-enhanced color image.

[0032] After obtaining the target intermediate image, the target intermediate image can be converted from the HSV color space to the RGB color space, thereby obtaining an energy-enhanced color image, that is, an energy-enhanced RGB image.

[0033] In this embodiment, the gradient matrix and enhancement information matrix of the converted image obtained after color space conversion of the original color image can be used to obtain an enhanced energy matrix, and then based on the mask energy matrix and enhancement energy matrix of the converted image, energy enhancement is performed on the edge area and detail area in the converted image. In this way, the clarity of the original color image can be effectively improved, thereby improving driving safety.

[0034] Furthermore, the above step S102, calculating the gradient matrix and the enhancement information matrix corresponding to the converted image, includes: Calculate the horizontal gradient information and the vertical gradient information corresponding to each pixel in the converted image respectively, and obtain the gradient matrix corresponding to the converted image based on the horizontal gradient information and the vertical gradient information corresponding to each pixel, wherein each element in the gradient matrix is ​​used to represent the comprehensive gradient information corresponding to each pixel, and the comprehensive gradient information corresponding to each pixel is determined comprehensively based on the horizontal gradient information and the vertical gradient information corresponding to each pixel; The neighborhood average pixel value corresponding to each pixel in the converted image is calculated respectively, and the neighborhood average pixel value corresponding to each pixel is compared with the pixel value corresponding to each pixel. According to the comparison result of each pixel, the enhancement information matrix corresponding to the converted image is obtained, wherein each element in the enhancement information matrix is ​​used to characterize the enhancement information of each pixel, and the enhancement information of each pixel is determined according to the comparison result of each pixel.

[0035] In one embodiment, any first-order derivative operator or second-order derivative operator can be used to respectively calculate the horizontal gradient information and the vertical gradient information corresponding to each pixel in the converted image, and based on the horizontal gradient information and the vertical gradient information corresponding to each pixel, a gradient matrix corresponding to the converted image is obtained. As an optional implementation, a Sobel operator (i.e., a Sobel operator) can be used for calculation. Specifically, the Sobel operator can be represented by the following matrix: ; ; In this way, the horizontal gradient information and vertical gradient information corresponding to each pixel in the converted image can be calculated according to the Sobel operator. The calculation formula is as follows: ; in, Represents the horizontal gradient information of the (i, j) pixel in the converted image. Represents the vertical gradient information of the (i, j) pixel in the converted image. Represents the pixel value of the (i, j) pixel in the converted image. Therefore, by using the Sobel convolution operator to perform convolution calculation on all pixels in the converted image, the horizontal gradient information and vertical gradient information corresponding to each pixel can be accurately and quickly obtained, and then based on the horizontal gradient information and vertical gradient information corresponding to each pixel, the gradient matrix corresponding to the converted image can be accurately and quickly obtained.

[0036] In addition, the neighborhood average pixel value corresponding to each pixel in the converted image can be calculated respectively, wherein the neighborhood size corresponding to each pixel can be set according to the actual situation, such as 3×3, 4×4, etc. Then the neighborhood average pixel value corresponding to each pixel is compared with the pixel value corresponding to each pixel, and different enhancement processing is performed on each pixel according to the different comparison results of each pixel. For example, when the neighborhood average pixel value corresponding to a certain pixel is less than or equal to the pixel value corresponding to the pixel, the pixel value corresponding to the pixel is increased by a preset pixel value; when the neighborhood average pixel value corresponding to a certain pixel is greater than the pixel value corresponding to the pixel, the pixel value corresponding to the pixel is subtracted by a preset pixel value. Finally, according to the updated pixel values ​​of each pixel, the enhancement information matrix corresponding to the converted image is obtained. In this way, the gap between the pixel values ​​of each pixel can be widened, so that the pixel value of the pixel corresponding to the larger pixel value is larger, and the pixel value of the pixel corresponding to the smaller pixel value is smaller, thereby effectively identifying the edge area and detail area in the image.

[0037] Furthermore, the enhancement information of each pixel is expressed by the following formula: ; in, Represents the enhanced information of the pixel in the i-th row and j-th column of the converted image. Represents the pixel value of the pixel at the i-th row and j-th column in the converted image, represents the weight coefficient for enhancing information strength, represents the neighborhood of the pixel in the i-th row and j-th column, and the size of the neighborhood is s×s. represents any pixel in the neighborhood, Represents the sum of the pixel values ​​of each pixel in the neighborhood.

[0038] According to the above formula, when the average pixel value of the neighborhood corresponding to a pixel point is less than or equal to the pixel value corresponding to the pixel point, the pixel value corresponding to the pixel point is increased by ; When the average pixel value of the neighborhood corresponding to a pixel point is greater than the pixel value corresponding to the pixel point, the pixel value corresponding to the pixel point is subtracted .in, The size of is positively correlated with the pixel value of the pixel point in the i-th row and j-th column. Therefore, the pixel value increased or decreased by each pixel point is positively correlated with the pixel value of its own pixel point, rather than increasing or decreasing a fixed pixel value. This is more conducive to flexible adjustment of the pixel value of each pixel point.

[0039] It should be noted that, based on the obtained enhancement information of each pixel point, it is possible to accurately determine whether each pixel point is in a flat area or a non-flat area (i.e., edge and detail area). Generally, the larger the pixel value of the pixel point corresponding to the pixel point in its enhancement information matrix in the non-flat area, the smaller the pixel value of the pixel point corresponding to the pixel point in its enhancement information matrix in the flat area, or even close to zero. Binarizing the obtained enhancement information can obtain an enhancement information result map, such as Figure 5 As shown. By obtaining the enhanced information of each pixel, the edge direction of the input image I is found, and some noise points can be isolated, and finally the enhanced information matrix O is obtained. According to the value of this enhanced information matrix, it can be judged whether the pixel is in the edge detail area or in the flat area. However, there are still some areas with complex noise and information that cannot be accurately judged, which require subsequent processing and optimization.

[0040] Furthermore, the above step S102, determining the enhanced energy matrix corresponding to the converted image based on the gradient matrix and the enhanced information matrix, includes: Based on the enhanced information matrix, determining the pulse value corresponding to each neuron in the preset neural network at different times, wherein the neurons in the preset neural network correspond one-to-one to the pixel points in the converted image; Obtaining the pulse value threshold corresponding to each neuron in the preset neural network at different times, and determining the ignition state corresponding to each neuron at different times based on the pulse value and threshold corresponding to each neuron at different times, wherein the ignition state includes ignition success and ignition failure, wherein the condition for ignition success is that the pulse value is greater than or equal to the threshold corresponding to the same time, and the condition for ignition failure is that the pulse value is less than the threshold corresponding to the same time; Based on the firing state of each neuron at different times and the enhancement energy of each pixel at different times, an enhancement energy matrix is ​​obtained, wherein the enhancement energy of each pixel at different times is calculated based on the gradient matrix.

[0041] In one embodiment, when the gradient matrix and the enhancement information matrix are input into the EDTNP system to determine the enhancement energy matrix corresponding to the converted image, the pulse value corresponding to each neuron in the preset neural network (i.e., the neural network in the EDTNP system) at different times can be determined based on the enhancement information matrix, and the pulse value threshold corresponding to each neuron in the preset neural network at different times can be obtained, and based on the pulse value and threshold corresponding to each neuron at different times, the ignition state corresponding to each neuron at different times can be determined. It should be noted that the EDTNP system has two rules: one is the ignition rule, which has a unified form , p>0. Among them, , represents the pulse value of the i-th neuron, represents the threshold of the i-th neuron, represents the firing condition of the i-th neuron, and p represents the pulse value generated by the i-th neuron after firing. The other is the forgetting rule. When p=0, it is called the forgetting rule. For the firing rule, there is an ignition condition: , the semantic description of the rule is: Assuming that the EDTNP system works at time t, if the ignition condition E is met, that is, , neurons Ignition rules will be used, then the neuron Ignition. When the ignition is successful, the neuron Number of pulses and threshold will change, and its enhanced energy will also change; when When the firing fails, the neuron Number of pulses and threshold There is no change, and its enhanced energy will also be set to zero. In this way, after determining the firing state of each neuron at different times, the enhanced energy matrix can be obtained based on the firing state of each neuron at different times and the enhanced energy of each pixel at different times.

[0042] In this embodiment, since the enhanced energy of each pixel point corresponding to each element in the enhanced energy matrix is ​​calculated based on the gradient matrix, that is, the larger the gradient amplitude, the larger the corresponding enhanced energy; the smaller the gradient amplitude, the smaller the corresponding enhanced energy, the enhanced energy matrix can enhance the energy of the pixel points in the non-flat area.

[0043] Furthermore, the above steps, based on the enhanced information matrix, determine the pulse values ​​corresponding to each neuron in the preset neural network at different times, including: Based on the enhancement information matrix, the enhancement information corresponding to each pixel in the converted image is determined; Obtain the pulse value corresponding to each neuron in the preset neural network at the previous moment, the pulse value consumed by each neuron each time it is fired, the weight coefficient between each neuron and its neighboring neurons, the pulse value received by each neuron from its neighboring neurons, and the pixel value of each pixel in the filter image; Determine whether the pulse value generated by each neuron in the preset neural network after ignition is greater than a preset threshold; In the case where there are first-class neurons in the preset neural network whose pulse values ​​after ignition are greater than a preset threshold, based on the enhancement information of the pixel points corresponding to each first-class neuron, the pulse value corresponding to each first-class neuron at the previous moment, the pulse value required for each ignition of each first-class neuron, the weight coefficient between each first-class neuron and the neighboring neurons, the pulse value received by each first-class neuron from the neighboring neurons, and the pixel value of the pixel point corresponding to each first-class neuron in the filter graph, the pulse value corresponding to each first-class neuron at the current moment is determined; In the case where there are second-type neurons in the preset neural network whose pulse values ​​after ignition are less than or equal to a preset threshold, the pulse value corresponding to the second-type neurons at the current moment is determined based on the enhancement information of the pixel points corresponding to each second-type neuron, the pulse value corresponding to each second-type neuron at the previous moment, the weight coefficient between each second-type neuron and the neighboring neurons, the pulse value received by each second-type neuron from the neighboring neurons, and the pixel values ​​of the pixel points corresponding to the second-type neurons in each filter graph.

[0044] In one embodiment, the following formula may be used to determine the pulse values ​​corresponding to each neuron in the preset neural network at different times: ; in, Representing neurons exist The pulse value corresponding to the moment, Representing neurons exist The pulse value corresponding to the moment, Representing neurons The pulse value required for each ignition, Represents the pixel value of the pixel (i, j) in the filter image, It's a neuron The neighborhood of is s×s, Indicates the neighborhood Any neuron in Representing neurons From the neighboring neurons The received pulse value, With neurons in the neighborhood There is a synapse between them, which corresponds to a weight coefficient , the size of this coefficient can reflect the closeness of the relationship between two neurons. Represents the enhanced information corresponding to the pixel (i, j) in the converted image H, Representing neurons The pulse value generated after ignition.

[0045] According to the above formula, the preset threshold is set to 1. The preset threshold can be set according to the actual situation and is not specifically limited in the present embodiment. To calculate ; For the second type of neurons, we can use To calculate .

[0046] It should be noted that neurons The threshold can be updated using the following formula: ; in, Representing neurons exist The threshold corresponding to the moment, Representing neurons exist The pulse value corresponding to the moment, is a preset value, Representing neurons The pulse value generated after ignition.

[0047] Through the above method, based on the enhanced information matrix, the pulse value corresponding to each neuron in the preset neural network at different times can be accurately determined, which is convenient for the subsequent determination of the firing state of each neuron at different times based on the pulse value and threshold corresponding to each neuron at different times.

[0048] Furthermore, the above steps, based on the firing state of each neuron at different times and the enhancement energy of each pixel at different times, obtain an enhancement energy matrix, including: Based on the gradient matrix, determine the target gradient direction corresponding to each pixel in the converted image; Based on the target gradient direction corresponding to each pixel point, determine the weighted sum of the pixel values ​​and weight values ​​corresponding to multiple neighboring pixel points of each pixel point in the target gradient direction of the respective neighborhood, wherein the weight value corresponding to each neighboring pixel point corresponds one-to-one to the value of the convolution operator corresponding to the target gradient direction; In the case where there are third-type neurons whose ignition state is successfully ignited in the preset neural network, a weighted summation result of the pixel points corresponding to each third-type neuron is determined as a weight coefficient of the pixel points corresponding to each third-type neuron, and based on the weight coefficient of the pixel points corresponding to each third-type neuron and the enhancement information of the pixel points corresponding to each third-type neuron, the enhancement energy of the pixel points corresponding to each third-type neuron is determined; When there are fourth-type neurons whose firing state is firing failure in the preset neural network, the enhancement energy of the pixel points corresponding to each fourth-type neuron is set to zero; An enhanced energy matrix is ​​obtained according to the enhanced energies corresponding to the pixel points corresponding to each third type of neuron and the enhanced energies corresponding to the pixel points corresponding to each fourth type of neuron.

[0049] In one embodiment, when determining the enhanced energy matrix, the target gradient direction corresponding to each pixel in the converted image can be determined based on the gradient matrix, and then the weighted sum of the pixel values ​​and weight values ​​corresponding to multiple neighboring pixel points in the target gradient direction of each pixel in the neighborhood of each pixel can be determined based on the target gradient direction of each pixel. Specifically, the following formula can be used for calculation: ; in, Indicates that the pixel point P is in the direction The weight coefficient on represents a neighborhood with a radius of r and a pixel point P as the center. The value of r can be set according to the actual situation, such as 7. Q is any pixel point in the neighborhood. Represents the pixel value corresponding to the neighborhood pixel point Q; for The neighborhood pixel point Q in the operator corresponds to the value of the operator. The value can be 1 to 18, each The value corresponds to a directional convolution operator.

[0050] In the case where there are third-type neurons whose ignition state is successful ignition in the preset neural network, the weighted sum of the pixel points corresponding to each third-type neuron is determined as the weight coefficient of the pixel points corresponding to each third-type neuron, and based on the weight coefficient of the pixel points corresponding to each third-type neuron and the enhancement information of the pixel points corresponding to each third-type neuron, the enhancement energy of the pixel points corresponding to each third-type neuron is determined. In the case where there are fourth-type neurons whose ignition state is failed ignition in the preset neural network, the enhancement energy of the pixel points corresponding to each fourth-type neuron is set to zero, which can be expressed by the following formula: ; in, Representing neurons Energy before the update, Representing neurons Renewed energy, Representing neurons The local weight coefficient of Representing neurons The pulse value generated after ignition, Represents the enhanced information corresponding to the pixel (i, j) in the converted image H, Representing neurons The weight coefficient in the direction of its own gradient.

[0051] If the neuron Ignite, satisfy All neurons, including neurons itself, will use the above formula for Edge Strength To update, the neuron Correlation coefficient with itself is 1, and the energy value transferred is .

[0052] Getting neurons Energy after ignition After that, we can also use neurons Energy after ignition The gradient matrix V input to the EDTNP system is updated as follows: ; According to the above process, a total of After iterations, the enhanced energy matrix S is updated. After the enhanced energy matrix S is output, the operation of the EDTNP system can be completed.

[0053] In this way, the enhanced energy matrix can be accurately obtained, which facilitates the subsequent accurate energy enhancement of the edge area and detail area in the converted image based on the mask energy matrix and the enhanced energy matrix.

[0054] Furthermore, the above step S103, based on the mask energy matrix and the enhancement energy matrix, performs energy enhancement on the edge area and the detail area in the converted image to obtain the target intermediate image, includes: Calculate the product of the mask energy matrix and the enhanced energy matrix, and obtain an initial intermediate image according to the calculation result, wherein the initial intermediate image only includes images of edge regions and detail regions after energy enhancement in the converted image; The initial intermediate image and the converted image are superimposed to obtain a target intermediate image, wherein the target intermediate image includes images of edge areas and detail areas after energy enhancement in the converted image, and images of flat areas without energy enhancement.

[0055] In one embodiment, in the process of obtaining the target intermediate image based on the mask energy matrix and the enhanced energy matrix, the enhanced energy matrix can be first used as the weight coefficient in the above-mentioned unsharp mask algorithm, and multiplied with the mask energy matrix, and the initial intermediate image is obtained according to the calculation result, wherein the initial intermediate image here only includes the images of the edge area and detail area after energy enhancement in the converted image. Then, the initial intermediate image and the converted image are superimposed to obtain the target intermediate image, wherein the target intermediate image here includes the images of the edge area and detail area after energy enhancement in the converted image, and the images of the flat area without energy enhancement. In other words, the target intermediate image is an HSV image containing the image information of the original color image and after energy enhancement of the edge. In this way, the color space conversion of the target intermediate image can be performed to obtain the color image after energy enhancement.

[0056] In one embodiment, the image enhancement process provided in the embodiment of the present application is as follows: Figure 6 As shown, it includes the following steps: 1. Perform color space conversion on the RGB color image I (equivalent to the original color image mentioned above) and convert it into the HSV component image H (equivalent to the converted image mentioned above); 2. Process the HSV component map H to obtain the gradient matrix V and the enhanced information matrix O corresponding to the HSV component map H, and then perform the EDTNP system function on the gradient matrix V and the enhanced information matrix O to obtain the enhanced energy matrix S; 3. Perform Gaussian filtering on the HSV component image H to obtain the filter image G, and subtract the HSV component image H from the filter image G to obtain the mask energy matrix M; 4. Multiply the enhanced energy matrix S and the mask energy matrix M, and superimpose the multiplication result back to the HSV component map H to obtain the enhanced image H′; 5. Perform color space conversion on the enhanced image H′ and restore it to the color enhanced image I′.

[0057] It should be noted that the input image H of EDTNP is the HSV image of the original color image I. First, the Sobel operator is used to calculate the gradient of image H to obtain the gradient matrix V and direction θ, and the superposition operator of the corresponding direction is obtained through its direction θ. Then the directional enhancement information matrix O of image H is calculated. This matrix can determine whether each pixel is in a non-flat area that can be enhanced. Then, according to the principle of the EDTNP system, the gradient matrix V and the enhancement information matrix O are subjected to EDTNP calculation, and the EDTNP edge and detail area enhancement energy matrix S activated after ignition can be obtained. At the same time, after Gaussian filtering of image H according to the unsharp mask processing method, the filter image G is obtained, and then the filter image G is subtracted from image H to obtain a mask energy matrix M. Since the unsharp mask is a global calculation for the entire image, it is not selective in itself. It cannot distinguish between edges, detail areas and flat areas during enhancement. Calculating the entire image will generate a lot of noise. The obtained enhancement energy matrix S is used to select edges and detail areas, and edges and details of different intensities have different energy values. The enhanced energy matrix S is multiplied by the mask energy matrix M and added back to the original HSV image to obtain the final enhanced image H′. Finally, it is restored to a color image, and the edge and detail areas are enhanced to obtain the final enhanced image I′.

[0058] Algorithm 1 introduces the execution process of UM-EDTNP. For a color image I, first execute the "Rgb2v()" function, which is used to obtain the HSV component map H of the RGB color image I; then execute the "GradientMap()" function, which is used to calculate the gradient amplitude of the HSV component map H and obtain the gradient information V; then execute the "enhancement()" function, which is used to calculate the enhancement information of the HSV component map H and obtain the enhancement information matrix O; then execute the "GaussBlur()" function, which is used to perform Gaussian filtering on the HSV component map H to obtain the filter map G; then execute the "Normalize()" function, which is used to normalize the enhancement information matrix O, which is a normalization function in [0,1]; then execute the "EDTNP()" function, which is used to calculate the enhancement energy matrix S based on the enhancement information matrix O and the gradient information V, and finally execute the "2vRgb()" function, which is used to convert the enhanced image H′ into a color space and restore it to the color enhancement map I′.

[0059] Algorithm 2 introduces the execution process of the EDTNP function. Taking the calculated enhanced information image O and gradient information V as input, U, T, P, and S can be initialized first. Specifically, U = = Function, initializes U, where Representing neurons The real-time status is initially 0. Indicates the size of the HSV component map; it can be expressed as T = = Function, initializes T, where Representing neurons The threshold value can be expressed as P = = Function, initialize P, the initial value is 0; you can use S = = Function, initializes S to 0 to output the enhanced image. Then, according to the gradient information V, the corresponding edge gradient direction is obtained, and then the weight matrix W of the local neighborhood is calculated. Each element in the weight matrix is ​​a In the edge and detail enhancement model of EDTNP, d is selected as 3. That is to say, for the weight matrix W , which is associated with other neurons in a square area with a neighborhood size of 3. When a neuron in the neighborhood fires, it can transfer energy to the central neuron. The energy value is calculated based on the distance and direction difference between neurons. Then the main iteration process of EDTNP is executed to update U and T. Corresponding When , S is updated, and the enhanced energy S is superimposed back to the gradient map V to enhance the connectivity of the edge and detail areas. After iterations, the enhanced energy matrix S is updated, and the operation of the EDTNP system is completed after S is output.

[0060] Combined with the unsharp mask process for image enhancement, a mask with edge energy is superimposed on the original image after each ignition iteration to enhance the edge and detail areas. At the beginning, the mask energy is concentrated in the strong edge area. As the number of superpositions increases, the mask energy in the weak edge and detail areas becomes stronger and stronger. In the process, the strong edge is enhanced to the greatest extent, while most of the noise is not enhanced, and even suppressed to a certain extent in the process of edge enhancement. Using the UM-EDTNP algorithm, the blurred image can be enhanced well to obtain a clearer vehicle image, which improves the driving safety of autonomous driving.

[0061] It should be noted that the clarity and accuracy of the edge structure mainly depends on the positioning of the edge direction. This application solves this problem by stacking the energy in the edge direction through an improved dynamic threshold neural P system to obtain the accurate edge direction. The ignition mechanism and direction feedback of the dynamic threshold neural P system are combined with the unsharp mask algorithm to enhance the edge of the blurred image so that the basic structure of the image is not offset and an enhanced image with less noise is obtained. This application starts from the dynamic threshold neural P system and the unsharp mask algorithm and combines them, but the traditional unsharp mask algorithm is very difficult to select enhanced pixels, and only performs an overall operation once, and the enhanced image obtained is poor. Therefore, the dynamic threshold neural P system is used to coordinate the ignition and dynamic threshold characteristics to improve the local neighborhood correlation of the image and detect the edges that need to be strengthened, maintain the shape of the edge, make the originally blurred edge clear and not distorted, and do not do too much processing in the flat area without edges to avoid noise explosion affecting the image quality.

[0062] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of an image enhancement device provided in an embodiment of the present application. Figure 7 As shown, the image enhancement device 700 includes: The first conversion module 701 is used to obtain an original color image and perform color space conversion on the original color image to obtain a converted image; A determination module 702 is used to calculate a gradient matrix and an enhancement information matrix corresponding to the converted image, and determine an enhancement energy matrix corresponding to the converted image based on the gradient matrix and the enhancement information matrix; A filtering module 703 is used to perform Gaussian filtering on the converted image to obtain a filter image, and obtain a mask energy matrix based on the converted image and the filter image; An energy enhancement module 704 is used to enhance the energy of edge areas and detail areas in the converted image based on the mask energy matrix and the enhancement energy matrix to obtain a target intermediate image; The second conversion module 705 is used to perform color space conversion on the target intermediate image to obtain an energy-enhanced color image.

[0063] Further, the determination module 702 includes: A first calculation submodule is used to respectively calculate the horizontal gradient information and the vertical gradient information corresponding to each pixel in the converted image, and obtain a gradient matrix corresponding to the converted image based on the horizontal gradient information and the vertical gradient information corresponding to each pixel, wherein each element in the gradient matrix is ​​used to represent the comprehensive gradient information corresponding to each pixel, and the comprehensive gradient information corresponding to each pixel is determined comprehensively based on the horizontal gradient information and the vertical gradient information corresponding to each pixel; The second calculation submodule is used to calculate the neighborhood average pixel value corresponding to each pixel point in the converted image, and compare the neighborhood average pixel value corresponding to each pixel point with the pixel value corresponding to each pixel point, and obtain the enhancement information matrix corresponding to the converted image according to the comparison results of each pixel point, wherein each element in the enhancement information matrix is ​​used to characterize the enhancement information of each pixel point, and the enhancement information of each pixel point is determined according to the comparison results of each pixel point.

[0064] Furthermore, the enhancement information of each pixel is expressed by the following formula: ; in, Represents the enhanced information of the pixel in the i-th row and j-th column of the converted image. Represents the pixel value of the pixel at the i-th row and j-th column in the converted image, represents the weight coefficient for enhancing information strength, represents the neighborhood of the pixel in the i-th row and j-th column, and the size of the neighborhood is s×s. represents any pixel in the neighborhood, Represents the sum of the pixel values ​​of each pixel in the neighborhood.

[0065] Furthermore, the determination module 702 further includes: A first determination submodule is used to determine the pulse value corresponding to each neuron in a preset neural network at different times based on the enhanced information matrix, wherein the neurons in the preset neural network correspond one-to-one to the pixel points in the converted image; The second determination submodule is used to obtain the pulse value threshold corresponding to each neuron in the preset neural network at different times, and determine the ignition state corresponding to each neuron at different times based on the pulse value and threshold corresponding to each neuron at different times, wherein the ignition state includes ignition success and ignition failure, wherein the condition for ignition success is that the pulse value is greater than or equal to the threshold corresponding to the same time, and the condition for ignition failure is that the pulse value is less than the threshold corresponding to the same time; The execution submodule is used to obtain an enhanced energy matrix based on the firing state of each neuron at different times and the enhanced energy of each pixel at different times, wherein the enhanced energy of each pixel at different times is calculated based on the gradient matrix.

[0066] Furthermore, the first determining submodule includes: A first determining unit, configured to determine, based on the enhancement information matrix, enhancement information corresponding to each pixel in the converted image; An acquisition unit is used to acquire the pulse value corresponding to each neuron in the preset neural network at the previous moment, the pulse value consumed by each neuron each time it is fired, the weight coefficient between each neuron and its neighboring neurons, the pulse value received by each neuron from its neighboring neurons, and the pixel value of each pixel point in the filter image; A judgment unit, used to judge whether the pulse value generated by each neuron in the preset neural network after ignition is greater than a preset threshold value; A second determination unit is used to determine the pulse value corresponding to each first-class neuron at the current moment based on the enhancement information of the pixel points corresponding to each first-class neuron, the pulse value corresponding to each first-class neuron at the previous moment, the pulse value consumed by each first-class neuron for each firing, the weight coefficient between each first-class neuron and the neighboring neurons, the pulse value received by each first-class neuron from the neighboring neurons, and the pixel value of the pixel point corresponding to each first-class neuron in the filter graph, when there are first-class neurons in the preset neural network whose pulse value generated after firing is greater than the preset threshold value; The third determination unit is used to determine the pulse value corresponding to the second type of neurons at the current moment based on the enhancement information of the pixel points corresponding to each second type of neurons, the pulse value corresponding to each second type of neurons at the previous moment, the weight coefficient between each second type of neurons and the neighboring neurons, the pulse value received by each second type of neurons from the neighboring neurons, and the pixel value of the pixel points corresponding to the second type of neurons in each filter graph, when there are second type of neurons in the preset neural network whose pulse value generated after ignition is less than or equal to the preset threshold.

[0067] Furthermore, the execution submodule includes: A fourth determining unit, used to determine a target gradient direction corresponding to each pixel point in the converted image based on the gradient matrix; a fifth determining unit, configured to determine, based on the target gradient direction corresponding to each pixel point, a weighted summation result of pixel values ​​and weight values ​​corresponding to a plurality of neighboring pixel points in the target gradient direction of each pixel point in its own neighborhood, wherein the weight value corresponding to each neighboring pixel point corresponds one-to-one to the value of the convolution operator corresponding to the target gradient direction; a sixth determining unit, for determining, when there are third-type neurons whose ignition state is successfully ignited in the preset neural network, a weighted summation result of the pixel points corresponding to each third-type neuron as a weight coefficient of the pixel points corresponding to each third-type neuron, and determining the enhancement energy of the pixel points corresponding to each third-type neuron based on the weight coefficient of the pixel points corresponding to each third-type neuron and the enhancement information of the pixel points corresponding to each third-type neuron; A setting unit, configured to set the enhancement energy of the pixel points corresponding to each fourth-type neuron to zero when there are fourth-type neurons whose ignition state is ignition failure in the preset neural network; The seventh determining unit is used to obtain an enhanced energy matrix according to the enhanced energies corresponding to the pixel points corresponding to each third type of neuron and the enhanced energies corresponding to the pixel points corresponding to each fourth type of neuron.

[0068] Furthermore, the energy enhancement module 704 includes: A third calculation submodule is used to calculate the product of the mask energy matrix and the enhanced energy matrix, and obtain an initial intermediate image according to the calculation result, wherein the initial intermediate image only includes images of edge areas and detail areas after energy enhancement in the converted image; The superposition submodule is used to superimpose the initial intermediate image and the converted image to obtain a target intermediate image, wherein the target intermediate image includes images of edge areas and detail areas after energy enhancement in the converted image, and images of flat areas without energy enhancement.

[0069] It should be noted that the image enhancement device 700 can implement the image enhancement method provided by any of the aforementioned method embodiments and can achieve the same technical effects, which will not be described in detail herein.

[0070] like Figure 8 As shown, the embodiment of the present application further provides an electronic device, including a processor 811, a communication interface 812, a memory 813 and a communication bus 814, wherein the processor 811, the communication interface 812, and the memory 813 communicate with each other through the communication bus 814. Memory 813, used for storing computer programs; In one embodiment of the present application, the processor 811 is used to implement the image enhancement method provided by any one of the aforementioned method embodiments when executing the program stored in the memory 813.

[0071] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the image enhancement method provided by any of the aforementioned method embodiments is implemented.

[0072] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0073] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0074] It should be understood that the terms used in the text are only for the purpose of describing specific example embodiments, and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used in the text may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0075] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An image enhancement method, characterized in that: The method comprises: Acquire an original color image, and perform color space conversion on the original color image to obtain a converted image; Calculating a gradient matrix and an enhancement information matrix corresponding to the converted image, and determining an enhancement energy matrix corresponding to the converted image based on the gradient matrix and the enhancement information matrix; Performing Gaussian filtering on the converted image to obtain a filter image, and obtaining a mask energy matrix based on the converted image and the filter image; Based on the mask energy matrix and the enhancement energy matrix, energy enhancement is performed on edge regions and detail regions in the converted image to obtain a target intermediate image; The target intermediate image is subjected to color space conversion to obtain an energy-enhanced color image.

2. The image enhancement method according to claim 1, characterized in that: The calculating of the gradient matrix and the enhancement information matrix corresponding to the converted image includes: Calculating the horizontal gradient information and the vertical gradient information corresponding to each pixel in the converted image respectively, and obtaining a gradient matrix corresponding to the converted image based on the horizontal gradient information and the vertical gradient information corresponding to each pixel, wherein each element in the gradient matrix is ​​used to represent the comprehensive gradient information corresponding to each pixel, and the comprehensive gradient information corresponding to each pixel is comprehensively determined based on the horizontal gradient information and the vertical gradient information corresponding to each pixel; The neighborhood average pixel value corresponding to each pixel point in the converted image is calculated respectively, and the neighborhood average pixel value corresponding to each pixel point is compared with the pixel value corresponding to each pixel point, and according to the comparison result of each pixel point, an enhancement information matrix corresponding to the converted image is obtained, wherein each element in the enhancement information matrix is ​​used to characterize the enhancement information of each pixel point, and the enhancement information of each pixel point is determined according to the comparison result of each pixel point.

3. The image enhancement method according to claim 2, characterized in that: The enhancement information of each pixel is expressed by the following formula: ； in, represents the enhancement information of the pixel at the i-th row and the j-th column in the converted image, represents the pixel value of the pixel at the i-th row and j-th column in the converted image, represents the weight coefficient for enhancing information strength, represents the neighborhood of the pixel point at the i-th row and j-th column, the size of the neighborhood is s×s, represents any pixel point in the neighborhood, Represents the sum of the pixel values ​​of each pixel in the neighborhood.

4. The image enhancement method according to claim 1, characterized in that: The step of determining an enhanced energy matrix corresponding to the converted image based on the gradient matrix and the enhanced information matrix includes: Based on the enhanced information matrix, determining the pulse value corresponding to each neuron in a preset neural network at different times, wherein the neurons in the preset neural network correspond one-to-one to the pixel points in the converted image; Obtaining the pulse value threshold corresponding to each neuron in the preset neural network at different times, and determining the ignition state corresponding to each neuron at different times based on the pulse value and threshold corresponding to each neuron at different times, wherein the ignition state includes ignition success and ignition failure, wherein the condition for satisfying the ignition success is that the pulse value is greater than or equal to the threshold corresponding to the same time, and the condition for satisfying the ignition failure is that the pulse value is less than the threshold corresponding to the same time; The enhanced energy matrix is ​​obtained based on the firing state of each neuron at different times and the enhanced energy of each pixel at different times, wherein the enhanced energy of each pixel at different times is calculated based on the gradient matrix.

5. The image enhancement method according to claim 4, characterized in that: Determining the pulse values ​​corresponding to each neuron in the preset neural network at different times based on the enhanced information matrix includes: Determining enhancement information corresponding to each pixel in the converted image based on the enhancement information matrix; Obtaining the pulse value corresponding to each neuron in the preset neural network at the previous moment, the pulse value required for each neuron to be fired each time, the weight coefficient between each neuron and the neighboring neurons, the pulse value received by each neuron from the neighboring neurons, and the pixel value of each pixel in the filter image; Determine whether the pulse value generated by each neuron in the preset neural network after ignition is greater than a preset threshold; In the case where there are first-type neurons in the preset neural network whose pulse values ​​after ignition are greater than the preset threshold, based on the enhancement information of the pixel points corresponding to each of the first-type neurons, the pulse values ​​corresponding to each of the first-type neurons at the previous moment, the pulse values ​​required for each ignition of each of the first-type neurons, the weight coefficients between each of the first-type neurons and neighboring neurons, the pulse values ​​received by each of the first-type neurons from the neighboring neurons, and the pixel values ​​of the pixel points corresponding to each of the first-type neurons in the filter graph, the pulse values ​​corresponding to each of the first-type neurons at the current moment are determined; In the case where there are second-type neurons in the preset neural network, the pulse value generated after ignition is less than or equal to the preset threshold, the pulse value corresponding to the second-type neurons at the current moment is determined based on the enhancement information of the pixel points corresponding to each of the second-type neurons, the pulse value corresponding to each of the second-type neurons at the previous moment, the weight coefficient between each of the second-type neurons and the neighboring neurons, the pulse value received by each of the second-type neurons from the neighboring neurons, and the pixel value of the pixel points corresponding to the second-type neurons in each of the filter graphs.

6. The image enhancement method according to claim 4, characterized in that: The enhancement energy matrix is ​​obtained based on the firing state of each neuron at different times and the enhancement energy of each pixel at different times, including: Based on the gradient matrix, determining the target gradient direction corresponding to each pixel point in the converted image; Based on the target gradient direction corresponding to each pixel point, determine the weighted sum of the pixel values ​​and weight values ​​corresponding to multiple neighboring pixel points of each pixel point in the target gradient direction of the respective neighborhood, wherein the weight value corresponding to each neighboring pixel point corresponds one-to-one to the value of the convolution operator corresponding to the target gradient direction; In the case where there are third-type neurons whose ignition state is successfully ignited in the preset neural network, a weighted sum of pixel points corresponding to each of the third-type neurons is determined as a weight coefficient of the pixel points corresponding to each of the third-type neurons, and based on the weight coefficient of the pixel points corresponding to each of the third-type neurons and the enhancement information of the pixel points corresponding to each of the third-type neurons, the enhancement energy of the pixel points corresponding to each of the third-type neurons is determined; In the case where there are fourth-type neurons whose ignition state is ignition failure in the preset neural network, the enhancement energy of the pixel points corresponding to each of the fourth-type neurons is set to zero; The enhanced energy matrix is ​​obtained according to the enhanced energies corresponding to the pixel points corresponding to the third type of neurons and the enhanced energies corresponding to the pixel points corresponding to the fourth type of neurons.

7. The image enhancement method according to claim 1, characterized in that: The step of performing energy enhancement on edge regions and detail regions in the converted image based on the mask energy matrix and the enhanced energy matrix to obtain a target intermediate image comprises: Calculating the product of the mask energy matrix and the enhanced energy matrix, and obtaining an initial intermediate image according to the calculation result, wherein the initial intermediate image only includes images of edge areas and detail areas after energy enhancement in the converted image; The initial intermediate image and the converted image are superimposed to obtain the target intermediate image, wherein the target intermediate image includes images of edge areas and detail areas after energy enhancement in the converted image, and images of flat areas without energy enhancement.

8. An image enhancement device, characterized in that: The device comprises: A first conversion module is used to obtain an original color image and perform color space conversion on the original color image to obtain a converted image; A determination module, used to calculate a gradient matrix and an enhancement information matrix corresponding to the converted image, and determine an enhancement energy matrix corresponding to the converted image based on the gradient matrix and the enhancement information matrix; A filtering module, configured to perform Gaussian filtering on the converted image to obtain a filter image, and obtain a mask energy matrix based on the converted image and the filter image; An energy enhancement module, used for performing energy enhancement on edge areas and detail areas in the converted image based on the mask energy matrix and the enhancement energy matrix to obtain a target intermediate image; The second conversion module is used to perform color space conversion on the target intermediate image to obtain an energy-enhanced color image.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, used to implement the image enhancement method according to any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image enhancement method according to any one of claims 1 to 7 is implemented.

Images