Image restoration method based on memristor array genetic algorithm

Abstract

The invention provides an image restoration method based on a memristor array genetic algorithm, and relates to the technical field of image processing. The method comprises the following steps: segmenting an image, converting a data matrix, and storing image gray value information by using a memristor array; encoding and initializing a population; calculating a fitness value, the process degrading the restored image into a degraded image, and comparing the degraded image with the original degraded image; biological heredity is simulated, in the process, the gray value of the image stored by the memristor is selected, crossed and varied, so that the gray value is converged to an optimal solution; and recovering the image, realizing memristive array iteration updating individuals through a genetic algorithm until a stopping criterion is met, and recovering the image. The method has the advantages that the advantages of storage and calculation integration, high-density array, parallel operation and the like of the memristor are utilized, partitioning processing is carried out on the image, the efficiency and the recovery effect of the algorithm are effectively improved, and the problems that due to the fact that the number of image pixels is large, an initial population is difficult to generate excellent individuals, and the convergence speed is low are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an image restoration method based on a memristor array genetic algorithm, which stores grayscale values ​​of a block image in a memristor array and performs image restoration by operating memristor units according to a genetic algorithm. Background Art

[0002] With the rapid development of information technology, images are increasingly used in various fields, such as the application of image processing technology in medical imaging, graphic printing, etc. However, in the process of image generation, transmission and processing, due to human and natural factors such as camera shake, poor lighting, haze, etc., image blur, distortion, and additional noise often occur, resulting in degradation phenomena such as image quality degradation.

[0003] Existing image restoration methods have some limitations. Traditional linear functions have difficulty handling complex image degradation problems. Although genetic algorithms (GAs) have certain advantages in solving complex nonlinear combinatorial optimization problems, in image restoration applications, due to the large number of image pixels, if they are not properly processed, it will lead to excessive computational burden. For example, when the image is not divided into blocks, for a large-sized image, the number of possible feasible solutions is huge, and it is difficult for the randomly generated initial population to obtain individuals with good characteristics, which leads to slow convergence. Even if the initial population size is increased, there will be huge storage space requirements and the effect of improving algorithm efficiency is not obvious. Summary of the invention

[0004] The purpose of the present invention is to provide an image restoration method based on a memristor array genetic algorithm to solve the problems of complex calculation, low efficiency and poor image degradation processing effect existing in the existing image restoration methods, and to improve the quality and efficiency of image restoration.

[0005] The present invention proposes an image restoration method based on a memristor array genetic algorithm, which stores the grayscale values ​​of the block image in a memristor array and operates the memristor unit according to the genetic algorithm to perform image restoration. A memristor is a circuit device with both storage and computing characteristics. It provides similarities between electronic devices and biological elements such as neurons. The array structure based on the memristor is similar to the chromosome structure in genetics. In addition, it also provides similarities with the image grayscale value matrix, which can be applied to the genetic algorithm for image restoration. At the same time, in the image restoration process, due to the large number of pixels in the image, the amount of calculation will be large, and the image restoration effect is general. Therefore, it is proposed to process the image in blocks.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] The present invention provides an image restoration method based on a memristor array genetic algorithm, comprising the following steps:

[0008] S1: Get the image to be restored, divide the image into image blocks according to pixels, and use an M×N memristor array to store the pixel value of a single image block, where each memristor unit corresponds to a pixel in the image; where M is the initial population size of the genetic algorithm, the first N-1 are the pixel values ​​of the image block, which serve as the genetic information of the individual in the genetic algorithm, and the last column is the individual fitness value in the genetic algorithm; use the memristor array to store the grayscale value of the image;

[0009] S2: Construct an image degradation model; specifically, Represents the coordinates of the spatial domain, defined is the original clear image in the spatial domain, is the degradation function, For noise, is a degraded image; the mathematical expression of the image degradation model is defined as:

[0010] ;

[0011] in Represents the convolution operation;

[0012] S3: Setting the fitness function of the genetic algorithm, which is defined based on the quality of image restoration, specifically:

[0013] ;

[0014] in is the fitness function; The image is restored as a result;

[0015] S4: Initialize the population and the memristor array. Each individual in the population represents a solution for restoring an image block. The initial population is generated randomly or based on existing knowledge, and the pixel values ​​of these individuals are stored in the memristor array.

[0016] S5: Execute the selection operation of the genetic algorithm; select individuals with high fitness as parents according to their fitness values ​​for subsequent crossover and mutation operations. Probability of being selected Defined as:

[0017] ;

[0018] Where n represents the population size, Calculated by fitness function;

[0019] S6: Perform a crossover operation to generate new offspring individuals by exchanging some genes between parent individuals. Mapping to the memristor array means exchanging the resistance values ​​of the corresponding memristor units.

[0020] S7: performing mutation operation, randomly selecting some genes in the offspring individuals for mutation, and mapping them to the memristor array means changing the resistance value of the corresponding memristor unit according to the changed gray value information;

[0021] S8: Repeat steps S5 to S7 until a predetermined number of iterations is reached or other stopping conditions are met.

[0022] S9: Select the individual with the highest fitness from the final population as the optimal solution, that is, the restored image block, and read it out from the memristor array; each image block obtained in step S1 is spliced ​​through steps S4 to S9 to obtain the restored image.

[0023] Furthermore, the image block size is A×B pixels, and the image block is moved from left to right and from top to bottom to traverse the image to be restored; the pixel matrix of the A×B image block is converted into a 1×AB matrix, and finally an M-row AB+1-column memristor array is formed.

[0024] Furthermore, a binary encoding method is used to store grayscale value information; specifically, for grayscale value storage of 0-255, 8 memristor units are used to store one byte of data; it is stipulated that the low resistance state of the memristor unit represents a binary "0", and the high resistance state represents a binary "1";

[0025] Write operation: Change the resistance state of the memristor unit by applying a specific voltage pulse to it. If you want to write "1", apply a positive voltage pulse to increase the resistance of the memristor unit to the state; if you want to write "0", apply a reverse voltage pulse or no voltage pulse to keep or return the memristor unit to the state;

[0026] Read operation: By applying a read voltage to the memristor cell , It must be small enough to avoid changing the storage value of the memristor unit; measure the current passing through the memristor unit, calculate the resistance value according to Ohm's law, and then obtain the storage value of the memristor unit according to the pre-set resistance-storage value correspondence.

[0027] Furthermore, the image restoration result is It is obtained by frequency domain method; specifically, the image is first Fourier transformed to convert the degradation model into frequency domain representation, and then the degradation function and image are estimated by solving the inverse problem in the frequency domain; in the frequency domain, the observed image is represented in the frequency domain as:

[0028] ;

[0029] in, represents the coordinates in the frequency domain, and Respectively represent the frequency components of the image in the horizontal and vertical directions, is the Fourier transform of the degraded image, is the Fourier transform of the degradation function, is the Fourier transform of the original image, is the Fourier transform of the noise;

[0030] Then the image is restored by inverse Fourier transform estimation :

[0031] ;

[0032] in, is the inverse Fourier transform, represents the inverse of the degradation function.

[0033] Furthermore, the crossover operation adopts a single-point crossover method, the crossover point is selected by a random function, and the chromosome is divided into two parts before and after the crossover point according to the position of the selected crossover point. The matching parents exchange gene values ​​before and after the crossover point to generate new offspring individuals.

[0034] Furthermore, the mutation operation adopts bit flipping; firstly, a gene position is randomly selected, the decimal gray value at the gene position is converted into an 8-bit binary code, then 1 bit is randomly selected for change, and finally the changed binary value is converted into a decimal gray value.

[0035] Furthermore, the memristor array includes memristor units, row and column selection lines, and peripheral circuits; each row of memristors is connected together through word lines, and each column of memristors is connected together through bit lines. When the resistance value of a memristor needs to be read, the word line and bit line corresponding to the memristor can be selected to operate it; the peripheral circuit is used to generate a voltage signal applied to the memristor to read the state of the memristor.

[0036] Furthermore, the predetermined number of iterations is set according to the complexity of the image, the initial population size, and the requirements for the restoration effect; other stopping conditions are met, including reaching a certain image restoration quality standard or the fitness value change of the population after multiple consecutive iterations is less than a set threshold.

[0037] The advantage of the present invention is that the block processing of images effectively improves the efficiency and restoration effect of the algorithm, and solves the problems of difficulty in generating excellent individuals and slow convergence speed due to the large number of image pixels in the initial population. At the same time, compared with non-block processing and traditional Gaussian and median filtering methods, the memristor-based genetic algorithm of the present invention has better performance in image restoration. The present invention demonstrates the feasibility of memristors in realizing genetic algorithms for image restoration, provides a reference for the further application of memristors in the field of image processing, and also provides ideas for the implementation of other intelligent algorithms using memristors. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solution of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0039] Figure 1 is a flow chart of the image restoration method based on memristor array genetic algorithm of the present invention;

[0040] Figure 2 is a structural diagram of the memristor array of the present invention;

[0041] Figure 3 is a flow chart of the genetic algorithm of the present invention;

[0042] Figure 4 This is the image restoration result diagram of the control experiment 1 of the present invention with or without image block division;

[0043] Figure 5 This is a graph of image restoration results of control experiment 2 of the present invention with different population sizes and iteration values ​​set;

[0044] Figure 6 3 is an image restoration result diagram using different models in control experiment 3 of the present invention. DETAILED DESCRIPTION

[0045] In order to more clearly illustrate the technical solution of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific examples. The specific implementation of the present invention is not limited to the following examples, and appropriate changes and modifications can be made according to actual needs without affecting the essence and scope of the present invention.

[0046] A genetic algorithm image restoration method based on memristor array, such as Figure 1 As shown, the following steps are included:

[0047] S1. Use a memristor array to store the pixel values ​​of the image, where each memristor unit corresponds to a pixel in the image; specifically, divide the image into blocks by pixels, set the size of each image block to 3×3 pixels, and move the image blocks from left to right and from top to bottom to traverse the image to be restored. Convert the 3×3 block data matrix into a 1×9 matrix. Assuming that the initial population size is M, the two-dimensional matrix formed is M×9. In addition, each individual also needs a space to store the fitness value. The expanded matrix is ​​M×10, and then an M×10 memristor array is obtained. Among them, the first 9 genes (the individual information stored in the first 9 columns) constitute the individual, the number of rows represents the population size, and the last column stores the individual fitness value.

[0048] The memristor array is used to store the grayscale value and individual adaptation value of the image. The grayscale value information is stored in binary encoding. For grayscale value storage of 0-255, 8 memristor units are used to store one byte of data. The low resistance state of the memristor unit is specified to represent binary "0", and the high resistance state represents binary "1". Write operation: Change the resistance state of the memristor unit by applying a specific voltage pulse to the memristor unit. If a "1" is to be written to the memristor, a forward voltage pulse of sufficient intensity is applied to increase the resistance of the memristor unit to the state; if a "0" is to be written, a reverse voltage pulse of appropriate intensity or no voltage pulse can be applied to keep the memristor unit at or return to the state. Read operation: By applying a smaller read voltage to the memristor unit , To be small enough to avoid changing the stored value of the memristor cell, a Measuring the current through the memristor unit, according to Ohm's law Calculate the resistance value, and then according to the preset resistance-storage value correspondence ( Corresponding to binary "0", Corresponding to binary "1"), the storage value of the memristor unit can be obtained.

[0049] The steps of initializing and calibrating the memristor array to ensure that it can accurately store and read the pixel values ​​of the image; the memristor array has a two-dimensional matrix structure and is composed of a plurality of memristor units, each of which can store and represent different pixel values ​​according to the change of its resistance value.

[0050] S2. Construct an image degradation model. Specifically, Represents the coordinates of the spatial domain, defined is the original clear image in the spatial domain, is the degradation function (blur kernel) representing the causes of image degradation such as blur operation and camera shake. For noise, is a degraded image; the mathematical expression of the image degradation model is defined as:

[0051]

[0052] in Represents a convolution operation.

[0053] The goal of image restoration is to estimate the original image from a known degraded image. Specifically, it is to reversely infer the blur kernel and noise in the degradation process and obtain a result as close as possible to the original image through the restoration algorithm.

[0054] The restoration technology used in the present invention is a frequency domain method. First, the image is Fourier transformed to convert the degradation model into a frequency domain representation, and then the degradation function and the restored image are estimated by solving the inverse problem in the frequency domain. In the frequency domain, convolution becomes a point multiplication, and the observed image can be expressed in the frequency domain as:

[0055]

[0056] in, represents the coordinates in the frequency domain, and Respectively represent the frequency components of the image in the horizontal and vertical directions, is the Fourier transform of the degraded image, is the Fourier transform of the degradation function, is the Fourier transform of the original image, is the Fourier transform of the noise.

[0057] Restoring the image in the frequency domain :

[0058]

[0059] in, represents the restored image in the frequency domain; represents the inverse of the degradation function. Substituting , we can get:

[0060]

[0061] Then the image is restored by inverse Fourier transform estimation :

[0062]

[0063] in, is the inverse Fourier transform.

[0064] S3. Setting the fitness function of the genetic algorithm, wherein the fitness function is defined based on the quality of image restoration; specifically, Represents the coordinates of the spatial domain, defined is the original clear image in the spatial domain, is the image restoration result, for Degenerate Function The image obtained after processing is for Degenerate Function The processed image is compared and The similarity of and The more similar, the and The more similar they are, the better the image restoration effect is.

[0065] So the fitness function Defined as:

[0066]

[0067] S4. Initialize the population. Each individual in the population represents a possible image restoration solution. The initial population is generated randomly or based on existing knowledge, and the pixel values ​​of these individuals are stored in the memristor array.

[0068] S5. Execute the selection operation of the genetic algorithm, such as Figure 3 As shown in the figure, individuals with higher fitness are selected as parents according to their fitness values ​​for subsequent crossover and mutation operations. Definition is the individual fitness value, each individual Probability of being selected Defined as:

[0069]

[0070] Here, n represents the population size.

[0071] S6. Perform a crossover operation to generate new offspring individuals by exchanging some genes between parent individuals (i.e., some pixel values ​​in the memristor array). As mentioned above, we designed image block processing, where one image block is an individual. Since the number of genes is small, a single-point crossover is used instead of a complex crossover. The crossover point is selected by a random function. According to the position of the selected crossover point, the chromosome is divided into two parts before and after the crossover point, and the paired parents exchange the gene values ​​before or after the crossover point. This operation is mapped to the memristor array by exchanging the resistance value of the corresponding memristor unit.

[0072] S7. Perform mutation operation, randomly select some genes in the offspring individuals for mutation (i.e., change the value of the corresponding pixel in the memristor array) to increase the diversity of the population. The mutation operation of the present invention adopts bit flipping, first randomly selects a gene position on the chromosome, converts the decimal gray value at the gene position into an 8-bit binary code, then randomly selects 1 bit for change, and finally converts the changed binary value into a decimal gray value. This operation is mapped to the memristor array to change the resistance value of the corresponding memristor unit according to the changed gray value information.

[0073] S8. Repeat steps S5 to S7 until a predetermined number of iterations is reached or other stop conditions are met. The predetermined number of iterations can be set according to factors such as the complexity of the image, the size of the initial population, and the requirements for the restoration effect. Other stop conditions may include reaching a certain image restoration quality standard or the fitness value of the population changing less than a set threshold after multiple consecutive iterations.

[0074] S9. Select the individual with the highest fitness from the final population as the optimal solution, i.e., the restored image block, and read it out from the memristor array. Perform the above steps for each image block, and splice the final image blocks to obtain the restored image.

[0075] Figure 2 is a structural diagram of the memristor array of the present invention. As shown in the figure, the structure includes the following parts:

[0076] The memristor unit is used to store the grayscale value of the image pixel, and the memory characteristic of the current-voltage relationship of the memristor is used to express the change of the grayscale value of the image pixel;

[0077] The row and column selection lines connect each row of memristors through word lines, and each column of memristors through bit lines. When the resistance value of a memristor needs to be read, it can be operated by selecting the word line and bit line corresponding to the memristor.

[0078] The peripheral circuit is used to generate a voltage signal applied to the memristor and read the state of the memristor.

[0079] Figure 4 This is the image restoration result diagram of the control experiment of the present invention with or without image block. As shown in the figure, the image restoration effect is better after the image is divided into blocks. Since there are many pixels and a wide range of pixels in the image without block processing, it is difficult to obtain good individuals by randomly generating the initial population. Therefore, the convergence speed in the later stage is very slow, and after the same number of iterations, the individual performance is still very poor. The image after the unblocked image restoration is less likely to obtain more information.

[0080] Figure 5This is the image restoration result diagram of the control experiment 2 of the present invention with different population sizes and iteration values. As shown in the figure, the figure shows that when the population size and iteration value reach a certain scale, the image restoration effect is better, which proves the feasibility of deploying genetic algorithms and memristors in image restoration.

[0081] Figure 6 This is the image restoration result diagram of the control experiment 3 of the present invention using different models. As shown in the figure, the figure shows that the image restoration effect is not good when the filtering operation is directly performed on the image, while the model proposed by the present invention has better effect.

[0082] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. An image restoration method based on a memristor array genetic algorithm, characterized in that: The following steps are involved: S1: Get the image to be restored, divide the image into image blocks according to pixels, and use an M×N memristor array to store the pixel value of a single image block, where each memristor unit corresponds to a pixel in the image; where M is the initial population size of the genetic algorithm, the first N-1 are the pixel values ​​of the image block, which serve as the genetic information of the individual in the genetic algorithm, and the last column is the individual fitness value in the genetic algorithm; use the memristor array to store the grayscale value of the image; S2: Construct an image degradation model; specifically, Represents the coordinates of the spatial domain, defined is the original clear image in the spatial domain, is the degradation function, For noise, is a degraded image; the mathematical expression of the image degradation model is defined as: ； in Represents the convolution operation; S3: Setting the fitness function of the genetic algorithm, which is defined based on the quality of image restoration, specifically: ； in is the fitness function; The result is image restoration; S4: Initialize the population and the memristor array. Each individual in the population represents a solution for restoring an image block. The initial population is generated randomly or based on existing knowledge, and the pixel values ​​of these individuals are stored in the memristor array. S5: Execute the selection operation of the genetic algorithm; select individuals with high fitness as parents according to their fitness values ​​for subsequent crossover and mutation operations. Probability of being selected Defined as: ； Where n represents the population size, Calculated by fitness function; S6: Perform a crossover operation to generate new offspring individuals by exchanging some genes between parent individuals. Mapping to the memristor array means exchanging the resistance values ​​of the corresponding memristor units. S7: performing mutation operation, randomly selecting some genes in the offspring individuals for mutation, and mapping them to the memristor array means changing the resistance value of the corresponding memristor unit according to the changed gray value information; S8: Repeat steps S5 to S7 until a predetermined number of iterations is reached or other stop conditions are met; S9: Select the individual with the highest fitness from the final population as the optimal solution, that is, the restored image block, and read it out from the memristor array; each image block obtained in step S1 is spliced ​​through steps S4 to S9 to obtain the restored image.

2. The image restoration method based on memristor array genetic algorithm according to claim 1, characterized in that: The image block size is A×B pixels, and the image block is moved from left to right and from top to bottom to traverse the image to be restored; the pixel matrix of the A×B image block is converted into a 1×AB matrix, and finally an M-row AB+1-column memristor array is formed.

3. The image restoration method based on memristor array genetic algorithm according to claim 1, characterized in that: Grayscale value information is stored in binary encoding. Specifically, for grayscale value storage of 0-255, 8 memristor units are used to store one byte of data. The low resistance state of the memristor unit represents binary "0", and the high resistance state represents binary "1". Write operation: Change the resistance state of the memristor unit by applying a specific voltage pulse to it; if "1" is to be written, a positive voltage pulse is applied to increase the resistance of the memristor unit to the state; if "0" is to be written, a reverse voltage pulse is applied or no voltage pulse is applied to keep or return the memristor unit to the state; Read operation: By applying a read voltage to the memristor cell , It must be small enough to avoid changing the storage value of the memristor unit; measure the current passing through the memristor unit, calculate the resistance value according to Ohm's law, and then obtain the storage value of the memristor unit according to the preset resistance-storage value correspondence.

4. The image restoration method based on memristor array genetic algorithm according to claim 1, characterized in that: The above is the image restoration result It is obtained by frequency domain method; specifically, the image is first Fourier transformed to convert the degradation model into frequency domain representation, and then the degradation function and image are estimated by solving the inverse problem in the frequency domain; in the frequency domain, the observed image is represented in the frequency domain as: ； in, represents the coordinates in the frequency domain, and Respectively represent the frequency components of the image in the horizontal and vertical directions, is the Fourier transform of the degraded image, is the Fourier transform of the degradation function, is the Fourier transform of the original image, is the Fourier transform of the noise; Then the image is restored by inverse Fourier transform estimation : ； in, is the inverse Fourier transform, represents the inverse of the degradation function.

5. The image restoration method based on memristor array genetic algorithm according to claim 1, characterized in that: The crossover operation adopts a single-point crossover method, and the crossover point is selected by a random function. According to the position of the selected crossover point, the chromosome is divided into two parts before and after the crossover point, and the matching parents exchange the gene values ​​before and after the crossover point to generate new offspring individuals.

6. The image restoration method based on memristor array genetic algorithm according to claim 1, characterized in that: The mutation operation adopts bit flipping; firstly, a gene position is randomly selected, the decimal gray value at the gene position is converted into an 8-bit binary code, then 1 bit is randomly selected for change, and finally the changed binary value is converted into a decimal gray value.

7. The image restoration method based on memristor array genetic algorithm according to claim 1, characterized in that: The memristor array includes memristor units, row and column selection lines, and peripheral circuits; each row of memristors is connected together through a word line, and each column of memristors is connected together through a bit line; when the resistance value of a memristor needs to be read, the word line and bit line corresponding to the memristor are selected to operate it; the peripheral circuit is used to generate a voltage signal applied to the memristor to read the state of the memristor.

8. The image restoration method based on memristor array genetic algorithm according to claim 1, characterized in that: The predetermined number of iterations is set according to the complexity of the image, the initial population size and the requirements for the restoration effect; other stopping conditions are met, including reaching a certain image restoration quality standard or the fitness value change of the population after multiple consecutive iterations is less than a set threshold.

Images