An image inpainting method based on evolutionary algorithm

Abstract

The application discloses an image repairing method based on an evolution algorithm. The application realizes image repairing by using an improved evolution algorithm. In the improved evolution algorithm, a parameter of an image repairing model is optimized and designed by using a harmony search, a fusion factor is adaptively generated, information of an optimal candidate solution is used to guide a search direction of the algorithm based on the fusion factor, a new candidate solution is generated, and the new candidate solution is subjected to a superior and inferior state. The application uses the information of the optimal candidate solution to guide the search direction of the algorithm, improves optimization performance of the algorithm, and thus improves an image repairing effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of digital image processing, and in particular to an image inpainting method based on evolutionary algorithm. BACKGROUND

[0002] Image inpainting is an important research direction in the field of digital image processing, which has wide applications in engineering practice, such as photo beautification, image document analysis and cultural relic restoration. In image inpainting, the parameters of image inpainting model often have a great influence on the effect of image inpainting. However, there is no perfect theoretical guidance for the setting of the parameters of image inpainting model. Therefore, technicians often need to repeatedly try to set different parameters of image inpainting model to improve the effect of image inpainting.

[0003] In order to more effectively set the parameters of image inpainting model, researchers try to use evolutionary algorithm to optimize the design of the parameters of image inpainting model. Harmony search algorithm is a superior evolutionary algorithm, which has obtained good optimization effect in many engineering optimization problems. However, the traditional harmony search algorithm applied to digital image inpainting has the defect that the effect of image inpainting needs to be improved. SUMMARY

[0004] The present application provides an image inpainting method based on evolutionary algorithm. It overcomes the defect that the traditional harmony search algorithm applied to digital image inpainting has the defect that the effect of image inpainting needs to be improved to some extent, and the present application can improve the effect of image inpainting.

[0005] The technical scheme of the present application: an image inpainting method based on evolutionary algorithm, comprising the following steps:

[0006] Step 1, input the image IM to be inpainted;

[0007] Step 2, input the pixel set RSet to be inpainted in the image IM;

[0008] Step 3, add the pixels in the image IM which do not belong to the pixel set RSet to be inpainted to the pixel set NSet;

[0009] Step 4, add the eight neighborhood pixels of each pixel in the pixel set RSet to be inpainted in the image IM to the pixel set HSet;

[0010] Step 5, record the intersection of the pixel set NSet and the pixel set HSet as the boundary pixel set ASet;

[0011] Step 6, add the eight neighborhood pixels of each pixel in the boundary pixel set ASet in the image IM to the pixel set TMSet;

[0012] Step 7: Denote the intersection of pixel set NSet and pixel set TMSet as pixel set TBSet;

[0013] Step 8: Add the pixels in image IM that belong to pixel set TBSet but not to neighboring pixel set ASet to the training pixel set BSet;

[0014] Step 9: Set the harmony memory size NP, the number of optimization parameters Dn, the maximum update generation MaxG, and the harmony memory value probability HMCR;

[0015] Step 10, set the current update generation G = 0;

[0016] Step 11: Randomly generate NP harmonic components to form a harmonic memory HM = {H1, H2, ..., H...} pi ,...,H NP}, where H pi ={H pi,1 H pi,2 ,...,H pi,dj ,...,H pi,Dn} represents the pi-th harmony in the harmony memory; harmony subscripts pi = 1, 2, ..., NP; each harmony in the harmony memory contains Dn optimization parameters; H pi,dj This represents the dj-th optimization parameter of the pi-th harmony in the harmony memory, with dimension subscripts dj = 1, 2, ..., Dn;

[0017] Step 12, calculate the fitness value (Fit) for each harmony in the harmony memory bank. pi ;

[0018] Step 13: Obtain the harmony with the smallest fitness value in the harmony memory bank, and denote it as the optimal harmony GBH;

[0019] Step 14: Obtain the harmony with the highest fitness value in the harmony memory bank, and denote it as the worst harmony GWH;

[0020] Step 15: If the current update generation G is less than the maximum update generation MaxG, then proceed to step 16; otherwise, proceed to step 26.

[0021] Step 16: Calculate the fusion factor SF according to formula (1);

[0022]

[0023] Where log2 is a logarithmic function with base 2;

[0024] Step 17, set the dimension variable tj = 1;

[0025] Step 18, generate a random real number rd between [0, 1], if rd is less than the harmony memory utilization probability HMCR, go to step 19, otherwise go to step 21;

[0026] Step 19, generate the tjth optimization parameter NH of the new harmony NH according to formula (2) tj :

[0027] NH tj = SF x GBH tj + (1 - SF) x H ri,tj + r1 x (GBH tj - GWH tj ) (2)

[0028] wherein, r1 is a random real number between [0, 1], GBH tj is the tjth optimization parameter in the optimal harmony GBH, GWH tj is the tjth optimization parameter in the worst harmony GWH, ri is a random integer between [1, PS], H ri,tj is the tjth optimization parameter of the ri th harmony in the harmony memory;

[0029] Step 20, go to step 22;

[0030] Step 21, generate the tjth optimization parameter NH of the new harmony NH according to formula (3) tj :

[0031] NH tj = XL tj + r2 x (XU tj - XL tj ) (3)

[0032] wherein, r2 is a random real number between [0, 1], XL tj is the lower limit of the value of the tjth optimization parameter, XU tj is the upper limit of the value of the tjth optimization parameter;

[0033] Step 22, set the dimension variable tj = tj + 1, if the dimension variable tj is less than the number of optimization parameters Dn, go to step 18, otherwise go to step 23;

[0034] Step 23, calculate the fitness value NFIT of the new harmony NH;

[0035] Step 24, if the fitness value of the new harmony NH is less than the fitness value of the worst harmony GWH, replace the worst harmony GWH with the new harmony NH, otherwise discard the new harmony NH;

[0036] Step 25, set the current update algebra G = G + 1, go to step 13;

[0037] Step 26, store the Dn optimization parameters in the optimal harmony GBH into the list GP;

[0038] Step 27, calculate the repair pixel value of all pixels in the repair pixel set RSet according to formula (4):

[0039]

[0040] wherein, GP dj is the djth optimization parameter in the list GP; DIS represents a function of calculating the distance between two pixels; AV hi is the pixel value of the hi th pixel in the adjacent boundary pixel set ASet; AX hi is the coordinate value of the hi th pixel in the adjacent boundary pixel set ASet, subscript hi = 1, 2, …, HL, and HL is the size of the adjacent boundary pixel set ASet; AX mi is the coordinate value of the mi th pixel in the adjacent boundary pixel set ASet, subscript mi = 1, 2, …, HL; PRV ki is the repair pixel value of the ki th pixel in the repair pixel set RSet; PX ki is the coordinate value of the ki th pixel in the repair pixel set RSet, subscript ki = 1, 2, …, KL, and KL is the size of the repair pixel set RSet;

[0041] Step 28, realize the repair of the image IM by using the repair pixel values of all pixels in the repair pixel set RSet obtained.

[0042] Further, the calculation process of the adaptive value is:

[0043] For the pi th harmony H pi in the harmony memory, store the Dn optimization parameters in the harmony H pi into the list SR; calculate the predicted pixel value of each pixel in the adjacent boundary pixel set ASet according to formula (5):

[0044]

[0045] wherein, SR dj is the djth optimization parameter in the list SR; BV ji is the pixel value of the ji th pixel in the training pixel set BSet; BX ji is the coordinate value of the ji th pixel in the training pixel set BSet, subscript ji = 1, 2, …, JL, and JL is the size of the training pixel set BSet; BXsi is the coordinate value of the si th pixel in the training pixel set BSet, subscript si = 1, 2, …, JL;

[0046] Harmony H is calculated according to formula (6) pi : pi :

[0047]

[0048] Wherein, PAV hi is the predicted pixel value of the hi th pixel in the boundary pixel set ASet.

[0049] The application discloses an image repairing method based on an evolutionary algorithm. The improved evolutionary algorithm is used to realize image repairing. In the improved evolutionary algorithm, a harmony search is used to optimize and design parameters of an image repairing model. On one hand, an adaptive fusion factor is proposed, beneficial information in a harmony memory is adaptively fused into an optimization process along with an evolutionary state, the algorithm is enabled to maintain diversity of solutions and improve local optimization capability, and on the other hand, information of an optimal harmony in the harmony memory is used to enhance optimization capability of the algorithm, thereby improving an image repairing effect. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a schematic diagram of training pixels, pixels to be repaired and boundary pixels.

[0051] Figure 2 is an image to be repaired in the embodiment.

[0052] Figure 3 is an image repaired by using the application. DETAILED DESCRIPTION

[0053] The technical scheme of the application is further specifically described below by means of embodiments and in combination with the drawings.

[0054] Embodiment:

[0055] In combination with the drawings, the specific implementation steps of the application are as follows:

[0056] Step 1, input an image IM to be repaired as shown in Figure 2 .

[0057] Step 2, input a pixel set RSet to be repaired in the image IM; a schematic diagram of the pixel set RSet to be repaired is as shown in Figure 1 .

[0058] Step 3, add pixels in the image IM not belonging to the pixel set RSet to be repaired to a pixel set NSet.

[0059] Step 4, add the eight-neighborhood pixels of each pixel in the pixel set RSet to be repaired in the image IM to the pixel set HSet;

[0060] Step 5, record the intersection of the pixel set NSet and the pixel set HSet as the adjacent boundary pixel set ASet; a schematic diagram of the adjacent boundary pixel set ASet is shown in Figure 1 ;

[0061] Step 6, add the eight-neighborhood pixels of each pixel in the adjacent boundary pixel set ASet in the image IM to the pixel set TMSet;

[0062] Step 7, record the intersection of the pixel set NSet and the pixel set TMSet as the pixel set TBSet;

[0063] Step 8, add the pixels in the image IM belonging to the pixel set TBSet and not belonging to the adjacent boundary pixel set ASet to the training pixel set BSet; a schematic diagram of the training pixel set BSet is shown in Figure 1 ;

[0064] Step 9, set the harmony memory size NP=10, the number of optimization parameters Dn=4, the maximum number of update generations MaxG=15, and the harmony memory value probability HMCR=0.95;

[0065] Step 10, set the current number of update generations G=0;

[0066] Step 11, randomly generate NP harmony groups to form a harmony memory HM={H1, H2,...,H pi ,...,H NP}, where H pi ={H pi,1 ,H pi ,2,...,H pi,dj ,...,H pi,Dn} represents the pi-th harmony in the harmony memory; the harmony subscript pi=1, 2,..., NP; each harmony in the harmony memory contains Dn optimization parameters; H pi,dj represents the dj-th optimization parameter of the pi-th harmony in the harmony memory, and the dimension subscript dj=1, 2,..., Dn;

[0067] Step 12, calculate the fitness value Fit pi of each harmony in the harmony memory;

[0068] Step 13, obtain the harmony with the minimum fitness value in the harmony memory, and record it as the optimal harmony GBH;

[0069] Step 14, the harmony memory bank with the largest fitness value is obtained and recorded as the worst harmony GWH;

[0070] Step 15, if the current update generation G is less than the maximum update generation MaxG, go to step 16, otherwise go to step 26;

[0071] Step 16, calculate the fusion factor SF according to formula (1);

[0072]

[0073] Wherein, log2 is the logarithmic function with base 2;

[0074] Step 17, set the dimension variable tj=1;

[0075] Step 18, generate a random real number rd between [0, 1], if rd is less than the harmony memory bank value probability HMCR, go to step 19, otherwise go to step 21;

[0076] Step 19, generate the tjth optimization parameter NH tj of the new harmony NH according to formula (2);

[0077] NH tj = SF x GBH tj + (1-SF) x H ri,tj + r1 x (GBH tj -GWH tj ) (2)

[0078] Wherein, r1 is a random real number between [0, 1], GBH tj is the tjth optimization parameter in the optimal harmony GBH, GWH tj is the tjth optimization parameter in the worst harmony GWH, ri is a random integer between [1, PS], H ri,tj is the tjth optimization parameter of the ri th harmony in the harmony memory bank;

[0079] Step 20, go to step 22;

[0080] Step 21, generate the tjth optimization parameter NH tj of the new harmony NH according to formula (3);

[0081] NH tj = XL tj + r2 x (XU tj -XL tj ) (3)

[0082] Wherein, r2 is a random real number between [0, 1], XL tjis the lower limit of the value of the tjth optimization parameter, XU tj is the upper limit of the value of the tjth optimization parameter;

[0083] Step 22, set the dimension variable tj = tj + 1, if the dimension variable tj is less than the number of optimization parameters Dn, go to step 18, otherwise go to step 23;

[0084] Step 23, calculate the fitness value NFIT of the new harmony NH;

[0085] Step 24, if the fitness value of the new harmony NH is less than the fitness value of the worst harmony GWH, replace the worst harmony GWH with the new harmony NH, otherwise discard the new harmony NH;

[0086] Step 25, set the current update generation G = G + 1, go to step 13;

[0087] Step 26, store the Dn optimization parameters in the optimal harmony GBH into the list GP;

[0088] Step 27, calculate the inpainting pixel value of all pixels in the set RSet according to formula (4):

[0089]

[0090] wherein, GP dj is the djth optimization parameter in the list GP; DIS represents a function of calculating the distance between two pixels; AV hi is the pixel value of the hi th pixel in the set ASet; AX hi is the coordinate value of the hi th pixel in the set ASet, subscript hi = 1, 2, …, HL, and HL is the size of the set ASet; AX mi is the coordinate value of the mi th pixel in the set ASet, subscript mi = 1, 2, …, HL; PRV ki is the inpainting pixel value of the ki th pixel in the set RSet; PX ki is the coordinate value of the ki th pixel in the set RSet, subscript ki = 1, 2, …, KL, and KL is the size of the set RSet;

[0091] Step 28, implement the inpainting of the image IM using the obtained inpainting pixel values of all pixels in the set RSet, and the inpainted image is as shown in Figure 3 .

[0092] In the embodiment, the calculation process of the fitness value is as follows:

[0093] For the pi-th harmony H in the harmony memory pi , the Dn optimization parameters in the harmony H pi are stored into the list SR; the predicted pixel value of each pixel in the neighboring boundary pixel set ASet is calculated according to the formula (5):

[0094]

[0095] wherein, SR dj is the dj-th optimization parameter in the list SR; BV ji is the pixel value of the ji-th pixel in the training pixel set BSet; BX ji is the coordinate value of the ji-th pixel in the training pixel set BSet, the subscript ji = 1, 2, …, JL, and JL is the size of the training pixel set BSet; BX si is the coordinate value of the si-th pixel in the training pixel set BSet, the subscript si = 1, 2, …, JL;

[0096] The fitness value Fit pi of the harmony H pi is calculated according to the formula (6):

[0097]

[0098] wherein, PAV hi is the predicted pixel value of the hi-th pixel in the neighboring boundary pixel set ASet.

Claims

1. An image inpainting method based on an evolutionary algorithm, characterized in that, Includes the following steps: Step 1, input the image to be repaired (IM); Step 2: Input the set of pixels to be repaired, RSet, from the image IM; Step 3: Add pixels in image IM that do not belong to the pixel set RSet to be repaired to the pixel set NSet; Step 4: Add the eight neighboring pixels of each pixel in the pixel set RSet to be repaired in the image IM to the pixel set HSet; Step 5: Denote the intersection of pixel set NSet and pixel set HSet as the neighboring pixel set ASet; Step 6: Add the eight neighboring pixels in the image IM to the pixel set TMSet for each pixel in the neighboring pixel set ASet; Step 7: Denote the intersection of pixel set NSet and pixel set TMSet as pixel set TBSet; Step 8: Add the pixels in image IM that belong to pixel set TBSet but not to neighboring pixel set ASet to the training pixel set BSet; Step 9: Set the harmony memory size NP, the number of optimization parameters Dn, the maximum update generation MaxG, and the harmony memory value probability HMCR; Step 10, set the current update generation G = 0; Step 11: Randomly generate NP harmonic components to form a harmonic memory HM = {H1, H2, ..., H...} pi ,...,H NP }, where H pi ={H pi,1 H pi ,2,...,H pi,dj ,...,H pi,Dn } represents the pi-th harmony in the harmony memory; harmony subscripts pi = 1, 2, ..., NP; each harmony in the harmony memory contains Dn optimization parameters; H pi,dj This represents the dj-th optimization parameter of the pi-th harmony in the harmony memory, with dimension subscripts dj = 1, 2, ..., Dn; Step 12, calculate the fitness value (Fit) for each harmony in the harmony memory bank. pi ; Step 13: Obtain the harmony with the smallest fitness value in the harmony memory bank, and denote it as the optimal harmony GBH; Step 14: Obtain the harmony with the highest fitness value in the harmony memory bank, and denote it as the worst harmony GWH; Step 15: If the current update generation G is less than the maximum update generation MaxG, then proceed to step 16; otherwise, proceed to step 26. Step 16: Calculate the fusion factor SF according to formula (1); Where log2 is a logarithmic function with base 2; Step 17, set the dimension variable tj = 1; Step 18: Generate a random real number rd between [0, 1]. If rd is less than the probability of taking a value from the harmony memory bank (HMCR), proceed to step 19; otherwise, proceed to step 21. Step 19: Generate the tjth optimized parameter NH of the new harmony NH according to formula (2). tj : NH tj =SF×GBH tj +(1-SF)×H ri,tj +r1×(GBH tj -GWH tj (2) Where r1 is a random real number between [0, 1], GBH tj It is the tjth optimization parameter in the optimal harmony GBH, GWH tj Let be the tjth optimization parameter in the worst harmony GWH, where ri is a random integer between [1, PS], and H ri,tj It is the tjth optimization parameter of the rith harmony in the harmony memory; Step 20, proceed to step 22; Step 21: Generate the tjth optimized parameter NH of the new harmony NH according to formula (3). tj : NH tj =XL tj +r2×(XU tj -XL tj ) (3) Where r2 is a random real number between [0, 1], and XL tj It is the lower bound of the value of the tj-th optimization parameter, XU tj It is the upper limit of the value of the tj-th optimization parameter; Step 22: Set the dimension variable tj = tj + 1. If the dimension variable tj is less than the number of optimization parameters Dn, go to step 18; otherwise, go to step 23. Step 23, calculate the adaptation value NFIT of the new harmony NH; Step 24: If the fitness value of the new harmony NH is less than the fitness value of the worst harmony GWH, then replace the worst harmony GWH with the new harmony NH; otherwise, discard the new harmony NH. Step 25: Set the current update generation G = G+1, then proceed to step 13; Step 26: Store the Dn optimized parameters in the optimal harmony GBH into the list GP; Step 27: Calculate the repair pixel values ​​for all pixels in the pixel set RSet to be repaired according to formula (4): Among them, GP dj It is the dj-th optimization parameter in the list GP; DIS represents the function that calculates the distance between two pixels; AV hi It is the pixel value of the hi-th pixel in the neighboring pixel set ASet; AX hi AX is the coordinate value of the hi-th pixel in the neighboring pixel set ASet, where the index hi = 1, 2, ..., HL, and HL is the size of the neighboring pixel set ASet; mi It is the coordinate value of the mi-th pixel in the neighboring pixel set ASet, with subscripts mi = 1, 2, ..., HL; PRV ki It is the repair pixel value of the ki-th pixel in the pixel set RSet to be repaired; PX ki It is the coordinate value of the ki-th pixel in the pixel set RSet to be repaired, with subscript ki = 1, 2, ..., KL, and KL is the size of the pixel set RSet to be repaired; Step 28: Use the repair pixel values ​​of all pixels in the obtained pixel set RSet to be repaired to repair the image IM.

2. The image inpainting method based on an evolutionary algorithm according to claim 1, characterized in that, The calculation process for the fitness value is as follows: For the pi-th harmony H in the harmony memory pi , will harmonize H pi The Dn optimization parameters are stored in the list SR; the predicted pixel value of each pixel in the neighboring pixel set ASet is calculated according to formula (5): Among them, SR dj It is the dj-th optimization parameter in the list SR; BV ji It is the pixel value of the ji-th pixel in the training pixel set BSet; BX ji BX is the coordinate value of the ji-th pixel in the training pixel set BSet, with indices ji = 1, 2, ..., JL, and JL being the size of the training pixel set BSet; si It is the coordinate value of the si-th pixel in the training pixel set BSet, with the index si = 1, 2, ..., JL; The harmonic H is calculated according to formula (6). pi Fit pi : Among them, PAV hi Let be the predicted pixel value of the hi-th pixel in the neighboring pixel set ASet.

Images