Image restoration method and system suitable for anchor frame interference

Abstract

The invention provides an image restoration method and system suitable for anchor frame interference, and relates to the technical field of image restoration, and the method comprises the steps: obtaining a first image which is a to-be-restored anchor frame interference image; a color difference value between each pixel of the first image and a preset anchor frame color is calculated, anchor frame areas are extracted according to the color difference values, a plurality of anchor frame areas are obtained, and each anchor frame area corresponds to one anchor frame; and calculating a corrected pixel value of each pixel point in the first image corresponding to the anchor frame area based on a linear fitting method to obtain an image without anchor frame interference. According to the method, the anchor frame area of the to-be-repaired anchor frame interference image is accurately detected according to the preset anchor frame color, and then the corrected pixel values of the pixel points of the to-be-repaired anchor frame interference image in the anchor frame area are calculated, so that high-quality repair of the anchor frame area is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image restoration, and in particular, to an image restoration method and system applicable to anchor box interference. Background Art

[0002] In recent years, image recognition technology has become a research hotspot in the operation and maintenance of railway infrastructure. Taking the railway power supply safety detection and monitoring system (6C system) as an example, this system collects catenary component images through high-definition camera devices to monitor the operation status of the catenary, reducing the burden of manual inspection. However, the image data still needs to be manually analyzed, and there are problems such as inconsistent interpretation criteria, missed detection, and misdetection. Combining artificial intelligence technology with railway operation and maintenance scenarios can not only improve the accuracy of judging facility faults and defects, but also evaluate the health status of various railway system devices, thereby strengthening the safety guarantee of railways.

[0003] With the development of artificial intelligence, railway image intelligent recognition methods have gradually matured, aiming to assist in discriminating various defects, defects and potential safety hazards, and improving efficiency. Taking the 6C system as an example, in the first stage, traditional image processing algorithms are used to manually design and extract features through a large amount of prior knowledge to achieve fault detection; in the second stage, traditional machine learning algorithms are adopted to improve the detection accuracy and speed; in the third stage, based on deep convolutional neural networks, features are automatically extracted to improve the detection accuracy and model applicability.

[0004] Deep convolutional neural networks require a large amount of sample data, especially images containing various fault types. However, due to the low occurrence frequency of some faults and the difficulty of reproduction, it is necessary to rely on the historical image samples of the railway department. These historical images often contain manually circled anchor boxes, and repairing these anchor box areas is the key to improving the generalization ability of the deep learning model. Currently, common background color filling or adjacent color repair is used, but when the thickness of the anchor box lines is large, these repair methods will cause obvious differences between the repaired image and the original image, accompanied by problems such as edge blurring, damage and fault. How to effectively repair anchor box interference and restore image details is an urgent problem to be solved currently. Summary of the Invention

[0005] The purpose of the present invention is to provide an image restoration method and system applicable to anchor box interference to improve the above problems. To achieve the above purpose, the technical solutions adopted by the present invention are as follows:

[0006] In the first aspect, the present application provides an image restoration method applicable to anchor box interference, including:

[0007] Obtain a first image, where the first image is an anchor box interference image to be restored;

[0008] Calculate the color difference between each pixel of the first image and the preset anchor box color, and extract the anchor box regions according to the color difference to obtain a plurality of the anchor box regions, with each of the anchor box regions corresponding to an anchor box;

[0009] Calculate the corrected pixel values of each pixel point in the first image corresponding to the anchor box region based on the linear fitting method to obtain an image without anchor box interference.

[0010] In a second aspect, the present application also provides an image restoration system applicable to anchor box interference, including:

[0011] An acquisition module, configured to acquire a first image, where the first image is an anchor box interference image to be restored;

[0012] A first processing module, configured to calculate the color difference between each pixel of the first image and the preset anchor box color, and extract the anchor box regions according to the color difference to obtain a plurality of the anchor box regions, with each of the anchor box regions corresponding to an anchor box;

[0013] A second processing module, configured to calculate the corrected pixel values of each pixel point in the first image corresponding to the anchor box region based on the linear fitting method to obtain an image without anchor box interference.

[0014] The beneficial effects of the present invention are:

[0015] The present invention first accurately detects the anchor box regions of the anchor box interference image to be restored according to the preset anchor box color, and then calculates the corrected pixel values of the pixel points in the anchor box regions of the anchor box interference image to be restored, so as to achieve high-quality restoration of the anchor box regions.

[0016] Other features and advantages of the present invention will be described in the subsequent specification, and part of them will become obvious from the specification, or can be understood by implementing the embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic flowchart of an image restoration method applicable to anchor box interference described in an embodiment of the present invention;

[0019] Figure 2 It is a schematic structural diagram of an image restoration system applicable to anchor box interference described in an embodiment of the present invention;

[0020] Figure 3 Schematic diagram of the division of the anchor box area described in the embodiments of the present invention;

[0021] Figure 4 Schematic diagram of the image restoration effect of the anchor box interference described in the embodiments of the present invention.

[0022] Reference numerals in the figure: 901, acquisition module; 902, first processing module; 903, second processing module; 9021, first processing unit; 9022, second processing unit; 9023, third processing unit; 9024, fourth processing unit; 9031, fifth processing unit; 9032, sixth processing unit; 9033, seventh processing unit; 9034, eighth processing unit; 90331, first processing sub-module; 90332, second processing sub-module; 90333, third processing sub-module; 90341, fourth processing sub-module; 90342, fifth processing sub-module; 90343, sixth processing sub-module; 90344, seventh processing sub-module; 90321, eighth processing sub-module; 90322, ninth processing sub-module; 90323, tenth processing sub-module. Detailed implementation manners

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Components of the embodiments of the present invention described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0024] It should be noted that: similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used for descriptive distinction and cannot be construed as indicating or implying relative importance.

[0025] Embodiment 1:

[0026] This embodiment provides an image restoration method applicable to anchor box interference.

[0027] See Figure 1 , which shows that this method includes steps S1, S2, and S3.

[0028] S1. Obtain a first image, where the first image is an anchor box interference image to be repaired;

[0029] S2. Calculate the color difference value between each pixel of the first image and a preset anchor box color, and extract the anchor box regions according to the color difference value to obtain a plurality of the anchor box regions. Each anchor box region corresponds to an anchor box. The accurate detection of the anchor box region is a prerequisite for the reliable operation of the image repair system;

[0030] Specifically, step S2 includes:

[0031] S21. Generate a color difference matrix according to the color difference value. The position of the elements of the color difference matrix corresponds one by one to the position of the pixel points of the first image, and the element value of the color difference matrix is the color difference value between the pixel of the first image at the corresponding position and the preset anchor box color. Among them, the same anchor box is generally drawn by a single-color line. Since the anchor box often covers the original image in a semi-transparent manner, and the compression storage of the image will also cause the anchor box color to be rendered around, the pixel color values in the anchor box region of the image to be repaired are not exactly the same. Therefore, in order to suppress the influence of different original image backgrounds, it is necessary to compare the similarity of different color values at the same scale.

[0032] Specifically, in this embodiment, record a set of color values corresponding to a pixel point as c = [r, g, b], where r, g, and b are the red, green, and blue values respectively, and the value range is an integer between 0 - 255, and then calculate the maximum value v max and the minimum value v min , and the calculation formula is:

[0033]

[0034] where, v max and v min are respectively the maximum value and the minimum value in the color components, and r, g, and b are the red, green, and blue values in the color components;

[0035] After obtaining the maximum value v max and the minimum value v min in a set of color values c = [r, g, b], define a color normalization transformation function T(c). The color normalization transformation function T(c) is used to unify the color scale. The color normalization transformation function T(c) is:

[0036]

[0037] Among them, c' = [r', g', b'] is the output value of the color normalization transformation function T(c), c = [r, g, b] is a set of color values before color normalization, r, g, and b are the red, green, and blue values in the color components before color normalization respectively, c' = [r', g', b'] is a set of color values after color normalization, and r', g', and b' are the red, green, and blue values in the color components after color normalization respectively, v max and v min are the maximum and minimum values of the color components in c = [r, g, b] respectively, θ is the amplification factor, and θ max is the amplification factor threshold. The amplification factor θ is used to normalize the maximum value of the color components to 255, and the amplification factor threshold θ max is used to suppress the amplification of slightly colored areas. The smaller the amplification factor threshold θ max , the higher the suppression degree. In this embodiment, the value is 15.

[0038] In the above formula, if the maximum value v max of the color components in a set of color values c = [r, g, b] is equal to the minimum value v min , it indicates that this point is not the pixel point corresponding to the anchor box area. Set this set of color values to 0 for subsequent amplification of the color difference to facilitate distinguishing whether it is the anchor box area; if not equal, based on the amplification factor θ, adjust the maximum value v max in the color value c = [r, g, b] to 255, adjust the minimum value v min to 0, and adjust the remaining components proportionally to achieve the purpose of color normalization and then unify the color scale.

[0039] After color normalization, calculate the color difference between any two sets of color values according to the color normalization transformation function T(c). The color difference calculation formula is:

[0040] g(c 1 , c 2 ) = |T(c 1 ) - T(c 2 )| = |r′ 1 - r′ 2 | + |g′ 1 - g′ 2 | + |b′ 1 - b′ 2 |

[0041] Among them, c 1 and c 2 are two sets of color values respectively, T(c 1 ) and T(c 2 ) are corresponding to the two sets of color values c 1 and c 2The values of the two color normalization transformation functions, r' 1 , g' 1 , b' 1 and r' 2 , g' 2 , b' 2 are the two sets of color components after the color normalization transformation function calculates the two sets of color values c 1 and c 2 respectively. g(c 1 , c 2 ) is the color difference value between the two sets of color values after color normalization;

[0042] After the above steps are completed, use the common color for drawing the anchor box as the color template, and record the color value of the color template as b. At the same time, record A as the color image to be repaired of m×n, where a i,j is the color value of the pixel in the i-th row and j-th column.

[0043] Calculate the color difference value between each pixel and the color template one by one according to the color difference calculation formula, and construct a color difference matrix D. Among them, the color difference value d i,j in the i-th row and j-th column is:

[0044] d i,j = g(a i.j , b);

[0045] Among them, d i,j is the color difference value in the i-th row and j-th column, and g(a i.j , b) is the color difference value between the color value a i.j of the pixel in the i-th row and j-th column of the color image A to be repaired after color normalization and the color value b of the color template. The values of each element of the color difference matrix D are calculated through this formula, and the positions of the elements of the obtained color difference matrix D correspond to the positions of the pixels of the color image A to be repaired.

[0046] If there are multiple color templates, record multiple color templates as [b 1 ,..., b K , where K is the number of color templates, then define the color difference value d i,j as the minimum value of the color difference between the pixel color value and each color template. The calculation formula is:

[0047] d i,j = min(g(a i.j , b 1 ), g(a i.j , b 2 ),..., g(a i.j , b K ));

[0048] Among them, di,j a is the color value of the pixel at the j-th column and the i-th row of the color image A to be repaired i.j and the color difference between the K color templates [b 1 ,..., b K after color normalization, and g(a i.j , b 1 ) is the color value a of the pixel at the j-th column and the i-th row of the color image A to be repaired after color normalization i.j and the color template b 1 ;

[0049] S22. Generate a target matrix according to the color difference matrix and a preset color difference threshold, where the element value in the target matrix is 1 or 0;

[0050] Specifically, after obtaining the color difference matrix, the smaller the value of a certain point in the color difference matrix, the greater the probability that the point is an anchor box area. Therefore, in this embodiment, by defining a color difference threshold, the points in the color difference matrix that are lower than the color difference threshold are regarded as candidate points, so as to obtain the candidate area of the anchor box, and the candidate area is represented in the form of a target matrix;

[0051] Specifically, denote the target matrix as E, and the element e i,j at the j-th column and the i-th row of the target matrix E is calculated as follows:

[0052]

[0053] where d i,j is the color difference value at the j-th column and the i-th row of the color difference matrix, is the color difference threshold. If the d i,j of a certain point in the color difference matrix does not exceed the color difference threshold , then the point is a target point, denoted as 1; otherwise, the color difference between the point and the color template is too large, so it is a background point, denoted as 0.

[0054] In the above formula, the color difference threshold needs to be appropriately valued. The larger the color difference threshold , the wider the candidate area, and it is easy to introduce too many false target areas; the smaller the color difference threshold , the stricter the candidate area, and it may miss the true target area. Through experimental analysis in this embodiment, it is found that the value of the color difference threshold is more appropriate between 30 and 80. In this embodiment, the color difference threshold is taken as 40.

[0055] S23. Generate a plurality of convex hull matrices based on a plurality of connected domains of the target matrix, where the connected domain is a region formed by connecting consecutive elements with an element value of 1 in the target matrix;

[0056] Specifically, since the rectangular line area where the anchor box is located is a connected overall structure with fixed shape features, in this embodiment, the connected components of the target matrix are first obtained. A connected component in the target matrix is a region formed by connecting elements with a continuous element value of 1. The present invention filters out small noises according to the number of target points in each connected component, and finally further screens each connected component according to the shape features of the connected component to obtain the final anchor box area. Let P represent the minimum convex submatrix covering a single connected component in the target matrix, with a size of w×h, where p i,j is the element in the i-th row and j-th column.

[0057] Further, in this embodiment, the area of the connected component is defined by the number of target points 1 in the minimum convex submatrix P, and the calculation formula is:

[0058]

[0059] where, p i,j is the element in the i-th row and j-th column of the minimum convex submatrix P with a size of w×h. When the area of the connected component is less than the set threshold, the connected component is filtered out as noise. This threshold is directly related to the thickness of the anchor box line. The wider the line, the larger the threshold should be. In the present invention, the value is taken as 100.

[0060] Since there may be multiple large connected components in the image to be repaired whose colors are close to the color template, it is necessary to further screen according to the shape features. Among them, the anchor box area is a rectangular structure surrounded by horizontal or vertical lines and is empty inside. This feature can be used for screening.

[0061] In this embodiment, the sides of the connected component are truncated according to a set ratio, that is, truncated into smaller rectangles, and then the internal area of the truncated area is further calculated. Whether the connected component is an anchor box is judged according to the numerical value of the internal area of the truncated area. The calculation formula of the internal area is as follows:

[0062]

[0063] where, the operator represents the largest integer not exceeding the internal value, p i,j is the element in the i-th row and j-th column of the minimum convex submatrix P with a size of w×h, γ is the truncation coefficient, which is related to the thickness of the anchor box line, and a value between 0.05 and 0.25 is more appropriate. In this embodiment, the value is taken as 0.2. When the calculated value of Inner(P) is 0, that is, the internal area is 0, the connected component is determined as the anchor box area.

[0064] S24. Screen the convex hull matrices according to the number and positions of the elements with element value 1 in each convex hull matrix, so as to obtain multiple screened convex hull matrices. One of the screened convex hull matrices corresponds to one of the anchor box regions. In this embodiment, preliminary screening is first performed according to the area of the connected component, and then further screening is performed according to the internal area of the connected component after preliminary screening, so as to achieve accurate detection of the anchor box region, laying a foundation for subsequent image repair of the anchor box region.

[0065] S3. Calculate the corrected pixel values of each pixel point in the first image corresponding to the anchor box region based on the linear fitting method to obtain an image without anchor box interference.

[0066] After obtaining the accurately detected anchor box region, it is also necessary to repair each pixel point in the first image corresponding to the anchor box region, that is, calculate the corrected pixel values of each pixel point in the first image corresponding to the calculated anchor box region. The specific steps include:

[0067] S31. Perform edge detection on each anchor box region to obtain the edge coordinates of each anchor box region in the width direction and the height direction, and divide each anchor box region into multiple first sub-regions and multiple second sub-regions according to the edge coordinates. Among them, in this embodiment, the first sub-region is the edge block, and the second sub-region is the vertex block. Then, repair algorithms are designed for the edge blocks and the vertex blocks respectively.

[0068] Specifically, as Figure 2 shown, in this embodiment, the anchor box region is divided into 4 edge blocks and 4 vertex blocks according to the outer edge and the inner edge. The two parallel long sides of the edge block are the outer edge line and the inner edge line of the anchor box respectively, and the two adjacent sides of the vertex block are the outer edge line of the anchor box. Due to the existence of burr noise, it is impossible to directly determine the edge coordinates by whether there are connected component pixel points. In this embodiment, an adaptive threshold method is used to determine the edge coordinates. Taking the width direction as an example, the edge coordinates to be obtained from left to right are denoted as x 1 、x 2 、x 3 and x 4 , where x 1 and x 4 are the outer coordinates, and x 2 and x 3 are the inner coordinates;

[0069] Furthermore, the specific steps for determining the edge coordinates according to the adaptive threshold method include:

[0070] Denote the minimum convex hull matrix of the connected component of the anchor box region as P, with a size of w×h. Calculate the projection sum vector s in the height direction. Its j-th component s j represents the projection sum at the width coordinate j, and the j-th component sj The calculation formula is as follows:

[0071]

[0072] where p i,j is the element in the i-th row and j-th column of the minimum convex sub-matrix P with size w×h;

[0073] Then, find the maximum value u of the vector s max and the minimum value u of the middle part min , and the calculation formula is:

[0074]

[0075] where u max and u min are the maximum value of the vector s and the minimum value of the middle part respectively, and the j-th component s j is the projection sum on the width coordinate j, 1 ≤ j ≤ w, and w is the number of columns of the small convex sub-matrix P. According to this method, u max and u min are obtained; the size of u max is related to the height of the anchor box, while the size of u min is related to the line thickness of the anchor box;

[0076] Then, according to the maximum value u of the vector s max and the minimum value u of the middle part min , find the outer threshold η out and the inner threshold η in , and the calculation formula is:

[0077]

[0078] where u max and u min are the maximum value of the vector s and the minimum value of the middle part respectively, η out and η in are the outer threshold and the inner threshold respectively;

[0079] Based on the outer threshold η out , search from both sides to the center to determine the outer coordinates. Taking x 1 as an example, j starts to increase from 1 until the following formula is satisfied, then this coordinate value is determined as x 1 , and the calculation formula is:

[0080]

[0081] where s j is the projection sum on the width coordinate j, and η out is the outer threshold;

[0082] Based on the inner threshold, retrieve from the center to both sides to determine the inner coordinates. Taking x 2 as an example, j decreases from until the following conditions are met, then the coordinate value is determined as x 2 , and the calculation formula is:

[0083]

[0084] Similarly, the edge coordinates in the height direction of the anchor box can be obtained. Denote the edge coordinates from top to bottom as y 1 , y 2 , y 3 , y 4 and, where y 1 and y 4 are the outer coordinates, and y 2 and y 3 are the inner coordinates. Based on the edge coordinates in the width and height directions, the precise partitioning of the anchor box area can be achieved. For example, the horizontal and vertical coordinate ranges of the upper side block are [x 2 , x 3 , y 1 , y 2 , and the horizontal and vertical coordinate ranges of the upper left vertex block are [x 1 , x 2 , y 1 , y 2 .

[0085] After the precise partitioning of the anchor box area, in this embodiment, the side blocks are repaired first, and then the vertex blocks are repaired.

[0086] S32. Translate the long sides of each of the first sub-regions according to the edge coordinates corresponding to each of the first sub-regions to obtain a plurality of third sub-regions, where the third sub-region is the region formed by translating the two long sides of the first sub-region;

[0087] In this embodiment, the first sub-block region is a side block, and the third sub-region is the region formed by translating the two long sides of the side block.

[0088] It can be understood that for edge block repair, not only a smooth and natural transition effect is required to avoid obvious edges between the repaired area and the original image, but also the consistency of structure and texture needs to be maintained to restore it to the original state as much as possible. The two long sides of the edge block are the anchor frame edges, the outside is the original image, and the inside is the area to be repaired. The repair result should smoothly transition with the outside of the long side while maintaining the consistency of internal and external structural details. Due to the limited thickness of the anchor frame line, the structural details are mainly reflected in the textures truncated by the edge block in the original image, and the complete structure of these textures should be restored as much as possible in the repaired image. This requires matching the original image outside the long side. Taking the upper edge block as an example, keep one long side fixed and move the other long side, and obtain the best matching position through the pixel color consistency.

[0089] In this embodiment, taking the upper edge block as an example, keep one long side of the upper edge block fixed, move the other long side, and then obtain the best matching position through the pixel color consistency before and after the movement.

[0090] Specifically, step S32 includes:

[0091] S321. Calculate a first threshold and a second threshold respectively according to the edge coordinates in the height direction and width direction of one of the first sub-regions, obtaining one first threshold and one second threshold, where the first threshold and the second threshold are respectively the maximum translation distances of partial regions of the first sub-region in the height direction and width direction;

[0092] S322. Translate the partial region of one of the first sub-regions according to one first threshold and one second threshold, and calculate the minimum color difference values of the pixel points of the first image corresponding to the partial region of one of the first sub-regions at different translation amounts, obtaining a plurality of minimum color difference values;

[0093] S323. Translate the partial region of each of the first sub-regions according to the plurality of minimum color difference values corresponding to each of the first sub-regions, obtaining a plurality of third sub-regions.

[0094] Specifically, in this embodiment, the first sub-region is an edge block, and the plurality of partial regions of the first sub-region are the regions corresponding to the two long sides of the edge block. Next, an example of edge block repair for an edge block with the horizontal and vertical coordinate ranges of [x 2 , x 3 , y 1 , y 2 is given:

[0095] Denote A s as the sub-graph corresponding to the anchor frame area in the image to be repaired, a i,j as the color value vector of the pixel at the i-th row and j-th column in it, ri,j , g i,j and b i,j are the red, green, and blue component values, respectively. For any two pixel points a i1,j1 and a i2,j2 , the color difference between them is defined as the sum of the differences of the three color components, and the calculation formula is:

[0096] |a i1,j1 - a i2,j2 | = |r i1,j1 - r i2,j2 | + |g i1,j1 - g i2,j2 | + |b i1,j1 - b i2,j2 |

[0097] where r i1,j1 , g i1,j1 , b i1,j1 are the red, green, and blue component values of pixel point a i1,j1 , respectively, and r i2,j2 , g i2,j2 , b i2,j2 are the red, green, and blue component values of pixel point a i2,j2 , respectively;

[0098] After determining the calculation method of the color difference between pixel points, a color difference function regarding the relative moving distance of the two long sides of the edge block is defined. Taking the lower long side fixed and the upper long side moving q pixel points as an example, the calculation formula is as follows:

[0099]

[0100] where q represents the moving length along the long side direction, in pixels. If q is negative, it means moving to the left; if q is positive, it means moving to the right. represents the pixel points of the upper long side of the edge block with the horizontal and vertical coordinate ranges of [x 2 , x 3 , y 1 , y 2 after moving, represents the pixel points of the fixed lower long side of the edge block with the horizontal and vertical coordinate ranges of [x 2 , x 3 , y 1 , y 2 , and f(q) is the calculated value of the color difference.

[0101] Within the set limit range, calculate the color difference function values for each moving amount, and select the minimum value among them, then the best matching position can be determined. The calculation formula is:

[0102]

[0103] Among them, q opt is the determined optimal movement amount, and q max is the maximum movement amount, which is directly related to the thickness of the edge block. The maximum movement amount q max of the present invention is calculated by the following formula:

[0104] q max = 3(y 2 - y 1 + 1);

[0105] Among them, y 2 and y 1 are the edge coordinates of the edge block with the horizontal and vertical coordinate ranges of [x 2 , x 3 , y 1 , y 2 .

[0106] S33. Calculate the corrected pixel values of the pixel points corresponding to each of the first sub-regions of the first image according to the multiple third sub-regions to obtain a second image;

[0107] Specifically, step S33 includes:

[0108] S331. Select a second pixel point and a third pixel point from the first image in the third sub-region according to the coordinates of each first pixel point of the first image in the first sub-region;

[0109] S332. Calculate the first pixel values from the second pixel point and the third pixel point according to the linear fitting method to obtain a plurality of the first pixel values;

[0110] S333. Correct the pixel values of the first pixel points corresponding to each of the first pixel values to obtain the second image.

[0111] Specifically, after determining the optimal matching position of the edge block with the horizontal and vertical coordinate ranges of [x 2 , x 3 , y 1 , y 2 , that is, the optimal movement amount q opt , the edge block area is repaired by the linear fitting method, and the corrected pixel values of each pixel point of the sub-graph A 2 , x 3 , y 1 , y 2 corresponding to the edge block are calculated. The calculation formula is as follows: s The calculation formula is as follows:

[0112]

[0113] Among them, is the sub-graph A to be repaired s The horizontal and vertical coordinate ranges of 2 , x 3 , y 1 , y 2 is the vector of the color values of the pixels repaired in the i-th row and j-th column of the edge block, q opt is the determined optimal movement amount, α is the proportionality coefficient calculated according to the position of the pixel point corresponding to the current color value vector in the edge block, j u and j d are respectively two coefficients for selecting the pixels used for linear fitting.

[0114] According to the above method, first determine the optimal movement amount between the two long sides of each edge block, and then perform linear fitting, and the lower side block, the left side block and the right side block can also be repaired.

[0115] Furthermore, when the edge block is long, its structural characteristics cannot ensure stable consistency in the long side direction. At this time, it can be cut along the long side direction and divided into several small edge blocks, and each small edge block is repaired according to the above method respectively.

[0116] S34. Calculate the corrected pixel values of the pixel points of the second image based on the second sub-region, and obtain a third image, where the third image is an image without anchor box interference.

[0117] Specifically, step S34 includes:

[0118] S341. According to the coordinates of each fourth pixel point of the second image in the second sub-region, select a fifth pixel point and a sixth pixel point in the width direction, and a seventh pixel point and an eighth pixel point in the height direction from the second image in the second sub-region;

[0119] S342. Perform linear fitting on the fifth pixel point and the sixth pixel point, and the seventh pixel point and the eighth pixel point respectively according to the linear fitting method, and obtain a plurality of second pixel values and a plurality of third pixel values, where one second pixel value and one third pixel value correspond to one fourth pixel point;

[0120] S343. Calculate the weighted average of the second pixel value and the third pixel value corresponding to each fourth pixel point according to the position of the second sub-region where each fourth pixel point is located, and obtain a plurality of fourth pixel values;

[0121] S344. According to each of the fourth pixel values, correct the pixel value of the corresponding fourth pixel point to obtain the third image.

[0122] Specifically, after all the strip repairs are completed, the vertex block needs to be repaired next. Two adjacent sides of the vertex block are the anchor box edges, and the other two sides are adjacent to the short sides of the strip. Therefore, the repair result needs to have a smooth and natural transition effect with the areas outside the four sides.

[0123] In this embodiment, a two-stage vertex block repair method is designed. In the first stage, linear fitting is performed in the horizontal and vertical directions respectively according to two pairs of parallel sides, and the intermediate results are saved; in the second stage, the weighted coefficients are calculated according to the distances of each pixel point from the 4 sides, and the intermediate results are superimposed to complete the strip repair.

[0124] Taking Figure 3 the upper left vertex block with the horizontal and vertical coordinate ranges of [x 1 , x 2 , y 1 , y 2 as an example, in the first stage, let and be the intermediate results of the horizontal and vertical linear fittings of the pixel at the i-th row and j-th column in the sub-graph A s to be repaired respectively. The calculation formula for horizontal linear fitting is:

[0125]

[0126] where is the horizontal linear fitting result of the pixel at the i-th row and j-th column in the sub-graph A s to be repaired, and α is the proportional coefficient calculated according to the position of the current pixel point in the vertex block with the horizontal and vertical coordinate ranges of [x 1 , x 2 , y 1 , y 2 ;

[0127] The calculation formula for vertical linear fitting is:

[0128]

[0129] where is the horizontal linear fitting result of the pixel at the i-th row and j-th column in the sub-graph A s to be repaired, and α is the proportional coefficient calculated according to the position of the current pixel point in the vertex block with the horizontal and vertical coordinate ranges of [x 1 , x 2 , y 1 , y 2 ;

[0130] To achieve a higher repair quality, the edge closer to the pixel to be repaired should have a greater impact on the repair result. Therefore, in the second stage, in this embodiment, the distance between each pixel and the four edges is compared pixel by pixel, the weighting coefficient is calculated according to the distance ratio, and then the intermediate result in the previous step is superimposed according to the weighting coefficient. The calculation formula is:

[0131]

[0132] where x min and y min are respectively the minimum values of the edges in the width direction and height direction of the vertex block corresponding to the color value vector to the vertex block with the horizontal and vertical coordinate ranges of [x 1 , x 2 , y 1 , y 2 , is the color value vector after repair of the pixel at the i-th row and j-th column in the vertex block of the sub-graph A s to be repaired, with the horizontal and vertical coordinate ranges of [x 2 , x 3 , y 1 , y 2 . β is the coefficient for calculating the weights of the horizontal direction linear fitting result and the vertical direction linear fitting result . For all the pixels in the vertex block with the horizontal and vertical coordinate ranges of [x 2 , x 3 , y 1 , y 2 , the above operations are performed to complete the repair of the vertex block.

[0133] Continue to repair the upper right vertex block, lower left vertex block and lower right vertex block according to the above steps, so as to complete the repair of the entire anchor box area. As Figure 4 shown, after the image repair of the anchor box interference is performed according to this example, the repaired image is identical to the original image, and there are no problems such as edge blurring, breakage and fault.

[0134] Embodiment 2:

[0135] As Figure 2 shown, this embodiment provides an image repair system applicable to anchor box interference. The system includes an acquisition module 901, a first processing module 902 and a second processing module 903;

[0136] The acquisition module 901 is used to acquire a first image, and the first image is an anchor box interference image to be repaired;

[0137] The first processing module 902 is configured to calculate the color difference value between each pixel of the first image and a preset anchor box color, extract the anchor box regions according to the color difference value, and obtain a plurality of the anchor box regions, where each of the anchor box regions corresponds to an anchor box;

[0138] The second processing module 903 is configured to calculate the corrected pixel values of the pixel points in the first image corresponding to the anchor box regions based on a linear fitting method, and obtain an image without anchor box interference.

[0139] The first processing module 902 includes a first processing unit 9021, a second processing unit 9022, a third processing unit 9023, and a fourth processing unit 9024:

[0140] The first processing unit 9021 is configured to generate a color difference matrix according to the color difference value, where the positions of the elements of the color difference matrix correspond one-to-one to the positions of the pixel points of the first image, and the element values of the color difference matrix are the color difference values between the pixels of the first image at the corresponding positions and the preset anchor box color;

[0141] The second processing unit 9022 is configured to generate a target matrix according to the color difference matrix and a preset color difference threshold, where the element values in the target matrix are 1 or 0;

[0142] The third processing unit 9023 is configured to generate a plurality of convex hull matrices based on a plurality of connected domains of the target matrix, where the connected domain is a region formed by connecting consecutive elements with an element value of 1 in the target matrix;

[0143] The fourth processing unit 9024 is configured to eliminate the convex hull matrices according to the number and positions of the elements with an element value of 1 in each convex hull matrix, and obtain a plurality of the convex hull matrices after elimination, where one convex hull matrix after elimination corresponds to one of the anchor box regions.

[0144] The second processing module 903 includes a fifth processing unit 9031, a sixth processing unit 9032, a seventh processing unit 9033, and an eighth processing unit 9034:

[0145] The fifth processing unit 9031 is configured to perform edge detection on each of the anchor box regions, obtain the edge coordinates of each of the anchor box regions in the width direction and the height direction, and divide each of the anchor box regions into a plurality of first sub-regions and a plurality of second sub-regions according to the edge coordinates;

[0146] The sixth processing unit 9032 is configured to translate a partial region of each of the first sub-regions according to the edge coordinates corresponding to each of the first sub-regions, and obtain a plurality of third sub-regions, where one of the third sub-regions is a region formed by translating a partial region of one of the first sub-regions;

[0147] A seventh processing unit 9033, configured to calculate a corrected pixel value of each first sub-region of the first image according to a plurality of the third sub-regions, so as to obtain a second image;

[0148] An eighth processing unit 9034, configured to calculate a corrected pixel value of a pixel point of the second image based on the second sub-region, so as to obtain a third image, where the third image is an image without anchor box interference.

[0149] The seventh processing unit 9033 includes a first processing sub-module 90331, a second processing sub-module 90332, and a third processing sub-module 90333:

[0150] The first processing sub-module 90331 is configured to select a second pixel point and a third pixel point from the first image in the third sub-region according to coordinates of each first pixel point of the first image in the first sub-region;

[0151] The second processing sub-module 90332 is configured to calculate first pixel values of the second pixel point and the third pixel point according to a linear fitting method, so as to obtain a plurality of the first pixel values;

[0152] The third processing sub-module 90333 is configured to correct a pixel value of each first pixel point corresponding to each first pixel value, so as to obtain the second image.

[0153] The eighth processing unit includes a fourth processing sub-module 90341, a fifth processing sub-module 90342, a sixth processing sub-module 90343, and a seventh processing sub-module 90344:

[0154] The fourth processing sub-module 90341 is configured to select a fifth pixel point and a sixth pixel point in the width direction and a seventh pixel point and an eighth pixel point in the height direction from the second image in the second sub-region according to coordinates of each fourth pixel point of the second image in the second sub-region;

[0155] The fifth processing sub-module 90342 is configured to perform linear fitting on the fifth pixel point and the sixth pixel point, and the seventh pixel point and the eighth pixel point respectively according to the linear fitting method, so as to obtain a plurality of second pixel values and a plurality of third pixel values, where one second pixel value and one third pixel value correspond to one fourth pixel point;

[0156] The sixth processing sub-module 90343 is configured to calculate a weighted average value of the second pixel value and the third pixel value corresponding to the fourth pixel point according to a position of the second sub-region where each fourth pixel point is located, so as to obtain a plurality of fourth pixel values;

[0157] The seventh processing sub-module 90344 is configured to correct the pixel value of each of the fourth pixel points corresponding to each of the fourth pixel values to obtain the third image.

[0158] The sixth processing unit 9032 includes an eighth processing sub-module 90321, a ninth processing sub-module 90322, and a tenth processing sub-module 90323:

[0159] The eighth processing sub-module 90321 is configured to calculate a first threshold and a second threshold respectively according to the edge coordinates in the height direction and the width direction of one of the first sub-regions, to obtain one first threshold and one second threshold, where the first threshold and the second threshold are respectively the maximum translation distances of partial regions of the first sub-region in the height direction and the width direction;

[0160] The ninth processing sub-module 90322 is configured to perform translation of the partial region of one of the first sub-regions according to one first threshold and one second threshold, and calculate the minimum color difference values of the pixel points of the first image corresponding to the partial region of one of the first sub-regions at different translation amounts, to obtain a plurality of the minimum color difference values;

[0161] The tenth processing sub-module 90323 is configured to perform translation of the partial region of each of the first sub-regions according to the plurality of minimum color difference values corresponding to each of the first sub-regions, to obtain a plurality of third sub-regions.

[0162] It should be noted that regarding the system in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0163] The above are only the preferred embodiments of the present invention, and are not used to limit the present invention. For those skilled in the art, the present invention may have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0164] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present invention, and all should be covered by the protection scope of the present invention.

Claims

1. An image restoration method suitable for anchor box interference, characterized in that: include: Acquire a first image, where the first image is an anchor frame interference image to be repaired; Calculating a color difference value between each pixel of the first image and a preset anchor frame color, extracting an anchor frame region according to the color difference value, and obtaining a plurality of anchor frame regions, each of which corresponds to an anchor frame; Based on a linear fitting method, a corrected pixel value of each pixel point in the first image corresponding to the anchor frame area is calculated to obtain an image without anchor frame interference.

2. The image restoration method applicable to anchor frame interference according to claim 1, characterized in that , the extraction of the anchor frame area according to the color difference value includes: Generate a color difference matrix according to the color difference value, wherein positions of elements of the color difference matrix correspond one-to-one to positions of pixels of the first image, and element values ​​of the color difference matrix are color difference values ​​between pixels of the first image at corresponding positions and a preset anchor frame color; Generate a target matrix according to the color difference matrix and a preset color difference threshold, wherein the element values ​​in the target matrix are 1 or 0; Generate multiple convex hull matrices based on multiple connected domains of the target matrix, where the connected domains are regions connected by elements whose continuous element values ​​are 1 in the target matrix; The convex hull matrix is ​​eliminated according to the number and position of elements with element values ​​of 1 contained in each convex hull matrix to obtain multiple convex hull matrices after elimination, and each convex hull matrix after elimination corresponds to one anchor frame area.

3. The image restoration method applicable to anchor frame interference according to claim 1, characterized in that , the method of calculating the corrected pixel value of each pixel point in the first image corresponding to the anchor frame area based on the linear fitting method includes: Performing edge detection on each of the anchor frame regions to obtain edge coordinates of each of the anchor frame regions in a width direction and a height direction, and dividing each of the anchor frame regions into a plurality of first sub-regions and a plurality of second sub-regions according to the edge coordinates; According to the edge coordinates corresponding to each of the first sub-regions, a partial region of each of the first sub-regions is translated to obtain a plurality of third sub-regions, wherein one of the third sub-regions is a region formed by translating a partial region of one of the first sub-regions; Calculating a corrected pixel value of a pixel point corresponding to each of the first sub-regions of the first image according to the plurality of the third sub-regions to obtain a second image; The corrected pixel values ​​of the pixels of the second image are calculated based on the second sub-region to obtain a third image, where the third image is an image without interference from the anchor box.

4. The image restoration method applicable to anchor frame interference according to claim 3, characterized in that , the step of calculating the corrected pixel value of the pixel point of the first image according to the third sub-region includes: Selecting a second pixel and a third pixel from the first image in the third sub-region according to the coordinates of each first pixel of the first image in the first sub-region; Calculate the first pixel values ​​of the second pixel point and the third pixel point according to a linear fitting method to obtain a plurality of the first pixel values; The pixel value of the first pixel point corresponding to each first pixel value is corrected according to each first pixel value to obtain the second image.

5. The image restoration method applicable to anchor frame interference according to claim 3, characterized in that , the step of calculating the corrected pixel value of the pixel point of the second image based on the second sub-region includes: According to the coordinates of each fourth pixel point of the second image in the second sub-region, select a fifth pixel point and a sixth pixel point in the width direction, and select a seventh pixel point and an eighth pixel point in the height direction from the second image in the second sub-region; According to a linear fitting method, linear fitting is performed on the fifth pixel point and the sixth pixel point, and the seventh pixel point and the eighth pixel point, respectively, to obtain a plurality of second pixel values ​​and a plurality of third pixel values, wherein one second pixel value and one third pixel value correspond to one fourth pixel point; According to the position of the second sub-region where each of the fourth pixel points is located, a weighted average of the second pixel value and the third pixel value corresponding to the fourth pixel point is calculated to obtain a plurality of fourth pixel values; The pixel value of the fourth pixel point corresponding to each fourth pixel value is corrected according to each fourth pixel value to obtain the third image.

6. An image restoration system suitable for anchor frame interference, characterized in that: include: An acquisition module, configured to acquire a first image, where the first image is an anchor frame interference image to be repaired; a first processing module, configured to calculate a color difference value between each pixel of the first image and a preset anchor frame color, extract an anchor frame region according to the color difference value, and obtain a plurality of anchor frame regions, each of which corresponds to an anchor frame; The second processing module is used to calculate the corrected pixel value of each pixel point in the first image corresponding to the anchor frame area based on a linear fitting method to obtain an image without anchor frame interference.

7. The image restoration system applicable to anchor frame interference according to claim 6, characterized in that: The first processing module comprises: A first processing unit, configured to generate a color difference matrix according to the color difference value, wherein positions of elements of the color difference matrix correspond one-to-one to positions of pixels of the first image, and element values ​​of the color difference matrix are color difference values ​​between pixels of the first image at corresponding positions and a preset anchor frame color; A second processing unit, configured to generate a target matrix according to the color difference matrix and a preset color difference threshold, wherein the element values ​​in the target matrix are 1 or 0; A third processing unit is used to generate multiple convex hull matrices based on multiple connected domains of the target matrix, where the connected domains are regions connected by elements whose continuous element values ​​are 1 in the target matrix; The fourth processing unit is used to remove the convex hull matrix according to the number and position of elements with element values ​​of 1 contained in each convex hull matrix, to obtain multiple convex hull matrices after removal, and each convex hull matrix after removal corresponds to one anchor frame area.

8. The image restoration system applicable to anchor frame interference according to claim 6, characterized in that: The second processing module comprises: a fifth processing unit, configured to perform edge detection on each of the anchor frame regions, obtain edge coordinates of each of the anchor frame regions in a width direction and a height direction, and divide each of the anchor frame regions into a plurality of first sub-regions and a plurality of second sub-regions according to the edge coordinates; a sixth processing unit, configured to translate a partial region of each of the first sub-regions according to edge coordinates corresponding to each of the first sub-regions, to obtain a plurality of third sub-regions, wherein one of the third sub-regions is a region formed by translating a partial region of one of the first sub-regions; A seventh processing unit, configured to calculate a corrected pixel value of a pixel point corresponding to each of the first sub-regions of the first image according to the plurality of the third sub-regions, to obtain a second image; An eighth processing unit is used to calculate the corrected pixel values ​​of the pixels of the second image based on the second sub-region to obtain a third image, where the third image is an image without interference from the anchor frame.

9. The image restoration system applicable to anchor frame interference according to claim 8, characterized in that: The seventh processing unit comprises: A first processing submodule, configured to select a second pixel point and a third pixel point from the first image within the third subregion according to the coordinates of each first pixel point of the first image within the first subregion; A second processing submodule, configured to calculate the first pixel values ​​of the second pixel point and the third pixel point according to a linear fitting method to obtain a plurality of the first pixel values; The third processing submodule is used to correct the pixel value of the first pixel point corresponding to each first pixel value according to each first pixel value to obtain the second image.

10. The image restoration system applicable to anchor frame interference according to claim 8, characterized in that: The eighth processing unit comprises: a fourth processing submodule, configured to select, from the second image within the second subregion, a fifth pixel point and a sixth pixel point in a width direction, and a seventh pixel point and an eighth pixel point in a height direction, according to the coordinates of each fourth pixel point of the second image within the second subregion; a fifth processing submodule, configured to perform linear fitting on the fifth pixel point and the sixth pixel point, and the seventh pixel point and the eighth pixel point respectively according to a linear fitting method to obtain a plurality of second pixel values ​​and a plurality of third pixel values, wherein one second pixel value and one third pixel value correspond to one fourth pixel point; a sixth processing submodule, configured to calculate, according to a position of each of the fourth pixel points in the second sub-region, a weighted average of the second pixel value and the third pixel value corresponding to the fourth pixel point, to obtain a plurality of fourth pixel values; The seventh processing submodule is used to correct the pixel value of the fourth pixel point corresponding to each fourth pixel value according to each fourth pixel value to obtain the third image.

Images