A method, device and storage medium for eliminating moiré patterns by combined linear transformation

Abstract

The present application discloses a method, device and storage medium for eliminating moiré by combining linear transformation, which is used to improve the moiré elimination effect on the basis of retaining image feature information during defect detection. The present application method includes: obtaining a first image, searching for an effective area in the first image; calculating the corner point coordinates of the effective area; performing geometric correction on the corner point coordinates to generate corrected corner point coordinates; generating a transformation matrix based on the corner point coordinates and the corrected corner point coordinates; using the transformation matrix to convert the first image into a corrected image; using bilinear interpolation and combining the grayscale of the effective area of ​​the first image to calculate the pixel grayscale value of the corrected image, and inserting the pixel grayscale value into the corrected image; calculating the scaling factor based on the actual magnification and the specified magnification, and generating a scaling vector based on the scaling factor; based on the scaling vector, performing affine transformation conversion compression on the corrected image to generate a second image; using bilinear interpolation to process the pixel grayscale value of the second image.

Description

Technical Field

[0001] The present application relates to the technical field of moiré processing, and in particular to a method, device and storage medium for eliminating moiré by combining linear transformation. Background Art

[0002] LCD screens have stable technology, clear imaging, and rich colors, and are widely used in many fields. In recent years, with the widespread use of medium and large-sized LCD screens in flat panels, automotive and other fields, the market has higher and higher requirements for the quality of display screens. At the same time, due to its strong plasticity, the size change range is relatively large, resulting in a large difference in its PPI. Therefore, during the AOI inspection process, conventional detection logic has no way to cope with such a large size change. In order to make the inspection more convenient, in the high-resolution camera, product images of different sizes and resolutions are converted into a unified magnification (Mapping). In this way, the same detection logic and parameters can be set to cope with display screens of different sizes and resolutions.

[0003] However, when an image with a larger magnification is converted into a smaller magnification, moiré patterns will appear all over the screen. These moiré patterns not only interfere with the image information, but also increase the difficulty of defect detection. In order to eliminate moiré patterns, the prior art uses multiple perspective transformations to neutralize the information of the high-frequency and low-frequency parts of the image, which can effectively remove the moiré patterns and reduce the proportion of moiré patterns by 75%. This can meet the needs of conventional defect detection, but defects with smaller contrast will still have an impact. In order to improve the moiré patterns in the defect detection process, the prior art attempts to use a variety of algorithms to eliminate the moiré patterns generated when the image magnification is directly changed, but the general algorithm for eliminating moiré patterns will also remove the feature information of the image, which will affect the subsequent defect detection. That is, the prior art cannot improve the moiré pattern elimination effect on the basis of retaining the image feature information during the defect detection process. Summary of the invention

[0004] The present application discloses a method, device and storage medium for eliminating moiré by combining linear transformation, which are used to improve the moiré elimination effect on the basis of retaining image feature information during defect detection.

[0005] The technical solution provided in this application is described below:

[0006] In a first aspect, the present application provides a method for eliminating moiré patterns by combining linear transformations, the method comprising:

[0007] Acquire a first image, and search for a valid area in the first image;

[0008] Calculate the coordinates of the corner points of the effective area;

[0009] Perform geometric correction on the corner point coordinates according to the resolution of the target display screen to generate corrected corner point coordinates;

[0010] Generate a transformation matrix based on the corner point coordinates and the corrected corner point coordinates;

[0011] transforming the first image into a rectified image using a transformation matrix;

[0012] Calculating the pixel grayscale value of the corrected image by using a bilinear interpolation method and combining the grayscale of the effective area of ​​the first image, and inserting the pixel grayscale value into the corrected image;

[0013] Calculate the scaling factor according to the actual magnification and the specified magnification, and generate a scaling vector according to the scaling factor;

[0014] According to the scaling vector, the rectified image is affine transformed and compressed to generate a second image;

[0015] The bilinear interpolation method is used to process the grayscale values ​​of the pixels of the second image.

[0016] Optionally, acquiring a first image and searching for a valid area in the first image includes:

[0017] A first image is acquired, and a grayscale edge detection method is used to search for a valid area in the first image.

[0018] Optionally, after performing affine transformation conversion compression on the rectified image to generate the second image according to the scaling vector, and before using bilinear interpolation to process the grayscale values ​​of the pixels of the second image, the method further includes:

[0019] Perform mean filtering on the second image.

[0020] Optionally, geometric correction is performed on the corner point coordinates according to the resolution of the target display screen to generate corrected corner point coordinates, including:

[0021] Calculate the long side distance of the effective area according to the corner point coordinates;

[0022] The actual magnification is calculated based on the resolution and long side distance of the target display;

[0023] Generate corrected corner coordinates using true magnification and resolution.

[0024] Optionally, using a bilinear interpolation method and combining the grayscale of the effective area of ​​the first image to calculate the pixel grayscale value of the corrected image, and inserting the pixel grayscale value into the corrected image, comprises:

[0025] Determine the position of the pixel to be inserted in the corrected image;

[0026] Calculate the original position coordinates of the pixel point to be inserted on the first image according to the inverse matrix of the transformation matrix;

[0027] Determine four adjacent integer coordinate points corresponding to the original position coordinates, and obtain gray value data of the four integer coordinate points;

[0028] Calculate the target grayscale according to the original position coordinates, the coordinates of four adjacent integer coordinate points, the grayscale value data and the preset bilinear interpolation formula;

[0029] Insert the target grayscale into the pixel position to be inserted in the corrected image.

[0030] Optionally, after processing the grayscale values ​​of the pixels of the second image by using the bilinear interpolation method, the method further includes:

[0031] Compressing the first image by multiples to generate a third image that matches the target display screen;

[0032] Acquire a first spectrum diagram and a second spectrum diagram of the second image and the third image by Fourier transform;

[0033] Extracting a first moiré feature region and a second moiré feature region from the first spectrum graph and the second spectrum graph;

[0034] A comparative analysis is performed on the first moiré feature region and the second moiré feature region to generate a moiré removal quality analysis.

[0035] Optionally, after compressing the first image to generate a third image matching the target display screen, and before acquiring the first spectrum graph and the second spectrum graph of the second image and the third image by Fourier transform, the method further includes:

[0036] Acquire a moiré-enhanced image, wherein the moiré-enhanced image has the same resolution as the second image and the moiré-enhanced image has the same resolution as the first image before compression, and the moiré-enhanced image exhibits moiré features when displayed on a target display screen;

[0037] Determine the moiré region of the moiré enhanced image by binarization processing;

[0038] Determining a region to be enhanced in the second image by using a moiré region of the moiré enhanced image;

[0039] Obtaining a first grayscale mean value and a second grayscale mean value of the area to be enhanced and the moiré area respectively;

[0040] Obtaining the first standard deviation and the second standard deviation of the area to be enhanced and the moiré area respectively;

[0041] Calculating a moiré adjustment coefficient according to the first grayscale mean and the second grayscale mean;

[0042] Calculate the moiré contrast enhancement coefficient according to the first standard deviation and the second standard deviation;

[0043] Moire enhancement processing is performed on the pixel points in the area to be enhanced on the second image according to the moire adjustment coefficient and the moire contrast enhancement coefficient to enhance the moire features on the second image.

[0044] A second aspect of the present application provides a device for eliminating moiré by combining linear transformations, comprising:

[0045] A first acquisition unit, used to acquire a first image and search for a valid area in the first image;

[0046] A first calculation unit, used to calculate the coordinates of the corner points of the effective area;

[0047] A first generating unit, configured to perform geometric correction on the corner point coordinates according to the resolution of the target display screen to generate corrected corner point coordinates;

[0048] A second generating unit, used for generating a transformation matrix according to the corner point coordinates and the corrected corner point coordinates;

[0049] A conversion unit, configured to convert the first image into a rectified image using a conversion matrix;

[0050] A second calculation unit, configured to calculate the pixel grayscale value of the corrected image by using a bilinear interpolation method and combining the grayscale of the effective area of ​​the first image, and insert the pixel grayscale value into the corrected image;

[0051] A third generating unit is used to calculate a scaling factor according to the actual magnification and the specified magnification, and generate a scaling vector according to the scaling factor;

[0052] A fourth generating unit, configured to perform affine transformation conversion compression on the rectified image according to the scaling vector to generate a second image;

[0053] The first processing unit is used to process the grayscale values ​​of pixels of the second image by using a bilinear interpolation method.

[0054] Optionally, the first acquisition unit includes:

[0055] A first image is acquired, and a grayscale edge detection method is used to search for a valid area in the first image.

[0056] Optionally, after the fourth generating unit and before the first processing unit, the device further includes:

[0057] The filtering unit is used to perform mean filtering on the second image.

[0058] Optionally, the first generating unit includes:

[0059] Calculate the long side distance of the effective area according to the corner point coordinates;

[0060] The actual magnification is calculated based on the resolution and long side distance of the target display;

[0061] Generate corrected corner coordinates using true magnification and resolution.

[0062] Optionally, the second computing unit includes:

[0063] Determine the position of the pixel to be inserted in the corrected image;

[0064] Calculate the original position coordinates of the pixel point to be inserted on the first image according to the inverse matrix of the transformation matrix;

[0065] Determine four adjacent integer coordinate points corresponding to the original position coordinates, and obtain gray value data of the four integer coordinate points;

[0066] Calculate the target grayscale according to the original position coordinates, the coordinates of four adjacent integer coordinate points, the grayscale value data and the preset bilinear interpolation formula;

[0067] Insert the target grayscale into the pixel position to be inserted in the corrected image.

[0068] Optionally, after the first processing unit, the device further includes:

[0069] A fifth generating unit, used for compressing the first image by a multiple to generate a third image matching the target display screen;

[0070] A second acquisition unit, used for acquiring a first spectrum diagram and a second spectrum diagram of the second image and the third image through Fourier transform;

[0071] An extraction unit, used for extracting a first moiré feature region and a second moiré feature region from the first spectrum graph and the second spectrum graph;

[0072] The sixth generating unit is used to perform comparative analysis on the first moiré feature area and the second moiré feature area to generate a moiré removal quality analysis.

[0073] Optionally, after the fifth generating unit and before the second acquiring unit, the device further includes:

[0074] A third acquisition unit is used to acquire a moiré-enhanced image, wherein the moiré-enhanced image has the same resolution as the second image and the same resolution as the first image before compression, and the moiré-enhanced image has moiré features when displayed on a target display screen;

[0075] A first determining unit, configured to determine a moiré region of a moiré enhanced image by binarization processing;

[0076] A second determining unit, configured to determine a to-be-enhanced region of the second image by using a moiré region of the moiré enhanced image;

[0077] A fourth acquisition unit, used to respectively acquire a first grayscale mean value and a second grayscale mean value of the area to be enhanced and the moiré area;

[0078] A fifth acquisition unit, used to respectively acquire a first standard deviation and a second standard deviation of the area to be enhanced and the moiré area;

[0079] A third calculation unit, configured to calculate a moiré adjustment coefficient according to the first grayscale mean and the second grayscale mean;

[0080] A fourth calculation unit, configured to calculate a moiré contrast enhancement coefficient according to the first standard deviation and the second standard deviation;

[0081] The second processing unit is used to perform moiré enhancement processing on the pixel points in the area to be enhanced on the second image according to the moiré adjustment coefficient and the moiré contrast enhancement coefficient, so as to enhance the moiré features on the second image.

[0082] A third aspect of the present application provides a device for eliminating moiré by combining linear transformations, the device comprising:

[0083] Processor, memory, input-output unit, and bus;

[0084] The processor is connected to the memory, the input and output unit, and the bus;

[0085] The memory stores a program, and the processor calls the program to execute the first aspect and any optional method in the first aspect.

[0086] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional method in the first aspect.

[0087] It can be seen from the above technical solutions that this application has the following advantages:

[0088] A transformation matrix is ​​generated according to the corner point coordinates and the correction corner point coordinates, and the first image is transformed into the correction image using the transformation matrix; the transformation matrix provides a mathematical basis for image transformation, and the image quality is improved by transforming the first image into the correction image.

[0089] The bilinear interpolation method is used in combination with the grayscale of the effective area of ​​the first image to calculate the pixel grayscale value of the corrected image, and the pixel grayscale value is inserted into the corrected image; the pixel grayscale value of the first image calculated by the bilinear interpolation method replaces the original pixel grayscale value of the corrected image, which can reduce the pixel discontinuity or jagged phenomenon that may appear in the corrected image, and at the same time can neutralize the high-frequency and low-frequency information in the image spatial frequency, so that the moiré pattern of the image is filtered out.

[0090] According to the scaling vector, the rectified image is transformed and compressed by affine transformation to generate the second image; through affine transformation, the scaling, rotation and other transformation operations of the rectified image can be realized. Then the grayscale value of the pixel point of the second image is processed by bilinear interpolation method; the pixel grayscale value of the rectified image calculated by bilinear interpolation method replaces the original pixel grayscale value of the second image, which can further improve the effect of eliminating moiré. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0092] Figure 1 A schematic diagram of an embodiment of a method for eliminating moiré by combining linear transformations in the present application;

[0093] Figure 2 A schematic diagram of an embodiment of a method for searching a valid area for the present application;

[0094] Figure 3 A schematic diagram of an embodiment of the image filtering method of the present application;

[0095] Figure 4 A schematic diagram of an embodiment of a method for generating corrected corner point coordinates for the present application;

[0096] Figure 5 A schematic diagram of an embodiment of a method for performing bilinear interpolation on an image according to the present application;

[0097] Figure 6 A schematic diagram of an embodiment of a method for analyzing the quality of moire removal in the present application;

[0098] Figure 7 A schematic diagram of an embodiment of a method for enhancing moiré features of the present application;

[0099] Figure 8 A schematic diagram of an embodiment of a device for removing moiré by combining linear transformation according to the present application;

[0100] Fig. 9 Another schematic diagram of a device for removing moiré patterns by combining linear transformations according to the present application;

[0101] Fig.10 This is a schematic diagram of the edge critical point in the AA area contour line of this application;

[0102] Fig.11 This is a schematic diagram of the edge fitting line in the AA area contour line of this application;

[0103] Fig.12 This is a schematic diagram of the projection of the geometric correction diagram for this application;

[0104] Fig.13 A schematic diagram of the compression graph projection of the structural affine transformation of this application;

[0105] Fig.14 A schematic diagram of the moiré elimination of the conventional compression method of the present application;

[0106] Fig.15 This is a schematic diagram of the moiré elimination process in this application. DETAILED DESCRIPTION

[0107] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0108] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0109] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0110] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0111] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0112] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0113] In the prior art, when an image with a larger magnification is converted into a smaller magnification, moiré patterns will appear all over the screen. These moiré patterns not only interfere with the image information, but also increase the difficulty of defect detection. In order to eliminate moiré patterns, in the prior art, multiple perspective transformations are performed to neutralize the information of the high-frequency and low-frequency parts of the image, which can effectively remove the moiré patterns and reduce the proportion of moiré patterns by 75%. This can meet the needs of conventional defect detection, but defects with smaller contrast will still have an impact. In order to improve the moiré patterns in the defect detection process, the prior art attempts to use a variety of algorithms to eliminate the moiré patterns generated when the image magnification is directly changed, but the general algorithm for eliminating moiré patterns will also remove the feature information of the image, which will affect the subsequent defect detection. That is, the prior art cannot improve the moiré pattern elimination effect on the basis of retaining the image feature information during the defect detection process.

[0114] Based on this, the present application discloses a method, device and storage medium for eliminating moiré by combining linear transformation, which are used to improve the moiré elimination effect on the basis of retaining image feature information during defect detection.

[0115] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0116] The method of the present application can be applied to a server, a device, a terminal or other devices with logic processing capabilities, and the present application does not limit this. For the convenience of description, the following description is made by taking the execution subject as an example of a terminal.

[0117] See also Figure 1 The present application provides an embodiment of a method for eliminating moiré by combining linear transformations, comprising:

[0118] 101. Acquire a first image, and search for a valid area in the first image;

[0119] First, the system or device captures and receives a first image; in the first image, the system performs a search process to locate and mark the effective area of ​​the first image. The effective area represents the effective area or features of the display screen in the first image when the camera captures, excluding the redundant background area.

[0120] 102. Calculate the coordinates of the corner points of the effective area;

[0121] After determining the effective area of ​​the first image, the system calculates the coordinates of the corner points (usually four corners) of the effective area. Based on the coordinates, the positions of the four corner points are marked in the effective area, and the corner point coordinates define the boundaries of the effective area.

[0122] 103. Perform geometric correction on the corner point coordinates according to the resolution of the target display screen to generate corrected corner point coordinates;

[0123] After calculating the corner point coordinates of the effective area, the system will perform geometric correction on the corner point coordinates according to the resolution of the target display screen and calculate the corrected corner point coordinates. The corrected corner point coordinates provide data support for subsequent steps.

[0124] 104. Generate a transformation matrix according to the corner point coordinates and the corrected corner point coordinates;

[0125] Based on the original corner point coordinates and the corrected corner point coordinates, the system calculates a transformation matrix. The transformation matrix describes the mapping relationship from the corner point coordinates to the corrected corner point coordinates.

[0126] 105. Convert the first image into a corrected image using a conversion matrix;

[0127] Using the transformation matrix generated in the previous steps, the system multiplies each point on the first image by the transformation matrix, and transforms the first image into a corrected image by means of a transformation method such as translation, scaling or rotation through the transformation matrix.

[0128] Please refer to Fig.12 , Fig.12 This is a schematic diagram of the projection of the geometric correction image. The red line in the figure represents the grayscale projection.

[0129] 106. Calculate the pixel grayscale value of the corrected image by using a bilinear interpolation method and combining the grayscale of the effective area of ​​the first image, and insert the pixel grayscale value into the corrected image;

[0130] In order to filter out the moiré patterns in the corrected image, the system uses bilinear interpolation to calculate the pixel grayscale values ​​of the corrected image based on the grayscale information of the effective area of ​​the first image; after calculating the appropriate pixel grayscale values, the pixel grayscale values ​​are inserted into the corrected image and replace the original pixel grayscale values ​​of the corrected image.

[0131] 107. Calculate a scaling factor according to the actual magnification and the specified magnification, and generate a scaling vector according to the scaling factor;

[0132] The system calculates a scaling factor based on the actual magnification of the target display screen and the magnification specified by the user. The system converts the calculated scaling factor to generate a scaling vector for guiding the subsequent first image scaling operation.

[0133] The specific calculation process is: given the actual magnification L of the image and the specified magnification L' to be compressed, the scaling factors sx and sy are calculated (where sx=sy=R=L' / L);

[0134] According to the scaling factor R, set the scaling vector to {0,R,0,0,R,0,0,0,1}.

[0135] 108. Perform affine transformation conversion and compression on the rectified image according to the scaling vector to generate a second image;

[0136] Using the scaling vector generated in the previous steps, the system performs an affine transformation on the rectified image; through the affine transformation, the rectified image is converted into a second image of a specified magnification.

[0137] Please refer to Fig.13 , Fig.13 This is a schematic diagram of the compression graph projection of the structural affine transformation. The red line in the figure represents the grayscale projection. Fig.12 and Fig.13 The figure reflects the changing trend of grayscale projection at different stages. The magnifications of the two figures are different. Fig.13 yes Fig.12 Formed by linear transformation.

[0138] 109. Use bilinear interpolation method to process the grayscale values ​​of the pixels of the second image.

[0139] In order to further improve the moiré removal effect, the system will again use bilinear interpolation to process the grayscale value of each pixel in the second image; based on the grayscale information of the effective area of ​​the corrected image, the system will use bilinear interpolation to calculate the pixel grayscale value of the second image; after calculating the appropriate pixel grayscale value, the pixel grayscale value is inserted into the second image and replaces the original pixel grayscale value of the second image.

[0140] In this embodiment, searching the effective area determines that the subsequent steps are only for the useful part of the image, providing an accuracy data basis for subsequent image processing and correction;

[0141] The corner point coordinates are geometrically corrected according to the resolution of the target display screen to generate corrected corner point coordinates; the corrected corner point coordinates are the corner point coordinates that eliminate the distortion of the target display screen caused by shooting angle, lens distortion, etc.

[0142] A transformation matrix is ​​generated according to the corner point coordinates and the correction corner point coordinates, and the first image is transformed into the correction image using the transformation matrix; the transformation matrix provides a mathematical basis for image transformation, and the image quality is improved by transforming the first image into the correction image.

[0143] The bilinear interpolation method is used in combination with the grayscale of the effective area of ​​the first image to calculate the pixel grayscale value of the corrected image, and the pixel grayscale value is inserted into the corrected image; the pixel grayscale value of the first image calculated by the bilinear interpolation method replaces the original pixel grayscale value of the corrected image, which can reduce the pixel discontinuity or jagged phenomenon that may appear in the corrected image, and at the same time can neutralize the high-frequency and low-frequency information in the image spatial frequency, so that the moiré pattern of the image is filtered out.

[0144] The scaling factor is calculated according to the actual magnification and the specified magnification, and the scaling vector is generated according to the scaling factor; by calculating the scaling factor and the scaling vector, the scaling ratio and size of the subsequent corrected image can be accurately controlled.

[0145] According to the scaling vector, the rectified image is converted and compressed by affine transformation to generate a second image; through affine transformation, scaling, rotation and other transformation operations of the rectified image can be achieved.

[0146] The grayscale values ​​of the pixels of the second image are processed by bilinear interpolation method; the grayscale values ​​of the pixels of the corrected image are calculated by bilinear interpolation method to replace the original grayscale values ​​of the pixels of the second image, which can further improve the effect of eliminating moiré.

[0147] See also Figure 2The present application provides an embodiment of a method for searching a valid area, comprising:

[0148] 201. Obtain a first image, and use a grayscale edge detection method to search for a valid area in the first image.

[0149] After acquiring the first image, the grayscale edge detection method is used to connect the edges of important information in the first image into edge contours, and irrelevant image information is removed; the area surrounded by the edge contour is defined as the valid area. Please refer to Fig.10 and Fig.11 , Fig.10 This is a schematic diagram of the edge critical points in the AA area contour line when the grayscale edge inspection method is used. The colored points are the edge critical points. Fig.11 It is a schematic diagram of the edge fitting line in the AA area contour line, and the blue line is the edge fitting line.

[0150] In this embodiment, a grayscale edge detection method is used to remove irrelevant image information in the first image, accurately locate the effective area, and provide an accuracy data basis for subsequent image processing and correction.

[0151] See also Figure 3 , the present application provides an embodiment of an image filtering method, comprising:

[0152] 301. Perform mean filtering on the second image;

[0153] Considering that there are still faint vertical moiré patterns on the corrected image, a 3*3 mean filter will be performed before the corrected image is affine transformed. Specifically, a 3*3 image matrix is ​​extracted from the corrected image, and then the average value of the 9 values ​​is extracted, and the average value is used to replace the original pixel value of the corrected image.

[0154] In this embodiment, mean filtering is used to effectively remove noise in the corrected image, reduce the sharpness of the corrected image, and make the image smoother.

[0155] See also Figure 4 The present application provides an embodiment of a method for generating corrected corner point coordinates, comprising:

[0156] 401. Calculate the long side distance of the effective area according to the corner point coordinates;

[0157] The long side distance of the effective area is determined by using the coordinates of the corner points of the effective area; the effective area is a rectangular area defined by four corner points, and the long side distance is obtained by directly calculating the distance between two horizontally or vertically adjacent corner points.

[0158] 402. Calculate the actual magnification according to the resolution and the long side distance of the target display screen;

[0159] Compare the long side distance of the active area with the resolution of the target display to calculate the true magnification. Since the resolution of the target display is usually in pixels, first convert the long side distance to pixels, and then compare the long side distance with the pixel size of the target display to get the true magnification.

[0160] According to the above steps, the long side distance s is calculated, and the resolution of the target display is known to be w or h, then the actual magnification L=s / w.

[0161] 403. Generate corrected corner point coordinates using the actual magnification and resolution.

[0162] After obtaining the calculated real magnification, the real magnification and the resolution of the target display are used to generate the correction corner coordinates. Given the corner coordinates (X, Y), the correction corner coordinates (X', Y') are calculated based on the calculated real magnification L and resolution w or h, where X'=(0, L*w, 0, L*w) and Y'=(0, 0, L*h, L*h).

[0163] In this embodiment, by calculating the long side distance of the effective area, the real magnification and the coordinates of the correction corner points, the necessary parameters and basis are provided for the geometric correction and scaling of the image.

[0164] See also Figure 5 The present application provides an embodiment of a method for performing bilinear interpolation on an image, comprising:

[0165] 501. Determine the position of the pixel to be inserted in the corrected image;

[0166] In this step, we first determine which pixel locations in the rectified image need to have new grayscale values ​​inserted.

[0167] 502. Calculate the original position coordinates of the pixel point to be inserted on the first image according to the inverse matrix of the transformation matrix;

[0168] The positions of pixels in the rectified image are calculated using the inverse matrix of the transformation matrix, and the positions of pixels in the rectified image are mapped back to corresponding positions in the first image.

[0169] 503. Determine four adjacent integer coordinate points corresponding to the original position coordinates, and obtain gray value data of the four integer coordinate points;

[0170] Since the original position point of the first image is a coordinate value with a decimal, the grayscale value of the original position point cannot be obtained by direct calculation, so obtaining four adjacent integer coordinate points can infer the integer coordinate point of the original position, and the grayscale value data of the four integer coordinate points provide data support for subsequent calculations.

[0171] 504. Calculate the target grayscale according to the original position coordinates, the coordinates of four adjacent integer coordinate points, the grayscale value data and a preset bilinear interpolation formula;

[0172] The original position coordinates, the coordinates of the four adjacent integer coordinate points, and the grayscale value data of the four adjacent integer coordinate points obtained in the above steps are input into the bilinear interpolation formula; bilinear interpolation is a method of interpolation in two-dimensional space, which calculates the grayscale value of the point to be interpolated based on the grayscale values ​​of the four points around the point to be interpolated and the coordinates of these points.

[0173] Specifically, the bilinear interpolation formula is:

[0174]

[0175] Note: i,j coordinates are integer parts, u,v coordinates are decimal parts, is the bilinear interpolation formula, is the gray value of the pixel at the corresponding position in the original image.

[0176] 505. Insert the target grayscale into the position of the pixel to be inserted in the corrected image.

[0177] The calculated target grayscale value is inserted into the corresponding pixel position in the corrected image. Each pixel in the corrected image will be assigned a grayscale value, completing the image transformation and interpolation processing.

[0178] In this embodiment, the position of the pixel to be inserted is determined in the corrected image, and it is determined which pixels in the corrected image need to be assigned new grayscale values; the corresponding position of each pixel in the corrected image in the original image is found through the inverse matrix; the grayscale value corresponding to the original position coordinates is obtained through bilinear interpolation, and the grayscale distribution of the corrected image is made more uniform, pixel discontinuity or aliasing is reduced, and some moiré patterns are eliminated.

[0179] See also Figure 6 The present application provides an embodiment of a method for analyzing the quality of moiré removal, comprising:

[0180] 601. Compress the first image by a multiple to generate a third image that matches the target display screen;

[0181] The magnification compression means reducing or enlarging the size of the first image to adapt to the number of pixels and size of the target display screen. The third image is obtained to provide a comparison image for subsequent analysis of the moiré removal quality.

[0182] It should be noted that to analyze the quality of moiré removal, it is necessary to capture the middle part of the third image to ensure that there is no interference from external factors such as characters in the captured image, so that the change of the moiré can be evaluated more accurately.

[0183] 602. Obtain a first spectrum diagram and a second spectrum diagram of the second image and the third image through Fourier transform;

[0184] Fourier transform is performed on the second image and the third image to obtain their corresponding first spectrogram and second spectrogram. The spectrogram shows the representation of the image in the frequency domain, which contains the intensity and phase information of various frequency components in the image.

[0185] 603. Extracting a first moiré feature region and a second moiré feature region from the first spectrum graph and the second spectrum graph;

[0186] The corresponding moiré feature regions are identified and extracted in the first spectrum map and the second spectrum map. Through the image processing in the above steps, the first spectrum map and the second spectrum map will produce different features, which are the theoretical basis for analyzing and comparing the moiré removal effects in the two images.

[0187] 604 . Perform comparative analysis on the first moiré feature region and the second moiré feature region to generate a moiré removal quality analysis.

[0188] From the comparison and analysis between the first spectrum and the second spectrum, it can be seen that in the frequency concentration area on both sides of the central axis of the first spectrum, a large bright area appears in the severely compressed moiré pattern; while the second spectrum generated by this method does not have the large bright area phenomenon;

[0189] The quality analysis of moiré removal shows that the method of this patent can improve the removal effect of moiré.

[0190] In this embodiment, Fourier transform converts the image from the spatial domain to the frequency domain, which facilitates the analysis of the frequency components in the image. The spectrum provides an intuitive representation of the frequency components in the image, which helps to identify and analyze the moiré features in the image; the characteristic areas of the two spectrums are compared and analyzed to evaluate the moiré removal effect of the present method.

[0191] See also Figure 7 The present application provides an embodiment of a method for enhancing moiré features, comprising:

[0192] 701. Acquire a moiré-enhanced image, wherein the moiré-enhanced image has the same resolution as the second image and the moiré-enhanced image has the same resolution as the first image before compression, and the moiré-enhanced image has moiré features when displayed on a target display screen;

[0193] In this embodiment, the terminal first obtains a moiré-enhanced image. The resolution of the moiré-enhanced image before compression is the same as that of the first image, and the resolution of the moiré-enhanced image after compression is the same as that of the second image. When the moiré-enhanced image is displayed on the target display screen, obvious moiré features can be observed, which specifically needs to reach the level of visualization.

[0194] It should be noted that the moiré-enhanced image needs to be similar to the content of the first image, and can be two images taken in the same scene with different camera parameters. The moiré-enhanced image can also be the aforementioned third image, so that the content of the two images is exactly the same, and the subsequent feature area enhancement can also be accurately enhanced. The premise is that the third image needs to have visible moiré. If very obvious moiré does not appear, a special moiré-enhanced image is still required.

[0195] 702. Determine a moiré region of the moiré enhanced image through binarization processing;

[0196] The terminal determines the moiré area of ​​the moiré enhanced image through binarization processing. Because the grayscale value of the moiré is significantly different from the grayscale value of the pixels in the normal area and has reached the level of visualization, the moiré area can be divided on the moiré enhanced image through binarization processing.

[0197] 703. Determine a region to be enhanced of the second image by using a moiré region of the moiré enhanced image;

[0198] After dividing the moiré area on the moiré enhanced image through binarization processing, the terminal maps the moiré area to the second image to determine the area to be enhanced on the second image. In fact, this area to be enhanced is equivalent to the area where moiré is prone to appear, but the moiré in this area is very obvious and needs to be enhanced through feature enhancement.

[0199] 704. Obtain a first grayscale mean value and a second grayscale mean value of the area to be enhanced and the moiré area respectively;

[0200] The terminal obtains the first grayscale mean value of the area to be enhanced and the moiré area respectively and the second grayscale mean .

[0201] 705. Obtain a first standard deviation and a second standard deviation of the region to be enhanced and the moiré region respectively;

[0202] The terminal obtains the first standard deviation of the area to be enhanced and the moiré area respectively and the second standard deviation .

[0203] 706. Calculate a moiré adjustment coefficient according to the first grayscale mean and the second grayscale mean;

[0204] Next, the terminal calculates the grayscale mean according to the first grayscale mean. and the second grayscale mean Calculate the moiré adjustment coefficient using the following formula:

[0205]

[0206] in, is a very small constant value to prevent the denominator from being 0. The ratio of the second grayscale mean value of the moiré region to the first grayscale mean value of the region to be enhanced is used as a part of the adjustment, and then the adjustment is performed to obtain a moiré adjustment coefficient suitable for moiré enhancement processing.

[0207] 707. Calculate a moiré contrast enhancement coefficient according to the first standard deviation and the second standard deviation;

[0208] Next, the terminal calculates the first standard deviation and the second standard deviation Calculate the moiré contrast enhancement coefficient using the following formula:

[0209]

[0210] in, is a very small constant value to prevent the denominator from being zero.

[0211] 708. Perform moiré enhancement processing on the pixel points in the area to be enhanced on the second image according to the moiré adjustment coefficient and the moiré contrast enhancement coefficient to enhance the moiré features on the second image.

[0212] Finally, the terminal performs moiré enhancement processing on the pixels on the second image according to the moiré adjustment coefficient and the moiré contrast enhancement coefficient to enhance the moiré features on the second image. The formula is as follows:

[0213]

[0214] in, is the grayscale of the pixel with coordinates (x, y) after enhancement, is the grayscale of the pixel with coordinates (x, y) before enhancement. R is the moiré adjustment coefficient, and T is the moiré contrast enhancement coefficient. The enhancement parameters calculated by the two can effectively enhance the moiré traces on the enhanced area, while it will not have a significant impact on the pixels without moiré.

[0215] Through the above-mentioned enhancement processing, some subtle but existing moiré patterns in the second image can be improved. In the subsequent analysis and detection, the moiré patterns in the enhanced area can be better detected, thereby improving the accuracy of the moiré elimination quality analysis.

[0216] Please refer to Fig.14 and Fig.15 , Fig.14 This is a schematic diagram of the moiré elimination of the traditional compression method. Fig.15 This is a schematic diagram of the moiré elimination of this application. It can be seen that Fig.15 The elimination effect is better.

[0217] See also Figure 8 The present application also provides a device for eliminating moiré by combining linear transformation, comprising:

[0218] A first acquisition unit 801 is used to acquire a first image and search for a valid area in the first image;

[0219] Optionally, the first acquiring unit 801 includes:

[0220] A first image is acquired, and a grayscale edge detection method is used to search for a valid area in the first image.

[0221] A first calculation unit 802 is used to calculate the coordinates of the corner points of the effective area;

[0222] The first generating unit 803 is used to perform geometric correction on the corner point coordinates according to the resolution of the target display screen to generate corrected corner point coordinates;

[0223] Optionally, the first generating unit 803 includes:

[0224] Calculate the long side distance of the effective area according to the corner point coordinates;

[0225] The actual magnification is calculated based on the resolution and long side distance of the target display;

[0226] Generate corrected corner coordinates using true magnification and resolution.

[0227] The second generating unit 804 is used to generate a transformation matrix according to the corner point coordinates and the corrected corner point coordinates;

[0228] A conversion unit 805, configured to convert the first image into a rectified image using a conversion matrix;

[0229] A second calculation unit 806 is used to calculate the pixel grayscale value of the corrected image by using a bilinear interpolation method and combining the grayscale of the effective area of ​​the first image, and insert the pixel grayscale value into the corrected image;

[0230] Optionally, the second computing unit 806 includes:

[0231] Determine the position of the pixel to be inserted in the corrected image;

[0232] Calculate the original position coordinates of the pixel point to be inserted on the first image according to the inverse matrix of the transformation matrix;

[0233] Determine four adjacent integer coordinate points corresponding to the original position coordinates, and obtain gray value data of the four integer coordinate points;

[0234] Calculate the target grayscale according to the original position coordinates, the coordinates of four adjacent integer coordinate points, the grayscale value data and the preset bilinear interpolation formula;

[0235] Insert the target grayscale into the pixel position to be inserted in the corrected image.

[0236] A third generating unit 807 is used to calculate a scaling factor according to the actual magnification and the specified magnification, and generate a scaling vector according to the scaling factor;

[0237] The fourth generating unit 808 is used to perform affine transformation conversion compression on the rectified image to generate a second image according to the scaling vector;

[0238] A filtering unit 809, configured to perform mean filtering on the second image;

[0239] A first processing unit 810 is used to process the grayscale values ​​of the pixels of the second image using a bilinear interpolation method;

[0240] The fifth generating unit 811 is used for compressing the first image by a multiple to generate a third image matching the target display screen;

[0241] A third acquisition unit 812 is used to acquire a moiré-enhanced image, wherein the moiré-enhanced image has the same resolution as the second image and the same resolution as the first image before compression, and the moiré-enhanced image has moiré features when displayed on a target display screen;

[0242] A first determining unit 813 is used to determine the moiré region of the moiré enhanced image by binarization processing;

[0243] A second determining unit 814, configured to determine a to-be-enhanced region of the second image by using a moiré region of the moiré enhanced image;

[0244] A fourth acquisition unit 815 is used to respectively acquire a first grayscale mean value and a second grayscale mean value of the area to be enhanced and the moiré area;

[0245] A fifth acquiring unit 816, configured to acquire a first standard deviation and a second standard deviation of the region to be enhanced and the moiré region respectively;

[0246] A third calculation unit 817, configured to calculate a moiré adjustment coefficient according to the first grayscale mean and the second grayscale mean;

[0247] A fourth calculation unit 818, configured to calculate a moiré contrast enhancement coefficient according to the first standard deviation and the second standard deviation;

[0248] The second processing unit 819 is used to perform moiré enhancement processing on the pixel points in the area to be enhanced on the second image according to the moiré adjustment coefficient and the moiré contrast enhancement coefficient, so as to enhance the moiré features on the second image;

[0249] A second acquisition unit 820 is used to acquire a first spectrum diagram and a second spectrum diagram of the second image and the third image through Fourier transform;

[0250] An extraction unit 821, configured to extract a first moiré feature region and a second moiré feature region from the first spectrum graph and the second spectrum graph;

[0251] The sixth generating unit 822 is used to perform comparative analysis on the first moiré feature region and the second moiré feature region to generate a moiré removal quality analysis.

[0252] See also Fig. 9 The present application also provides a device for eliminating moiré by combining linear transformation, comprising:

[0253] Processor 901, memory 902, input and output unit 903, bus 909;

[0254] The processor 901 is connected to the memory 902, the input and output unit 903 and the bus 904;

[0255] The memory 902 stores a program, and the processor 901 calls the program to execute any of the above methods.

[0256] The present application also relates to a computer-readable storage medium, on which a program is stored. When the program is run on a computer, the computer executes any of the above methods.

[0257] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0258] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0259] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0260] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0261] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A method for eliminating moiré by combining linear transformations, characterized in that: The method comprises: Acquire a first image, and search for a valid area in the first image; Calculating the coordinates of the corner points of the effective area; Performing geometric correction on the corner point coordinates according to the resolution of the target display screen to generate corrected corner point coordinates; Generate a transformation matrix according to the corner point coordinates and the corrected corner point coordinates; converting the first image into a rectified image using the transformation matrix; Calculating the pixel grayscale value of the corrected image by using a bilinear interpolation method and combining the grayscale of the effective area of ​​the first image, and inserting the pixel grayscale value into the corrected image; Calculate the scaling factor according to the actual magnification and the specified magnification, and generate a scaling vector according to the scaling factor; According to the scaling vector, the corrected image is affine transformed and compressed to generate a second image; Processing the grayscale values ​​of the pixels of the second image using a bilinear interpolation method; Compressing the first image by a multiple to generate a third image matching the target display screen; Acquire a first spectrum diagram and a second spectrum diagram of the second image and the third image by Fourier transform; Extracting a first moiré feature region and a second moiré feature region of the first spectrum graph and the second spectrum graph; A comparative analysis is performed on the first moiré feature region and the second moiré feature region to generate a moiré removal quality analysis.

2. The method according to claim 1, characterized in that The acquiring of the first image and searching for a valid area in the first image comprises: A first image is acquired, and a grayscale edge detection method is used to search for a valid area in the first image.

3. The method according to claim 1, characterized in that After performing affine transformation conversion and compression on the rectified image to generate a second image according to the scaling vector, and before using bilinear interpolation to process the grayscale values ​​of the pixels of the second image, the method further includes: Perform mean filtering on the second image.

4. The method according to claim 1, characterized in that The corner point coordinates are geometrically corrected according to the resolution of the target display screen to generate corrected corner point coordinates, including: Calculate the long side distance of the effective area according to the corner point coordinates; Calculating the actual magnification according to the resolution of the target display screen and the long side distance; Corrected corner point coordinates are generated using the actual magnification and the resolution.

5. The method according to claim 1, characterized in that Calculating the pixel grayscale value of the corrected image by using a bilinear interpolation method and combining the grayscale of the effective area of ​​the first image, and inserting the pixel grayscale value into the corrected image, comprising: Determine the position of the pixel to be inserted in the corrected image; Calculating the original position coordinates of the position of the pixel point to be inserted on the first image according to the inverse matrix of the transformation matrix; Determine four adjacent integer coordinate points corresponding to the original position coordinates, and obtain gray value data of the four integer coordinate points; Calculating a target grayscale according to the original position coordinates, the coordinates of four adjacent integer coordinate points, the grayscale value data and a preset bilinear interpolation formula; The target grayscale is inserted into the position of the pixel to be inserted in the corrected image.

6. The method according to claim 1, characterized in that After compressing the first image to generate a third image matching the target display screen, and before acquiring the first spectrum graph and the second spectrum graph of the second image and the third image by Fourier transform, the method further includes: Acquire a moiré-enhanced image, wherein the moiré-enhanced image has the same resolution as the second image and the moiré-enhanced image has the same resolution as the first image before compression, and the moiré-enhanced image exhibits moiré features when displayed on the target display screen; Determine the moiré region of the moiré enhanced image by binarization processing; Determining a region to be enhanced in the second image by using a moiré region of the moiré enhanced image; Obtaining a first grayscale mean value and a second grayscale mean value of the area to be enhanced and the moiré area respectively; Obtaining a first standard deviation and a second standard deviation of the area to be enhanced and the moiré area respectively; Calculating a moiré adjustment coefficient according to the first grayscale mean and the second grayscale mean; Calculate the moiré contrast enhancement coefficient according to the first standard deviation and the second standard deviation; Moire enhancement processing is performed on the pixel points in the area to be enhanced on the second image according to the moire adjustment coefficient and the moire contrast enhancement coefficient to enhance the moire features on the second image.

7. A device for eliminating moiré by combining linear transformations, characterized in that: The device comprises: A first acquisition unit, configured to acquire a first image and search for a valid area in the first image; A first calculation unit, used to calculate the coordinates of the corner points of the effective area; A first generating unit, configured to perform geometric correction on the corner point coordinates according to a resolution of a target display screen to generate corrected corner point coordinates; A second generating unit, used for generating a transformation matrix according to the corner point coordinates and the corrected corner point coordinates; a conversion unit, configured to convert the first image into a rectified image using the conversion matrix; A second calculation unit, configured to calculate the pixel grayscale value of the corrected image by using a bilinear interpolation method and combining the grayscale of the effective area of ​​the first image, and insert the pixel grayscale value into the corrected image; A third generating unit is used to calculate a scaling factor according to the actual magnification and the specified magnification, and generate a scaling vector according to the scaling factor; a fourth generating unit, configured to perform affine transformation conversion compression on the rectified image according to the scaling vector to generate a second image; A first processing unit, configured to process the grayscale values ​​of pixels of the second image by using a bilinear interpolation method; A fifth generating unit, used for compressing the first image by a multiple to generate a third image matching the target display screen; A second acquisition unit, configured to acquire a first spectrum diagram and a second spectrum diagram of the second image and the third image through Fourier transform; An extraction unit, used for extracting a first moiré feature region and a second moiré feature region from the first spectrum graph and the second spectrum graph; The sixth generating unit is used to perform comparative analysis on the first moiré feature area and the second moiré feature area to generate a moiré removal quality analysis.

8. A device for eliminating moiré by combining linear transformations, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, performs the method according to any one of claims 1 to 6.

Images