An image processing method and related device for multi-directional edge texture filtering

Abstract

The present application discloses an image processing method and related device for multi-directional edge texture filtering, which are used to improve the accuracy and robustness of edge detection. The present application method includes: obtaining a captured image of a display screen to be detected, and performing corner correction on the captured image to obtain a standard image; generating a target weight sequence based on a filtering step size parameter and a sine function, and selectively reversing the target weight sequence according to an input polarity parameter, the polarity parameter is used to control the edge detection direction, and the edge detection direction includes positive polarity edge detection, negative polarity edge detection, and bipolar edge detection; constructing a target convolution kernel according to the input direction parameter, polarity parameter, and target weight sequence, the direction parameter is used to control the filtering direction, and the filtering direction includes column direction filtering, row direction filtering, and multi-directional filtering; performing a convolution operation on the standard image based on the target convolution kernel, and integrating the output image according to the polarity parameter and the direction parameter to obtain the target image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and related device for multi-directional edge texture filtering. Background Art

[0002] With the rapid development of display technology, especially the widespread application of high-end display screens such as LCD, OLED and MicroLED, the demand for quality control in the production process is increasing. AOI inspection technology uses automated means to conduct real-time inspection of display screens, which can effectively identify defects that occur during the production process, such as bright spots, dark spots, color differences, scratches and other visual defects. This not only improves the accuracy and efficiency of inspection, but also reduces manual intervention and improves the stability of the production line.

[0003] During the display manufacturing process, uneven materials, improper backlight system design, or inaccurate display component installation can cause a large area of ​​uneven brightness. This defect is usually manifested as uneven distribution of brightness or color in certain areas of the display. Since this unevenness is often gradual rather than a clear boundary, the detection algorithm needs to be very sensitive to identify these tiny changes.

[0004] In the prior art, edge detection and texture enhancement are the key steps in the defect image detection and analysis. Traditional edge detection methods usually detect edge features in images through a single-directional filter, which is easily affected by noise and complex backgrounds. When processing images with complex textures or multi-directional edges, there are also problems such as poor directional selectivity, poor noise resistance, and low processing efficiency, resulting in unsatisfactory final detection results. Summary of the invention

[0005] The present application provides an image processing method and related device for multi-directional edge texture filtering, which are used to improve the accuracy and robustness of edge detection.

[0006] The first aspect of the present application provides an image processing method for multi-directional edge texture filtering, comprising:

[0007] Acquire a captured image of the display screen to be inspected, and perform corner correction on the captured image to obtain a standard image;

[0008] Generate a target weight sequence based on a filter step size parameter and a sine function, and selectively invert the target weight sequence according to an input polarity parameter, wherein the polarity parameter is used to control an edge detection direction, wherein the edge detection direction includes positive polarity edge detection, negative polarity edge detection, and bipolarity edge detection;

[0009] Constructing a target convolution kernel according to the input direction parameter, the polarity parameter and the target weight sequence, wherein the direction parameter is used to control the filtering direction, and the filtering direction includes column direction filtering, row direction filtering and multi-directional filtering;

[0010] Performing a convolution operation on the standard image based on the target convolution kernel to obtain a filtered output image;

[0011] The output image is integrated according to the polarity parameter and the direction parameter to obtain a target image.

[0012] Optionally, after performing a convolution operation on the standard image based on the target convolution kernel to obtain a filtered output image, the method further includes:

[0013] Calculating an offset according to the filtering step size parameter and the polarity parameter, and generating an affine transformation matrix according to the offset and the direction parameter;

[0014] The output image is geometrically transformed and corrected based on the affine transformation matrix.

[0015] Optionally, the performing geometric transformation correction on the output image based on the affine transformation matrix includes:

[0016] Applying the affine transformation matrix to the domain of the output image to obtain a target domain;

[0017] Filling a preset minimum grayscale value in the target definition domain;

[0018] The affine transformation matrix is ​​applied to the filled output image to perform a translation operation, and bilinear interpolation is used to perform pixel value resampling.

[0019] Optionally, before generating the target weight sequence based on the filtering step size parameter and the sine function, the method further includes:

[0020] The filtering step size parameter is calculated by combining the texture periodicity and edge features of the standard image.

[0021] Optionally, the calculating of the filtering step size parameter in combination with the texture periodicity and edge features of the standard image includes:

[0022] Performing spectrum feature analysis on the standard image using Fourier transform to obtain spectrum amplitude of the standard image, and determining a local maximum value in the spectrum amplitude;

[0023] The texture frequency of the standard image is calculated according to the local maximum value, and the filtering step size parameter is calculated according to the texture frequency.

[0024] Optionally, generating a target weight sequence based on a filtering step size parameter and a sine function includes:

[0025] Generate a sine value sequence according to the filter step size parameter and the sine function;

[0026] Generate a first weight sequence based on the sine value sequence by linear interpolation and the sine function, and invert the first weight sequence to generate a second weight sequence;

[0027] Combining the first weight sequence and the second weight sequence to obtain a target weight sequence;

[0028] The first weight sequence Single2 is: ;

[0029] The second weight sequence Single1 is: ;

[0030] Where Bias is the initial offset, is the filtering step size parameter, is the sine value sequence, k is a positive integer, and reverse represents the inversion process.

[0031] Optionally, performing corner correction on the captured image to obtain a standard image includes:

[0032] Determine a set of boundary point coordinates of a display screen area in the captured image, and perform straight line fitting on the set of boundary point coordinates to obtain a plurality of line segments enclosing the display screen area;

[0033] Determine a set of actual corner points of the display screen area according to the intersection points of the plurality of line segments, and intercept an initial image surrounded by the set of actual corner points;

[0034] Calculate a theoretical corner point set according to the display screen to be detected and the camera internal parameters;

[0035] A projection transformation matrix is ​​determined according to the actual corner point set and the theoretical corner point set, and the captured image is corrected using the projection transformation matrix to obtain a standard image.

[0036] A second aspect of the present application provides an image processing system for multi-directional edge texture filtering, comprising:

[0037] An acquisition unit, used for acquiring a captured image of the display screen to be detected, and performing corner correction on the captured image to obtain a standard image;

[0038] A generating unit, for generating a target weight sequence based on a filter step size parameter and a sine function, and selectively inverting the target weight sequence according to an input polarity parameter, wherein the polarity parameter is used to control an edge detection direction, wherein the edge detection direction includes positive polarity edge detection, negative polarity edge detection, and bipolarity edge detection;

[0039] A construction unit, used to construct a target convolution kernel according to an input direction parameter, the polarity parameter and the target weight sequence, wherein the direction parameter is used to control a filtering direction, and the filtering direction includes column-direction filtering, row-direction filtering and multi-direction filtering;

[0040] A convolution unit, configured to perform a convolution operation on the standard image based on the target convolution kernel to obtain a filtered output image;

[0041] An integration unit is used to integrate the output image according to the polarity parameter and the direction parameter to obtain a target image.

[0042] Optionally, the system further comprises: a correction unit, wherein the correction unit is used to:

[0043] Calculating an offset according to the filtering step size parameter and the polarity parameter, and generating an affine transformation matrix according to the offset and the direction parameter;

[0044] The output image is geometrically transformed and corrected based on the affine transformation matrix.

[0045] Optionally, the correction unit is specifically used for:

[0046] Applying the affine transformation matrix to the domain of the output image to obtain a target domain;

[0047] Filling a preset minimum grayscale value in the target definition domain;

[0048] The affine transformation matrix is ​​applied to the filled output image to perform a translation operation, and bilinear interpolation is used to perform pixel value resampling.

[0049] Optionally, the system further includes: a computing unit, wherein the computing unit is configured to:

[0050] The filtering step size parameter is calculated by combining the texture periodicity and edge features of the standard image.

[0051] Optionally, the computing unit is specifically used for:

[0052] Performing spectrum feature analysis on the standard image using Fourier transform to obtain spectrum amplitude of the standard image, and determining a local maximum value in the spectrum amplitude;

[0053] The texture frequency of the standard image is calculated according to the local maximum value, and the filtering step size parameter is calculated according to the texture frequency.

[0054] Optionally, the generating unit is specifically used for:

[0055] Generate a sine value sequence according to the filter step size parameter and the sine function;

[0056] Generate a first weight sequence based on the sine value sequence by linear interpolation and the sine function, and invert the first weight sequence to generate a second weight sequence;

[0057] Combining the first weight sequence and the second weight sequence to obtain a target weight sequence;

[0058] The first weight sequence Single2 is: ;

[0059] The second weight sequence Single1 is: ;

[0060] Where Bias is the initial offset, is the filtering step size parameter, is the sine value sequence, k is a positive integer, and reverse represents the inversion process.

[0061] Optionally, the acquiring unit is specifically used for:

[0062] Determine a set of boundary point coordinates of a display screen area in the captured image, and perform straight line fitting on the set of boundary point coordinates to obtain a plurality of line segments enclosing the display screen area;

[0063] Determine a set of actual corner points of the display screen area according to the intersection points of the plurality of line segments, and intercept an initial image surrounded by the set of actual corner points;

[0064] Calculate a theoretical corner point set according to the display screen to be detected and the camera internal parameters;

[0065] A projection transformation matrix is ​​determined according to the actual corner point set and the theoretical corner point set, and the captured image is corrected using the projection transformation matrix to obtain a standard image.

[0066] A third aspect of the present application provides an image processing device for multi-directional edge texture filtering, the device comprising:

[0067] Processor, memory, input-output unit, and bus;

[0068] The processor is connected to the memory, the input and output unit, and the bus;

[0069] The memory stores a program, and the processor calls the program to execute the first aspect and any optional multi-directional edge texture filtering image processing method in the first aspect.

[0070] A fourth aspect of the present application provides a computer-readable storage medium having a program stored thereon. When the program is executed on a computer, the program executes the image processing method of the first aspect and any optional multi-directional edge texture filtering in the first aspect.

[0071] It can be seen from the above technical solutions that this application has the following advantages:

[0072] First, the target weight sequence is generated by filtering step parameters and sine functions, which effectively suppresses the leakage of high-frequency noise, and selectively inverts according to polarity parameters, thereby achieving a comprehensive response to positive, negative and bipolar edges. The convolution kernel is constructed by controlling the filter direction according to the input direction parameter, and the convolution operation is performed using the convolution kernel, so that the filtered output image can carefully highlight the edge changes in the image. The integration operation ensures the integration of the detection results of each direction and polarity, and finally provides a high-precision target image. By adjusting the input direction parameters and polarity parameters, the comprehensive detection of multi-polarity and multi-directional edges can be achieved, which can effectively capture small edge features, improve the accuracy and robustness of edge detection, and is suitable for image processing in complex texture environments such as display screens. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0074] Figure 1 A schematic flow chart of an embodiment of an image processing method for multi-directional edge texture filtering provided by the present application;

[0075] Figure 2 A schematic diagram of the processing effect of the image processing method of multi-directional edge texture filtering provided in this application;

[0076] Figure 3 A flowchart of another embodiment of the image processing method for multi-directional edge texture filtering provided by the present application;

[0077] Figure 4 A schematic diagram of another embodiment of the structure of the image processing system for multi-directional edge texture filtering provided by the present application;

[0078] Figure 5A schematic structural diagram of an embodiment of an image processing device for multi-directional edge texture filtering provided in the present application. DETAILED DESCRIPTION

[0079] The present application provides an image processing method and related device for multi-directional edge texture filtering, which are used to improve the accuracy and robustness of edge detection.

[0080] It should be noted that the image processing method for multi-directional edge texture filtering provided in this application can be applied to a terminal or a server. For example, the terminal can be a smart phone or a computer, a tablet computer, a smart TV, a smart watch, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of explanation, this application uses a terminal as an example for the execution subject.

[0081] See also Figure 1 , Figure 1 An embodiment of an image processing method for multi-directional edge texture filtering provided by the present application comprises:

[0082] 101. Obtain a captured image of the display screen to be inspected, and perform corner correction on the captured image to obtain a standard image;

[0083] In the quality control of display screens, it is usually necessary to detect the uniformity, brightness, color, surface flaws, uneven brightness, etc. In order to effectively detect the display screen, it is necessary to capture the actual image of the display screen when it is in working state (such as displaying a certain picture or a black screen), that is, to obtain the photographed image of the display screen to be tested.

[0084] During shooting, due to factors such as the camera angle, focal length, and distortion of the imaging system, the captured image may be geometrically deformed. For example, perspective distortion may occur due to an incorrect shooting angle. This deformation makes it difficult for subsequent image processing to proceed normally. Therefore, after obtaining the captured image of the display screen to be tested, it is necessary to use a corner correction method to eliminate the geometric distortion and convert the image into a standard display screen image. The purpose of corner correction is to eliminate the geometric distortion of the image and correct the display screen area in the captured image to a shape corresponding to the display screen.

[0085] 102. Generate a target weight sequence based on a filter step size parameter and a sine function, and selectively invert the target weight sequence according to an input polarity parameter, wherein the polarity parameter is used to control an edge detection direction, and the edge detection direction includes positive polarity edge detection, negative polarity edge detection, and bipolarity edge detection;

[0086] The filter in edge detection often needs to be weighted averaged according to the local pixels of the image. The choice of weights directly affects the filter's response to different structures in the image (such as edges, textures, etc.). As a smooth periodic function, the sine function can be used to generate a smoothly changing weight sequence. The filter step length parameter controls the step interval of each position in the weight sequence and determines the amplitude of change of each weight. The filter step length parameter can be set according to the actual application situation and is not limited here. According to the filter step length parameter combined with the sine function, a series of values ​​that gradually change from positive to negative can be generated within a certain filter step length range, that is, the target weight sequence is generated. Using the sine function to generate the target weight sequence helps reduce the leakage of high-frequency components in the image and avoid excessive noise caused by sharpening.

[0087] After generating the target weight sequence, it is also necessary to selectively reverse the target weight sequence according to the polarity parameter. The polarity parameter (Polarity) controls the direction of edge detection. Polarity determines whether the edge response is based on the change from "light to dark" or "dark to light" in the image. In actual display defect detection, the brightness and color changes in the image are not always unidirectional. In many cases, there are gradients from light to dark and dark to light. Therefore, by changing the polarity parameter, the detection algorithm can make the most appropriate edge response in different scenarios. By selectively reversing the target weight sequence through the polarity parameter, the detection direction can be flexibly adjusted to meet the needs of different scenarios.

[0088] In this embodiment, the edge detection direction specifically includes positive polarity edge detection, negative polarity edge detection and bipolar edge detection, which are described below respectively:

[0089] Positive polarity edge detection (Polarity=0): Edge detection is sensitive to light-to-dark changes (transitions from light areas to dark areas in the image). In this mode, the generated target weight sequence remains unchanged, that is, the positive and negative weight values ​​are arranged in the natural order of the sine function.

[0090] Negative polarity edge detection (Polarity=1): Edge detection is sensitive to dark-to-light changes (transition from dark areas to light areas). In this mode, the direction of the response to the edge is changed by reversing the target weight sequence, that is, swapping the original positive and negative values.

[0091] Bipolar edge detection (Polarity=2): This mode is sensitive to both light-to-dark and dark-to-light changes, that is, it detects both rising and falling edges in the image. In this mode, the target weight sequence considers both positive and negative changes, and in subsequent steps, bipolar edge detection is achieved by integrating these two changes and adding the results of positive and negative edge detection.

[0092] 103. Construct a target convolution kernel according to the input direction parameter, polarity parameter and target weight sequence. The direction parameter is used to control the filtering direction. The filtering direction includes column direction filtering, row direction filtering and multi-directional filtering.

[0093] In this embodiment, the construction of the convolution kernel for filtering needs to consider not only the size of the weight, but also the directionality of the convolution kernel according to the direction parameter and polarity parameter to ensure that the convolution kernel can adapt to different detection requirements. The direction parameter (Direction) determines the filtering direction of the convolution kernel, and the filtering direction includes column-direction filtering, row-direction filtering, and multi-direction filtering, which are described below:

[0094] Column-wise filtering (Direction=0): The weights of the convolution kernels are arranged along the column direction of the image, so that the filtering operation mainly responds to the edges in the up and down directions, that is, to the vertical edges.

[0095] Row-wise filtering (Direction=1): The weights of the convolution kernel are arranged along the row direction of the image, responding to the edges in the left and right directions, that is, responding to the horizontal edges.

[0096] Multi-directional filtering (Direction=2): Multi-directional filtering refers to combining convolution kernels in multiple directions to respond to edge changes in multiple directions at the same time, and can detect vertical and horizontal edge features in the image. It should be noted that for multi-directional filtering, convolution kernels in different directions can be generated separately, or convolution kernels in multiple directions can be combined into a composite convolution kernel, which is not limited here.

[0097] The direction parameter, polarity parameter and target weight sequence jointly determine the specific shape of the convolution kernel and its response to edge features in the image. Combining the direction parameter and polarity parameter, the convolution kernel that meets the needs of different scenes can be flexibly constructed, so that it can respond to specific edge or texture features in image processing. Specifically, the input direction parameter determines the shape and weight distribution of the convolution kernel, and the input polarity parameter determines the arrangement order of the target weight sequence. Positive polarity means weighting from light to dark, negative polarity means weighting from dark to light, and bipolarity means both respond at the same time. By controlling the input polarity parameter and direction parameter, the final generated convolution kernel can flexibly respond to uneven brightness and edge features in the image in different scenes, thereby effectively improving the accuracy of defect detection.

[0098] 104. Perform a convolution operation on the standard image based on the target convolution kernel to obtain a filtered output image;

[0099] Based on the target convolution kernel constructed in step 103, multiple convolution operations are performed on the standard image, and the image is processed by the target convolution kernel. At each position, the convolution kernel and the pixel value of the image area are weighted and summed to generate a new image pixel value, and then the edge texture features in the image are extracted to obtain a filtered output image. Through the convolution operation, the edge or texture in the image will be enhanced: for positive polarity filtering, the edge from light to dark in the image will be enhanced; for negative polarity filtering, the edge from dark to light in the image will be enhanced; for multi-directional filtering, multiple edge features in the image will be enhanced at the same time.

[0100] 105. Integrate the output image according to the polarity parameter and the direction parameter to obtain the target image.

[0101] In step 104, in particular, when there is bipolar or multi-directional filtering, multiple output images will be generated. At this time, it is necessary to merge the results of multiple filtering, that is, integrate these output images to obtain the final target image. The purpose of the integration is to ensure that the edge features of different directions and polarities can be responded to simultaneously in the final target image. If it is multi-directional filtering or bipolar detection, it is necessary to add the convolution results of each direction or polarity to obtain the final target image. Specifically, for the case of bipolar filtering (Polarity=2), it is necessary to perform positive and negative polarity filtering operations respectively, and then merge the results of the two. The merging operation can be performed by simple addition, or different weights are given to different results according to the importance of the image to ensure that the target image can respond to both polarity edge changes. For the case of multi-directional filtering (Direction=2), it is necessary to merge the convolution results of the two directions to finally obtain a target image containing horizontal and vertical edge features.

[0102] In the image of uneven brightness defect in a large area of ​​the display screen, the edge transformation may have multi-directional and multi-polar characteristics, so it is necessary to integrate the output image according to the polarity parameter and the direction parameter. The integrated target image will contain more edge information, which can more accurately detect the potential defect area or texture features in the image, thereby effectively extracting the uneven brightness defect in the display screen image. The following is an example of one of the combinations (Direction=2 and Polarity=1):

[0103] Direction=2 and Polarity=1 means that both horizontal and vertical filtering are used and the enhancement map offset operation will be performed. Convolution filtering is performed on the image to be processed in the horizontal and vertical directions respectively, and the left-to-right search map Image_L_to_R, the right-to-left search map Image_R_to_L, the top-to-bottom search map Image_T_to_B, and the bottom-to-top search map Image_B_to_T are obtained. Image_L_to_R and Image_R_to_L are added to obtain the left-to-right filter map Image_LR, and the top-to-bottom filter map Image_TB is obtained in the same way. Finally, Image_LR and Image_TB are added to obtain the target image. Please refer to Figure 2 , Figure 2 (a) in the figure is the standard image. Figure 2 (b) in the figure is the final target image.

[0104] In this embodiment, the target weight sequence is first generated by filtering step parameters and sine functions, which effectively suppresses the leakage of high-frequency noise, and selectively reverses according to the polarity parameters, thereby achieving a comprehensive response to positive, negative and bipolar edges. The convolution kernel is constructed by controlling the filtering direction according to the input direction parameters, and the convolution kernel is used to perform a convolution operation, so that the filtered output image can carefully highlight the edge changes in the image, and the integration operation ensures the integration of the detection results of each direction and each polarity, and finally provides a high-precision target image. By adjusting the input direction parameters and polarity parameters, the comprehensive detection of multi-polarity and multi-directional edges can be achieved, and the fine edge features can be effectively captured, which improves the accuracy and robustness of edge detection, and is suitable for image processing in complex texture environments such as display screens.

[0105] The following is a detailed description of the image processing method for multi-directional edge texture filtering provided by this application. Figure 3 , Figure 3 Another embodiment of the image processing method for multi-directional edge texture filtering provided by the present application includes:

[0106] 301. Obtain a captured image of the display screen to be inspected, and perform corner correction on the captured image to obtain a standard image;

[0107] The purpose of corner point correction is to eliminate geometric distortion of the image and correct the display screen area in the captured image to a shape corresponding to the display screen. In this embodiment, the steps of corner point correction specifically include:

[0108] A1. Determine a set of boundary point coordinates of the display screen area in the captured image, and perform straight line fitting on the set of boundary point coordinates to obtain a number of line segments that wrap the display screen area;

[0109] The captured image is processed by an image processing algorithm (such as edge detection or threshold segmentation) to determine the coordinate set of the boundary points of the display area. These boundary points represent the outer contour of the display. Then, a straight line fitting algorithm (such as Hough transform) is used to process the boundary point coordinate set to obtain several straight lines that wrap the display area. The purpose of this step is to remove the noise effect in the image through geometric fitting and obtain the precise edge of the display.

[0110] A2. determining a set of actual corner points of the display screen area according to the intersection points of a plurality of line segments, and intercepting an initial image surrounded by the set of actual corner points;

[0111] By calculating the intersection points of several line segments, the actual corner points of the display screen area can be determined. These intersection points are the corners of the display screen. Using these actual corner points, the initial image surrounded by the actual corner points can be intercepted, that is, the image content in the display screen area in the captured image can be intercepted.

[0112] A3. Calculate a theoretical corner point set based on the display screen to be detected and the camera internal parameters; determine a projection transformation matrix based on the actual corner point set and the theoretical corner point set, and use the projection transformation matrix to correct the captured image to obtain a standard image.

[0113] According to the logical resolution of the display screen and the camera internal parameters, the theoretical corner point set of the display screen area in the captured image after shape and size correction can be calculated. The theoretical corner point set is an ideal position calculated based on the known display screen size and camera mapping, usually a standardized rectangular or quadrilateral coordinate set. By comparing the position difference between the actual corner point set and the theoretical corner point set, a projection transformation matrix can be calculated. The projection transformation matrix is ​​used to convert the actual corner point set in the captured image to the ideal position of the theoretical corner point set. Finally, the calculated projection transformation matrix is ​​used to perform perspective transformation on the captured image to obtain a standardized image. This step ensures that the display screen area in the captured image is accurately corrected in size and shape, eliminates errors caused by factors such as camera angle and distortion, and provides a reliable image basis for subsequent multi-directional edge detection and texture analysis.

[0114] 302. Calculate the filtering step size parameter by combining the texture periodicity and edge features of the standard image;

[0115] The filter step size parameter can be used as an open parameter. Users can freely adjust the filter step size according to actual needs, so that the filter effect can be flexibly optimized according to different application scenarios and image features. For example, when processing images with rich details, using a smaller filter step size can retain more details, while in images with more noise or more complex backgrounds, a larger step size helps reduce the impact of noise and highlight the main edge features.

[0116] However, in this embodiment, an adaptive filtering step size mode is also provided, which can automatically calculate the step size based on the texture periodicity and edge features of the image, that is, the size of the step size can be automatically adjusted according to the texture periodicity and edge distribution of the image, so that the filter can automatically calculate the appropriate filtering step size parameters according to the image content to achieve the optimal processing effect.

[0117] Furthermore, in the adaptive filtering step mode, the main frequency component of the image can be extracted by Fourier transform to calculate the periodic texture length, and the filtering step parameter can be adjusted by observing the edge filtering effect and whether texture mismatch or artifacts occur. The terminal specifically performs the following operations:

[0118] B1. Use Fourier transform to analyze the spectrum characteristics of the standard image, obtain the spectrum amplitude of the standard image, and determine the local maximum value in the spectrum amplitude;

[0119] First, the standard image is subjected to Fourier transform to analyze the spectral characteristics of the image. Formula 1 is used to convert the periodic structure (such as texture periodicity) in the image into energy distribution in the frequency domain. Then, the spectrum amplitude is obtained according to Formula 2. After Gaussian smoothing, the local maximum is found through the maximum detection algorithm. In the spectrum amplitude diagram, the local maximum represents the peak value of a specific frequency component in the spectrum, which corresponds to the area with significant texture and structural changes in the image. By detecting the local maximum values ​​in the spectrum, the main texture frequency components of the image can be extracted. The specific formulas 1 and 2 used are as follows:

[0120] ;(Formula 1)

[0121] (Formula 2)

[0122] in is the gray value of the standard image in the spatial domain, is a complex value in the frequency domain, and v represent the horizontal and vertical frequency components, respectively, M and N are the width and height of the image, respectively, and RE and IM represent the real and imaginary parts of the complex number.

[0123] B2. The texture frequency of the standard image is calculated based on the local maximum value, and the filter step size parameter is calculated based on the texture frequency.

[0124] Determine the main texture frequency based on the local maximum point ( , ), the texture frequency represents the periodic changes of texture details in the image, so the filter step size parameter is calculated based on the texture frequency and formula 3.

[0125] (Formula 3)

[0126] Where W and H are the width and height of the image respectively, and are the filtering step size parameters in the x and y directions respectively.

[0127] Through Fourier transform-based spectrum analysis, the filter step size parameters can be adaptively adjusted according to the texture characteristics of the image, so that finer filter step sizes can be used in areas with complex textures to ensure accurate capture of texture details, while larger step sizes can be used in areas with simpler textures to improve processing efficiency and reduce unnecessary calculations.

[0128] 303. Generate a sine value sequence according to the filter step size parameter and the sine function;

[0129] In order to reduce the leakage of high-frequency components of the image, according to the filter step size parameter calculated in step 302, the sine function is combined with Formula 4 to generate a value from 0 to Sine value sequence, where k is a positive integer and Pitch is the filter step size parameter:

[0130] (Formula 4)

[0131] 304. Generate a first weight sequence based on the sine value sequence through linear interpolation and a sine function, and invert the first weight sequence to generate a second weight sequence;

[0132] Based on the sine value sequence, a preliminary first weight sequence is generated by linear difference and sine function using Formula 5 and rounded, where Bias is the initial offset:

[0133] (Formula 5)

[0134] By performing linear interpolation on the basis of the generated sine value sequence, the generated first weight sequence is made smoother, avoiding possible sharpening or discontinuous changes, and ensuring the continuity of the first weight sequence. After that, the first weight sequence Single2 is reversed to generate its reverse sequence, that is, the second weight sequence is generated. , The purpose of generating the second weight sequence is to create a symmetrical weight distribution during the filtering process to ensure a balanced response of the filter in both directions without biasing towards one direction.

[0135] 305. Combine the first weight sequence and the second weight sequence to obtain a target weight sequence;

[0136] The target weight sequence can be obtained by merging the first weight sequence and the second weight sequence , to achieve the control of weight symmetry in the edge detection process, thereby enhancing the sensitivity and robustness of image edge features.

[0137] It should be noted that the second weight sequence needs to be removed when merging The last element and first weight sequence 2 to avoid repeated calculation of the center value when merging.

[0138] 306. Selectively invert the target weight sequence according to the input polarity parameter, where the polarity parameter is used to control the edge detection direction, where the edge detection direction includes positive polarity edge detection, negative polarity edge detection, and bipolarity edge detection;

[0139] In this embodiment, it is necessary to determine whether the target weight sequence needs to be adjusted based on the input polarity parameter Polarity. The inversion is performed to select the arrangement of different target weight sequences, so as to control the edge detection direction, as shown in Formula 6:

[0140] (Formula 6)

[0141] right Round off to the nearest integer. If it is not 0, the weight ratio is further adjusted according to Formula 7:

[0142] (Formula 7)

[0143] 307. Construct a target convolution kernel according to the input direction parameter, polarity parameter and target weight sequence, wherein the direction parameter is used to control the filtering direction, and the filtering direction includes column direction filtering, row direction filtering and multi-directional filtering;

[0144] Construct the target convolution kernel according to the input direction parameter, polarity parameter and target weight sequence:

[0145] If Direction=0, it means column-wise filtering, and the corresponding target convolution kernel is:

[0146] (Formula 8)

[0147] If Direction=1, it means row-wise filtering, and the corresponding target convolution kernel is:

[0148] (Formula 9)

[0149] If Direction=2, two convolution kernels in the column direction and row direction are constructed simultaneously.

[0150] 308. Perform a convolution operation on the standard image based on the target convolution kernel to obtain a filtered output image;

[0151] In this embodiment, step 308 is similar to step 104 in the above embodiment and will not be described again here.

[0152] 309. Calculate an offset according to the filtering step size parameter and the polarity parameter, and generate an affine transformation matrix according to the offset and the direction parameter;

[0153] In this embodiment, considering that misalignment may occur when using the derivative convolution kernel, it is necessary to perform steps 309 to 310 to perform geometric transformation on the output image to correct the offset.

[0154] First, it is necessary to calculate the offset according to the filter step size parameter and the polarity parameter, and generate an affine transformation matrix for representing the geometric transformation according to the offset and the direction parameter. The specific calculation formula of the offset is shown in Formula 10:

[0155] (Formula 10)

[0156] Among them, Translate represents the offset, and Pitch is the filter step parameter. is the polarity parameter.

[0157] An affine transformation matrix HomMat2D can be generated according to the offset and direction parameters, specifically:

[0158] If Direction=0, it means translation in the column direction. ,

[0159] If Direction=1, it means translation in the row direction. .

[0160] 310. Perform geometric transformation correction on the output image based on the affine transformation matrix;

[0161] According to the affine transformation matrix generated in step 309, the output image is geometrically transformed and corrected to achieve the purpose of eliminating deformation, correcting edges, adjusting image position, etc., to ensure that the output image can be correctly aligned and possible geometric distortion is removed, and finally an output image with the correct position is obtained.

[0162] Specifically, the process of geometric transformation correction is: apply the affine transformation matrix to the definition domain of the output image to obtain the target definition domain; fill the preset minimum grayscale value in the target definition domain; apply the affine transformation matrix to the filled output image for translation operation, and use bilinear interpolation to resample the pixel values.

[0163] That is, apply the above affine transformation matrix to the definition domain ROI of the output image Perform an affine transformation to obtain a new domain position When performing geometric transformations, some new areas may appear in the image, that is, areas in the target domain that are not covered by the original image. Since these areas may not have corresponding pixels in the original image after the image transformation, in order to fill these blank areas, the output image after filtering is Fill the area with the preset minimum grayscale value . Finally, the affine transformation matrix is ​​used on the entire filtered image The translation operation is performed and the pixel value is resampled using bilinear difference. This operation ensures the compensation of the image in the boundary area of ​​the domain, so that the results are consistent before and after the transformation. Through these steps, the output image can be effectively corrected by geometric transformation, the dislocation problem caused by geometric distortion can be eliminated, the image offset caused by filter convolution can be corrected, and the detection accuracy can be further improved.

[0164] 311. Integrate the output image according to the polarity parameter and the direction parameter to obtain the target image.

[0165] In this embodiment, step 311 is similar to step 105 in the aforementioned embodiment and will not be described again here.

[0166] In this embodiment, the target weight sequence is first generated by adaptively calculating the filter step length parameter and the sine function, which effectively suppresses the leakage of high-frequency noise, and selectively reverses according to the polarity parameter, thereby achieving a comprehensive response to positive, negative and bipolar edges. The filter direction is controlled according to the input direction parameter to construct a convolution kernel, and the convolution kernel is used to perform a convolution operation, so that the filtered output image can carefully highlight the edge changes in the image, and the integration operation ensures the integration of the detection results of each direction and each polarity, and finally provides a high-precision target image. By adjusting the input direction parameter and polarity parameter, the comprehensive detection of multi-polarity and multi-directional edges can be achieved, and the fine edge features can be effectively captured, which improves the accuracy and robustness of edge detection and is suitable for image processing in complex texture environments such as display screens. In addition, this embodiment also calculates the periodic texture length by extracting the main frequency component of the image through Fourier transform, and adjusts the filter step length parameter by observing the edge filtering effect and whether texture mismatch or artifact occurs, so as to achieve an adaptive filter step length mode. In addition, a domain compensation strategy is also applied to the output image to correct the image offset caused by the filter convolution, thereby further improving the detection accuracy.

[0167] The following is a detailed description of the image processing system for multi-directional edge texture filtering provided by this application. Figure 4 , Figure 4 Another embodiment of the image processing system for multi-directional edge texture filtering provided by the present application comprises:

[0168] The acquisition unit 401 is used to acquire a captured image of the display screen to be detected, and perform corner correction on the captured image to obtain a standard image;

[0169] A generating unit 402, configured to generate a target weight sequence based on a filter step size parameter and a sine function, and selectively invert the target weight sequence according to an input polarity parameter, wherein the polarity parameter is used to control an edge detection direction, wherein the edge detection direction includes positive polarity edge detection, negative polarity edge detection, and bipolarity edge detection;

[0170] A construction unit 403 is used to construct a target convolution kernel according to the input direction parameter, polarity parameter and target weight sequence, wherein the direction parameter is used to control the filtering direction, and the filtering direction includes column direction filtering, row direction filtering and multi-directional filtering;

[0171] A convolution unit 404 is used to perform a convolution operation on the standard image based on the target convolution kernel to obtain a filtered output image;

[0172] The integration unit 405 is used to integrate the output images according to the polarity parameters and the direction parameters to obtain the target image.

[0173] Optionally, the system further includes: a correction unit 406, the correction unit 406 is used to:

[0174] An offset is calculated according to a filter step size parameter and a polarity parameter, and an affine transformation matrix is ​​generated according to the offset and the direction parameter;

[0175] The output image is geometrically transformed and corrected based on the affine transformation matrix.

[0176] Optionally, the correction unit 406 is specifically configured to:

[0177] Applying the affine transformation matrix to the domain of the output image yields the target domain;

[0178] Fill the target definition domain with the preset minimum grayscale value;

[0179] Apply an affine transformation matrix to the padded output image for translation and resample the pixel values ​​using bilinear interpolation.

[0180] Optionally, the system further includes: a computing unit 407, the computing unit 407 being configured to:

[0181] The filter step size parameters are calculated by combining the texture periodicity and edge features of the standard image.

[0182] Optionally, the calculation unit 407 is specifically configured to:

[0183] Use Fourier transform to analyze the spectrum characteristics of the standard image, obtain the spectrum amplitude of the standard image, and determine the local maximum value in the spectrum amplitude;

[0184] The texture frequency of the standard image is calculated based on the local maximum value, and the filter step size parameter is calculated based on the texture frequency.

[0185] Optionally, the generating unit 402 is specifically configured to:

[0186] Generate a sine value sequence according to the filter step size parameter and the sine function;

[0187] Generate a first weight sequence based on the sine value sequence by linear interpolation and a sine function, and invert the first weight sequence to generate a second weight sequence;

[0188] Merge the first weight sequence and the second weight sequence to obtain a target weight sequence;

[0189] The first weight sequence Single2 is: ;

[0190] The second weight sequence Single1 is: ;

[0191] Where Bias is the initial offset, is the filter step size parameter, is a sine value sequence, k is a positive integer, and reverse means reversal processing.

[0192] Optionally, the acquiring unit 401 is specifically configured to:

[0193] Determine a set of boundary point coordinates of a display screen area in a captured image, and perform straight line fitting on the set of boundary point coordinates to obtain a plurality of line segments that wrap the display screen area;

[0194] Determine the actual corner point set of the display screen area according to the intersection points of the plurality of line segments, and intercept the initial image surrounded by the actual corner point set;

[0195] Calculate the theoretical corner point set based on the display screen to be detected and the camera internal parameters;

[0196] The projection transformation matrix is ​​determined according to the actual corner point set and the theoretical corner point set, and the captured image is corrected using the projection transformation matrix to obtain a standard image.

[0197] In this embodiment, the functions of each unit are the same as those described above. Figure 1 or Figure 3 The steps in the method embodiment shown correspond to each other and will not be repeated here.

[0198] This application also provides an image processing device for multi-directional edge texture filtering. Figure 5 , Figure 5 An embodiment of an image processing device for multi-directional edge texture filtering provided by the present application includes:

[0199] Processor 501, memory 502, input and output unit 503, bus 504;

[0200] The processor 501 is connected to the memory 502, the input and output unit 503 and the bus 504;

[0201] The memory 502 stores a program, and the processor 501 calls the program to execute any of the above-mentioned image processing methods for multi-directional edge texture filtering.

[0202] The present application also relates to a computer-readable storage medium, on which a program is stored. When the program is run on a computer, the computer executes any of the above-mentioned image processing methods for multi-directional edge texture filtering.

[0203] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0204] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0205] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0206] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0207] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, etc. Various media that can store program codes.

Claims

1. An image processing method for multi-directional edge texture filtering, characterized in that: The image processing method comprises: Acquire a captured image of the display screen to be inspected, and perform corner correction on the captured image to obtain a standard image; Generate a target weight sequence based on a filter step size parameter and a sine function, and selectively invert the target weight sequence according to an input polarity parameter, wherein the polarity parameter is used to control an edge detection direction, wherein the edge detection direction includes positive polarity edge detection, negative polarity edge detection, and bipolarity edge detection; Constructing a target convolution kernel according to the input direction parameter, the polarity parameter and the target weight sequence, wherein the direction parameter is used to control the filtering direction, and the filtering direction includes column direction filtering, row direction filtering and multi-directional filtering; Performing a convolution operation on the standard image based on the target convolution kernel to obtain a filtered output image; The output image is integrated according to the polarity parameter and the direction parameter to obtain a target image.

2. The image processing method according to claim 1, characterized in that: After performing the convolution operation on the standard image based on the target convolution kernel to obtain a filtered output image, the method further includes: Calculating an offset according to the filtering step size parameter and the polarity parameter, and generating an affine transformation matrix according to the offset and the direction parameter; The output image is geometrically transformed and corrected based on the affine transformation matrix.

3. The image processing method according to claim 2, characterized in that: The performing geometric transformation correction on the output image based on the affine transformation matrix comprises: Applying the affine transformation matrix to the domain of the output image to obtain a target domain; Filling a preset minimum grayscale value in the target definition domain; The affine transformation matrix is ​​applied to the filled output image to perform a translation operation, and bilinear interpolation is used to perform pixel value resampling.

4. The image processing method according to claim 1, characterized in that: Before generating the target weight sequence based on the filtering step size parameter and the sine function, the method further includes: The filtering step size parameter is calculated by combining the texture periodicity and edge features of the standard image.

5. The image processing method according to claim 4, characterized in that: The step of calculating the filtering step size parameter by combining the texture periodicity and edge features of the standard image comprises: Performing spectrum feature analysis on the standard image using Fourier transform to obtain spectrum amplitude of the standard image, and determining a local maximum value in the spectrum amplitude; The texture frequency of the standard image is calculated according to the local maximum value, and the filtering step size parameter is calculated according to the texture frequency.

6. The image processing method according to any one of claims 1 to 5, characterized in that: The generating of the target weight sequence based on the filtering step size parameter and the sine function comprises: Generate a sine value sequence according to the filter step size parameter and the sine function; Generate a first weight sequence based on the sine value sequence by linear interpolation and the sine function, and invert the first weight sequence to generate a second weight sequence; Combining the first weight sequence and the second weight sequence to obtain a target weight sequence; The first weight sequence Single2 is: ; The second weight sequence Single1 is: Single1=Single2 reverse ; Where Bias is the initial offset, is the filtering step size parameter, is the sine value sequence, k is a positive integer, and reverse represents the inversion process.

7. The image processing method according to any one of claims 1 to 5, characterized in that: The performing corner correction on the captured image to obtain a standard image comprises: Determine a set of boundary point coordinates of a display screen area in the captured image, and perform straight line fitting on the set of boundary point coordinates to obtain a plurality of line segments enclosing the display screen area; Determine a set of actual corner points of the display screen area according to the intersection points of the plurality of line segments, and intercept an initial image surrounded by the set of actual corner points; Calculate a theoretical corner point set according to the display screen to be detected and the camera internal parameters; A projection transformation matrix is ​​determined according to the actual corner point set and the theoretical corner point set, and the captured image is corrected using the projection transformation matrix to obtain a standard image.

8. An image processing system for multi-directional edge texture filtering, characterized in that: The image processing system comprises: An acquisition unit, used for acquiring a captured image of the display screen to be detected, and performing corner correction on the captured image to obtain a standard image; A generating unit, for generating a target weight sequence based on a filter step size parameter and a sine function, and selectively inverting the target weight sequence according to an input polarity parameter, wherein the polarity parameter is used to control an edge detection direction, wherein the edge detection direction includes positive polarity edge detection, negative polarity edge detection, and bipolarity edge detection; A construction unit, used to construct a target convolution kernel according to an input direction parameter, the polarity parameter and the target weight sequence, wherein the direction parameter is used to control a filtering direction, and the filtering direction includes column-direction filtering, row-direction filtering and multi-direction filtering; A convolution unit, configured to perform a convolution operation on the standard image based on the target convolution kernel to obtain a filtered output image; An integration unit is used to integrate the output image according to the polarity parameter and the direction parameter to obtain a target image.

9. An image processing device for multi-directional edge texture filtering, characterized in that: The image processing device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the method according to any one of claims 1 to 7 is performed.

Images