Image brightness self-adaptive adjusting method for template matching and target recognition method

Abstract

This application discloses an image brightness adaptive adjustment method and a target recognition method for template matching. The adjustment method includes: determining multiple image brightness values; matching multiple source images with different image brightness values ​​with a template to obtain multiple cross-correlation coefficients for evaluating the matching degree; fitting a relationship curve between image brightness values ​​and cross-correlation coefficients; determining the optimal image brightness value corresponding to the maximum value of the cross-correlation coefficient in the relationship curve; and adjusting the brightness of the image to be processed to the optimal image brightness value. After adaptively adjusting the image brightness, this application enables the source image to perform template matching with optimal brightness, thereby ensuring the success rate of template matching and improving the robustness of the template matching algorithm to environmental changes.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to an image brightness adaptive adjustment method and a target recognition method for template matching, which can be applied to vision-guided industrial robots for grasping and handling. Background Technology

[0002] Template matching is a fundamental and commonly used pattern recognition method in computer vision. It addresses the problem of whether a given reference image appears in a source image, and if so, where it appears. It is widely applied to object recognition and localization. Here, the reference image is called the template, and the image to be matched is called the source image. The template matching process involves performing a two-dimensional convolution operation between the template and the source image, calculating their similarity, and finally identifying the position where the similarity index is highest as the optimal matching position. Currently, there are two main types of template matching algorithms, based on the chosen image index: grayscale-based template matching algorithms and feature-based template matching algorithms. In grayscale-based template matching algorithms, reference and comparison sequences are generated based on the grayscale values ​​of the template and source images. The cross-correlation (CC) coefficient is used as an evaluation index for the similarity between the template and the source image, calculating the correlation between these two sequences. In feature-based template matching algorithms, object edges in the image are typically selected as features. First, an edge extraction algorithm is used to extract the edges of objects in both the template and source images. Then, a normalized cross-correlation coefficient is used to evaluate their similarity.

[0003] Template matching algorithms are widely used in practice due to their simplicity and high efficiency. A typical application example is vision-guided robotic handling systems, where a robot is equipped with a vision system consisting of an industrial camera, lens, and light source to visually locate the workpiece to be grasped. The visual positioning algorithm typically employs template matching. First, an image of a standard workpiece is acquired at the grasping position and used as a template. Then, this image is compared with the source image of the workpiece to be grasped to obtain the workpiece's position and angle information in the robot's coordinate system. Finally, this position and angle information is provided to the robot for the grasping operation.

[0004] Although template matching algorithms have been integrated as standard operators into various vision processing software, such as OpenCV, Halcon, VisionPro, and VisionMaster, and have been successfully applied to workpiece recognition and localization in practice, these standard template matching algorithms are susceptible to interference factors during industrial field applications, leading to template matching failures. The most significant factors are drastic changes in ambient lighting and the inconsistency of the source image with the workpiece to be matched, resulting in overexposure or underexposure of the source image. This causes significant differences in grayscale or edge features between the source image and the template, reducing the cross-correlation coefficient of the template matching and, in severe cases, causing template matching failure.

[0005] To address the challenge of template matching under drastic changes in ambient lighting and workpiece features, current methods involve machine vision engineers using trial and error to determine the source image brightness that yields the best template match, based on the environment and part surface quality. This approach is not only inefficient, requiring extensive manual adjustments for different conditions, but also lacks robustness; further changes in ambient lighting and workpiece features can cause the template matching algorithm to fail. Therefore, a method is needed that can adaptively adjust the source image brightness based on the current environment and workpiece features, thereby further improving the applicability of standard template matching algorithms in practical applications. Summary of the Invention

[0006] This application provides an image brightness adaptive adjustment method and a target recognition method for template matching, which solves the problems of poor algorithm robustness and low efficiency in adjusting the brightness of the image to be matched by manual trial and error in existing template matching methods.

[0007] On one hand, embodiments of this application provide an image brightness adaptive adjustment method for template matching, including:

[0008] Determine multiple image brightness values;

[0009] Multiple source images with different image brightness values ​​are matched with the template to obtain multiple cross-correlation coefficients used to evaluate the degree of matching;

[0010] The relationship curve between image brightness values ​​and cross-correlation coefficients was obtained by fitting the curve.

[0011] Determine the optimal image brightness value corresponding to the maximum cross-correlation coefficient in the relationship curve;

[0012] Adjust the brightness of the image to be processed to the optimal image brightness value.

[0013] On the other hand, embodiments of this application also provide a target recognition method, including:

[0014] Obtain the image to be processed;

[0015] Set the brightness of the image to be processed to the optimal image brightness value using the method described above;

[0016] The template matching method is used to perform target recognition on the image to be processed after brightness adjustment.

[0017] The image brightness adaptive adjustment method and target recognition method for template matching in this application have the following advantages:

[0018] 1. Using the cross-correlation coefficient of template matching as the evaluation index, the brightness of the source image to be matched is adaptively adjusted, which solves the problem that existing template matching algorithms need to manually adjust the brightness of the source image to ensure the effectiveness of the template matching algorithm when the ambient lighting and workpiece surface quality change. This improves the robustness and automation of the template matching algorithm.

[0019] 2. The computational complexity of the image brightness adaptive adjustment method proposed in this application is: Therefore, the algorithm can obtain the optimal image brightness value after a finite number of iterations. Furthermore, the template matching cross-correlation coefficient threshold and the maximum number of iterations can be set as termination conditions based on actual conditions, allowing the algorithm to stop after a finite number of runs and ensuring its real-time performance.

[0020] 3. By integrating the image brightness adaptive adjustment method proposed in this application with the standard template matching algorithm in existing visual processing software, the adaptability of the original template matching algorithm to environmental changes in practical use is improved. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart of an image brightness adaptive adjustment method for template matching provided in this application embodiment;

[0023] Figure 2 A graph showing the relationship between image brightness values ​​and template matching cross-correlation coefficients provided in the embodiments of this application;

[0024] Figure 3 This is a schematic diagram of the convex function ternary search process provided in an embodiment of this application;

[0025] Figure 4 A flowchart illustrating the three-part method provided in this application embodiment. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] In existing technologies, to eliminate the influence of illumination changes, Jia K. uses the Normalized Cross Correlation (NCC) coefficient as a measure of similarity between the source image and the template. During NCC calculation, the gray value of each pixel in both the template and the source image is normalized by subtracting the product of the average gray value of the entire image and its standard deviation. Therefore, this method is robust to linear changes in illumination conditions. In CN110472674A, the edges of the objects to be matched in the template and source image are first extracted using an edge detection operator, and then the matching similarity is calculated. This method is robust to nonlinear changes in illumination conditions because the gradient vector in the image is also normalized during object edge detection. However, when the above two methods are used under drastic changes in illumination conditions, the gray-scale features of the source image may differ too much from the template due to underexposure or overexposure, or the edge gradient changes of objects in the source image may be too small, making the edges unrecognizable and causing template matching failure. CN112396563A proposes an image enhancement algorithm based on adaptive gain modulation of local brightness, aiming to solve the halo effect in images and shift the brightness of different parts of the image towards the human eye's sensitive range, thereby improving objective contrast and visual quality. Since template matching methods focus on the consistency of features between the source image and the template, directly applying the above method to template matching algorithms yields poor results. CN104036474A proposes an automatic adjustment method for image brightness and contrast, also aiming to improve overall image quality. However, this method requires training a neural network suitable for local image brightness adjustment using a large number of image samples before it can be used, lacking the real-time performance required for direct integration with template matching methods. Therefore, existing methods are not image brightness adjustment algorithms specifically designed for template matching and cannot be directly applied to solve the problem of template matching failure caused by interference factors in practical applications.

[0028] To address the problems in the prior art, this application proposes a method for adaptively adjusting the brightness of the source image based on the ambient lighting and the surface quality of the workpiece. This method enables the source image to perform template matching at the optimal brightness, thereby ensuring the success rate of template matching and improving the robustness of the template matching algorithm to environmental changes.

[0029] Figure 1 A flowchart illustrating an image brightness adaptive adjustment method for template matching, provided in an embodiment of this application. This application provides an image brightness adaptive adjustment method for template matching, comprising:

[0030] S100 determines multiple image brightness values.

[0031] For example, suppose the preset image brightness value range is The image brightness value range can be divided into multiple intervals using equal or unequal intervals, with the endpoint values ​​of each interval serving as multiple image brightness values. The image brightness values ​​obtained using equal interval division are represented as follows:

[0032]

[0033] in, Indicates the first i Each image brightness value, n This represents the number of image brightness values. After determining the image brightness values, the brightness of each source image can be set to the corresponding image brightness value, resulting in multiple source images with different image brightness values.

[0034] S110, Match multiple source images with different image brightness values ​​with the template to obtain multiple cross-correlation coefficients used to evaluate the degree of matching.

[0035] For example, the template is a pre-acquired image. After matching the template and the source image using a template matching method, a cross-correlation number representing the similarity between each source image and the template can be obtained. In this application, ... express.

[0036] S120, the relationship curve between image brightness value and cross-correlation coefficient is obtained by fitting.

[0037] For example, after template matching between the source image and the template is completed, a cross-correlation coefficient is generated for each source image. Since the image brightness value of each source image is different, there is a one-to-one correspondence between the image brightness value and the cross-correlation coefficient. By using the image brightness value as the horizontal axis and the cross-correlation coefficient as the vertical axis, a Cartesian coordinate system with multiple points can be established. The points in this Cartesian coordinate system represent the correspondence between each image brightness value and the cross-correlation coefficient.

[0038] Based on fundamental knowledge of digital image processing and practical results of template matching, it is known that source images with insufficient brightness or overexposure will lose grayscale and edge features, resulting in a low degree of template matching. Therefore, there exists an optimal image brightness value within a finite range of image brightness values ​​that maximizes the matching degree between the source image and the template. Based on this, it can be inferred that the relationship between image brightness values ​​and cross-correlation coefficients can be described by a single-peaked function. To obtain this function, the least squares method can be used to fit the dataset of image brightness values ​​and cross-correlation coefficients. Plotting the data pairs on a two-dimensional plane reveals that the relationship between the two is approximately a power function, expressed as:

[0039]

[0040] like Figure 2 As shown. Taking the logarithm of both sides of the above equation, we get:

[0041]

[0042] For the collected For each pair of data, use the least squares method to establish the variance error expression:

[0043]

[0044] By differentiation, we obtain:

[0045]

[0046]

[0047] After simplification, we obtain the following system of equations:

[0048]

[0049] Solving the above system of equations yields the curve parameters. and The optimal estimate:

[0050]

[0051]

[0052] The final functional expression for the image brightness value and the cross-correlation coefficient is obtained as follows:

[0053]

[0054] It should be noted that the function expression obtained in this step is unchanged for the same type of template. Therefore, when performing template matching for the same type of object, this step only needs to be executed offline once.

[0055] S130, determine the optimal image brightness value corresponding to the maximum cross-correlation coefficient in the relationship curve.

[0056] For example, during execution, S130 also determines the corresponding image brightness search range based on a predetermined cross-correlation coefficient threshold, and searches for the optimal image brightness value within the image brightness search range.

[0057] Specifically, the threshold for the cross-correlation coefficient is: When the cross-correlation coefficient is less than this value, the corresponding image brightness value is not searched. Therefore, the corresponding brightness value search interval can be represented as:

[0058]

[0059] Intuitively, when the brightness value of the source image is too low or too high, the template matching degree will inevitably be low. Therefore, there is no need to search for the brightness at this time, which reduces the number of iterations and computational load of the optimization algorithm and improves the running efficiency of the algorithm.

[0060] Furthermore, a three-part method can be used to search for the optimal image brightness value within the image brightness search range. Specifically, the three-part method is used to search for the brightness value within the search range. The algorithm iterates within the interval and updates the endpoints of that interval to find the optimal value of the unimodal function. For example... Figure 3 As shown in the figure This represents the image brightness value corresponding to the maximum cross-correlation coefficient, i.e., the optimal image brightness value to be determined. The endpoints of the brightness value search interval are respectively denoted as... , First, calculate the interval. The two third points, left and right, are respectively represented as , Then, calculate the function values ​​corresponding to the two trisection points. and And by comparing the magnitudes of these two values, the next search interval is determined. If ,according to Figure 3 It can be seen To the left of the extreme point, the left endpoint of the brightness value search range is set. Updated to three-pointers Conversely, it can be known that... Since it's to the right of the extreme point, the right endpoint of the brightness value search interval is updated to a third point. The above calculation process is iterated, updating and narrowing the search range for brightness values ​​step by step. The range.

[0061] In the embodiments of this application, to ensure the optimization process ends after a limited number of calculations, a threshold for the interval length or a maximum number of iterations can be set. When the length of the updated image brightness search interval is less than the set interval length threshold, or the number of update iterations reaches the set maximum number of iterations, the optimization process ends, and the optimal image brightness value is output. The three-part method solution process is as follows: Figure 4 As shown. Based on computational complexity theory, it has been proven that the computational complexity of the ternary search is the same as that of the binary search, both being... Therefore, the algorithm's finite convergence is theoretically guaranteed.

[0062] S140, adjust the brightness of the image to be processed to the optimal image brightness value.

[0063] For example, the image to be processed can be acquired by various vision systems. For instance, in this embodiment of the application, a vision system including an industrial camera, lens and light source can be set on the handling robot to acquire the image to be processed.

[0064] This application also provides a target recognition method, including:

[0065] Obtain the image to be processed;

[0066] Set the brightness of the image to be processed to the optimal image brightness value using the method described above;

[0067] The template matching method is used to perform target recognition on the image to be processed after brightness adjustment.

[0068] For example, since the image to be processed has been set to the optimal image brightness value, the template matching method will obtain the highest cross-correlation coefficient when identifying targets in the image to be processed, thereby effectively improving the accuracy of target recognition.

[0069] The image brightness adaptive bar adjustment method for template matching proposed in this application has been applied in a vision-guided robotic workpiece handling system. Since the loading station where the workpiece is placed lacks a precise tooling positioning mechanism, different positional deviations occur each time the workpiece is placed. Therefore, a vision system is needed to acquire workpiece position information to guide the robot in handling it. The vision system hardware includes an industrial camera, lens, and light source. The vision processing software uses Hikvision's VisionMaster, employing its high-precision template matching operator as the workpiece positioning algorithm. The standard template matching operator execution flow is as follows: First, an image of the workpiece placed at the loading station is acquired to extract the template. A template matching algorithm based on edge features is selected to extract representative workpiece contours as features. Then, the source image of the workpiece is acquired and matched with the template to obtain the position and angular deviation of the workpiece relative to the template. Finally, the workpiece position and angular deviation given by the template matching are sent to the robot to achieve precise workpiece handling.

[0070] However, due to the unstable lighting conditions at the site, the brightness of the source image acquired by the vision system varies greatly depending on whether it is morning / evening or sunny / cloudy. Furthermore, the tool path for different textures on the workpiece surface due to different processing batches can also alter the template matching features. These two main reasons prevent the template matching algorithm from providing correct results, and the robot cannot obtain the workpiece's position information. To address these issues, the image brightness adaptive adjustment method proposed in this application is integrated with the high-precision template matching algorithm in the VisionMaster vision processing software. The specific execution process of the algorithm includes:

[0071] (1) Collect workpiece templates for one type of workpiece;

[0072] (2) Acquire the source image of the workpiece to be matched, and set the image brightness value to within... Ten values ​​are taken at intervals of 15 and template matching is performed on each value to obtain the corresponding cross-correlation count.

[0073] (3) The least squares method is used to calculate the functional expression between the image brightness value and the cross-correlation coefficient, resulting in the power function expression: The expression is the same for the same type of workpiece, so the first three steps in the execution process only need to be executed once during the algorithm initialization phase. After obtaining the expression, it does not need to be executed before each template matching.

[0074] (4) Set the threshold for cross-correlation coefficient of template matching The image brightness search interval is calculated as follows: The algorithm iterates within this interval using a ternary trigonometric method, and sets the maximum number of iterations. Interval length threshold The algorithm finds the optimal image brightness value through iteration. By comparing the fitted image brightness values ​​with the cross-correlation coefficients, it can be seen that the optimal image brightness values ​​obtained through analytical calculation are the same as those obtained through iterative optimization using the ternary method, thus verifying the correctness of the model and algorithm.

[0075] (5) The source image was adjusted using the optimal image brightness value, and then a high-precision template matching operator was executed, resulting in a cross-correlation coefficient of 0.977, indicating a high degree of matching between the source image and the template. The output of this operator is the two-dimensional coordinates of the workpiece center. and the rotation angle of the workpiece relative to the template image. This result shows the position and orientation of the workpiece being handled by the robot.

[0076] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0077] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for adaptive image brightness adjustment for template matching, characterized in that, include: Determine multiple image brightness values: Divide a preset range of image brightness values ​​into multiple intervals, and use the endpoint values ​​of each interval as multiple image brightness values; Multiple source images with different image brightness values ​​are matched with the template to obtain multiple cross-correlation coefficients used to evaluate the degree of matching; The relationship curve between the image brightness value and the cross-correlation coefficient was obtained by fitting the least squares method. The optimal image brightness value corresponding to the maximum cross-correlation coefficient in the relationship curve is determined. The corresponding image brightness search interval is determined according to the predetermined cross-correlation coefficient threshold, and the optimal image brightness value is searched in the image brightness search interval using the three-part method. The brightness of the image to be processed is adjusted to the optimal image brightness value.

2. The image brightness adaptive adjustment method for template matching according to claim 1, characterized in that, When updating the image brightness search interval using the three-part method, the iteration ends when the length of the updated image brightness search interval is less than the set interval length threshold.

3. The image brightness adaptive adjustment method for template matching according to claim 1, characterized in that, When updating and iterating the image brightness search range using the three-part method, the iteration ends when the number of updates and iterations reaches the set maximum number of iteration steps.

4. A target recognition method, characterized in that, include: Obtain the image to be processed; The brightness of the image to be processed is set to the optimal image brightness value according to the method described in any one of claims 1-3; The template matching method is used to perform target recognition on the image to be processed after brightness adjustment.

Images