Optical image detection threshold adaptive adjustment method and system based on reinforcement learning

Abstract

The present application relates to a method and system for adaptive adjustment of optical image detection threshold based on reinforcement learning, belonging to the field of machine vision and intelligent detection technology, which solves the technical problems of existing industrial vision detection, such as threshold dependence on manual setting, lack of adaptability leading to decline in detection accuracy, and inability to optimize in real time. The solution is: a method for adaptive adjustment of optical image detection threshold based on reinforcement learning, including the following steps: S1, collecting the original image of the measured object; S2, preprocessing the original image; S3, constructing a state vector based on multi-dimensional feature quantities; S4, outputting actions by a reinforcement learning strategy network; S5, updating the detection threshold based on the actions and applying it; S6, calculating the detection quality evaluation index and the reward signal; S7, modeling and training the reinforcement learning strategy network and updating the parameters to form a threshold adaptive adjustment strategy. Compared with the prior art, the present application has the advantages of not depending on manual setting, strong adaptability, high precision, real-time optimization, and wide applicability.

Description

Technical Field

[0001] This invention belongs to the field of machine vision and intelligent detection technology, specifically relating to an adaptive adjustment method and system for optical image detection threshold based on reinforcement learning. Background Technology

[0002] With the continuous improvement of automation and informatization in industrial manufacturing processes, machine vision-based inspection technology has been widely applied in various production scenarios such as metal processing, electronic packaging, textile materials, semiconductor manufacturing, and food packaging. Industrial vision inspection typically involves acquiring images of workpiece surfaces using cameras, followed by preprocessing, segmentation, feature extraction, and classification to achieve tasks such as surface defect identification, dimensional inspection, assembly quality assessment, or structural contour positioning. To ensure the accuracy and stability of the inspection results, vision systems often rely on a large number of manually set parameters, including brightness thresholds, texture contrast thresholds, edge response thresholds, binarization thresholds, filter kernel scales, and segmentation region thresholds. These thresholds and parameters are highly sensitive to changes in ambient light, material reflectivity, texture complexity, and equipment position, requiring frequent adjustments during actual production; otherwise, problems such as false detections, missed detections, positioning misalignments, and blurred boundaries can easily occur.

[0003] Current industrial vision systems generally employ fixed thresholds or parameter adjustment methods based on heuristic rules. Fixed thresholds rely on a set of "global thresholds" calibrated manually and used long-term. When environmental changes occur, such as variations in lighting, lens focal length adjustments, batch differences in surface materials, or dust particle obstruction, these static thresholds cannot reliably adapt to new conditions, leading to a significant decrease in detection accuracy. Another type of method uses adaptive thresholding algorithms based on statistical features or gradient changes, such as Otsu thresholding, local adaptive thresholding, or histogram-based dynamic thresholding. These methods typically rely on fixed mathematical models and cannot be optimized in real-time according to the complexity of specific production lines. For example, in scenarios involving highly reflective metals, low-contrast surfaces, or small-sized defect detection, traditional adaptive algorithms are prone to excessive threshold shifts or unstable oscillations, resulting in discontinuous detection results or error accumulation. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies and solve the technical problems in current industrial visual inspection, such as threshold dependence on manual setting, lack of adaptability leading to decreased detection accuracy, and inability to optimize in real time, this invention provides an adaptive adjustment method and system for optical image detection threshold based on reinforcement learning.

[0005] The present invention is achieved through the following technical solutions.

[0006] This invention provides a reinforcement learning-based adaptive threshold adjustment method for optical image detection, comprising the following steps:

[0007] S1. Acquire the original image of the object being tested;

[0008] S2. Perform grayscale normalization, noise reduction, and contrast enhancement on the original image to obtain a preprocessed image;

[0009] S3. Extract multidimensional features from the preprocessed image, and normalize and vectorize the multidimensional features to construct the state vector at the current moment;

[0010] S4. The reinforcement learning policy network receives the state vector from step S3 and outputs the action.

[0011] S5. Update the detection threshold based on the action, and apply the updated threshold parameter to the preprocessed image to obtain the detection result image;

[0012] S6. Calculate the detection quality evaluation index and reward signal based on the detection result image;

[0013] S7. Reinforcement learning strategy network modeling, training, and parameter updating; after continuous parameter updates, a threshold adaptive adjustment strategy is formed.

[0014] Furthermore, the multidimensional feature quantities in step S3 include the global grayscale histogram. Local regional variance Gradient magnitude distribution Edge pixel density Saturation ratio of high reflectivity areas .

[0015] Furthermore, in step S5:

[0016] based on Obtain the updated threshold parameters t is time t. For the detection vector, , , For action, , For a pre-designed set of discrete or finite continuous actions, , The global binarization threshold is used. For locally adaptive threshold window size, For the edge detection operator threshold, This represents the sharpening intensity factor;

[0017] Updated threshold parameters Apply to preprocessed images Image segmentation, edge detection, and defect identification are performed to obtain the detection result image. t represents the detection time or the t-th frame of the image. These are the pixel coordinates in the image.

[0018] Furthermore, in step S6:

[0019] pass Calculate the quality evaluation index of testing , , , These are the weighting coefficients. Used to measure the accuracy and completeness of target region segmentation. Used to measure the continuity of edge detection and positioning accuracy. Used to measure the degree of false detection of noise or false edges;

[0020] pass Calculate reward signal , To smooth out the penalty coefficient.

[0021] Furthermore, in step S7:

[0022] definition For state Next action The estimated long-term cumulative return obtained satisfies , Let be the state vector at time t. m is the state dimension. , As a discount factor, For all candidate actions in the next state;

[0023] Define the timing difference error as ;

[0024] Will have parameter vector function Update based on the aforementioned timing difference error, satisfying... , For learning rate, For the function's parameter vector The gradient is used to form a threshold adaptive adjustment strategy after continuous parameter updates.

[0025] Furthermore, the system includes:

[0026] An optical imaging module is used to acquire the original image of the object under test;

[0027] An image preprocessing module is used to perform grayscale normalization, noise reduction, and contrast enhancement on the original image.

[0028] A state vector construction module is used to extract multidimensional features and construct a state vector.

[0029] An output action module, which is used to cause the reinforcement learning policy network to output an action;

[0030] A threshold execution module, which is used to update threshold parameters;

[0031] A detection and evaluation module, which generates a reward signal and judges the detection effect;

[0032] A reinforcement learning policy network modeling, training, and parameter update module is used to form a threshold adaptive adjustment strategy after continuous parameter updates.

[0033] Furthermore, in the threshold adaptive adjustment strategy, the system receives a positive reward when the threshold selection causes the detection result to approach the true boundary or separates the defect region; and a penalty is imposed when the threshold selection causes edge breakage, over-segmentation, or missed detection. With continuous reward feedback, the reinforcement learning strategy network learns the optimal mapping between the threshold and image features under different optical conditions, forming the threshold adaptive adjustment strategy.

[0034] Furthermore, the reinforcement learning policy network includes a state encoding layer, a feature fusion layer, and an action decision layer. The state encoding layer is used to extract and compress multi-dimensional features. The feature fusion layer is used to integrate image state information from different dimensions. The action decision layer is used to output an estimate of the threshold-adjusted action or the action probability distribution.

[0035] Furthermore, the threshold execution module includes a threshold caching unit, an edge operator parameter update unit, and a preprocessing parameter synchronization unit. The threshold caching unit is used to store and smooth the threshold adjustment results and provide stable threshold parameters between adjacent detection cycles. The edge operator parameter update unit is used to update the response threshold or operator parameters of the edge detection operator according to the threshold adjustment results, thereby adjusting the sensitivity and positioning accuracy of edge detection. The preprocessing parameter synchronization unit is used to synchronize the updated threshold parameters with the parameters in the image preprocessing process, so that the image preprocessing, threshold segmentation, and edge detection processes are consistent.

[0036] The beneficial effects achieved by this invention are as follows: First, this invention proposes an adaptive threshold adjustment method for optical image detection based on reinforcement learning. Through reinforcement learning policy network modeling, training, and parameter updating, the threshold parameter is dynamically adjusted in real time, eliminating the need for manual setting and maintaining excellent detection performance under complex optical conditions. Second, this invention constructs a complete closed loop from output action, threshold execution, detection evaluation to the adaptive threshold adjustment strategy, realizing adaptive adjustment of the optical image detection threshold. This allows the system to automatically select approximately optimal threshold parameters under different lighting conditions, material properties, and noise environments, significantly improving the accuracy and robustness of optical detection. Third, the reinforcement learning policy network modeling, training, and parameter updating do not require a complex retraining process, enabling continuous optimization through real-time feedback in actual production environments, improving system deployment efficiency. Furthermore, this invention is widely applicable to various optical detection tasks such as metal surface defect detection, electronic component solder joint quality assessment, glass or ceramic crack identification, and mechanical structure edge measurement, demonstrating good versatility and promotional value.

[0037] Compared with existing technologies, this invention has the advantages of not relying on manual settings, strong adaptability, high precision, real-time optimization, and wide versatility. Attached Figure Description

[0038] Figure 1 This is a structural block diagram of the system of the present invention;

[0039] Figure 2 This is a simulated optical image constructed in an embodiment of the present invention;

[0040] Figure 3 This is the image after adding noise interference to the ideal optical image in the embodiments of the present invention;

[0041] Figure 4 This is the result of segmenting a noisy optical image using the optimal threshold obtained through a reinforcement learning policy network in this embodiment of the invention;

[0042] Figure 5 This is the cumulative reward change curve obtained by the reinforcement learning policy network in several training iterations in an embodiment of the present invention. Detailed Implementation

[0043] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0044] Example 1: As Figures 1 to 5 As shown, the reinforcement learning-based adaptive threshold adjustment method for optical image detection includes the following steps:

[0045] S1. Acquire the original image of the object being tested;

[0046] Specifically, the object under test is imaged using an optical imaging module to obtain the original image, denoted as... t represents the detection time or the t-th frame of the image. These are the pixel coordinates in the image.

[0047] S2. Perform grayscale normalization, noise reduction, and contrast enhancement on the original image to obtain a preprocessed image;

[0048] Specifically, the original image is processed by an image preprocessing module to perform grayscale normalization, noise reduction, and contrast enhancement to obtain a preprocessed image, denoted as . .

[0049] S3. Extract multidimensional features from the preprocessed image, and normalize and vectorize the multidimensional features to construct the state vector at the current moment;

[0050] Specifically, multidimensional features are extracted from the preprocessed image, including the global gray-level histogram. Local regional variance Gradient magnitude distribution Edge pixel density Saturation ratio of high reflectivity areas The current state vector is constructed by normalizing and vectorizing the features. m is the state dimension; the state vector expresses factors that have a significant impact on the optical detection results, such as lighting conditions, texture complexity, edge sharpness and noise level. When constructing the state vector, features that have a significant impact on the detection results but have low computational cost should be selected to construct the state vector, so as to avoid too many complex features that will increase the delay.

[0051] S4. The reinforcement learning policy network receives the state vector from step S3 and outputs the action.

[0052] The action is output through a reinforcement learning strategy network. , , For a pre-designed set of discrete or finite continuous actions, the action can be represented as an adjustment step size for each parameter. , The global binarization threshold is used. For locally adaptive threshold window size, For the edge detection operator threshold, To sharpen the intensity coefficients, the reinforcement learning policy network employs a lightweight network structure, which improves the stability of detection. Upon receiving the state vector, the reinforcement learning policy network... Then, the probability distribution or value assessment of each candidate action is output, thereby establishing a mathematical model. The mapping relationship.

[0053] S5. Update the detection threshold based on the action, and apply the updated threshold parameter to the preprocessed image to obtain the detection result image;

[0054] Specifically, based on Obtain the updated threshold parameters t is time t. For the detection vector, , ;

[0055] Updated threshold parameters Apply to preprocessed images Image segmentation, edge detection, and defect identification are performed to obtain the detection result image. t represents the detection time or the t-th frame of the image. These are the pixel coordinates in the image.

[0056] S6. Calculate the detection quality evaluation index and reward signal based on the detection result image;

[0057] Specifically, through Calculate the quality evaluation index of testing Testing quality evaluation indicators It can be based on a combination of criteria such as edge continuity, defect region integrity, and noise missegmentation. , , These are the weighting coefficients. Used to measure the accuracy and completeness of target region segmentation. Used to measure the continuity of edge detection and positioning accuracy. Used to measure the degree of false detection of noise or false edges;

[0058] pass Calculate reward signal , To smooth out the penalty coefficient, the main purpose is to avoid drastic fluctuations in the threshold over time, ensuring that the system's detection results remain sufficiently stable even when the strategy has not yet fully converged, thus guaranteeing the stability of the industrial detection process.

[0059] S7. Reinforcement learning strategy network modeling, training, and parameter updating; after continuous parameter updates, a threshold adaptive adjustment strategy is formed.

[0060] definition For state Next action The estimated long-term cumulative return obtained satisfies , Let be the state vector at time t. m is the state dimension. , The discount factor is set to 0.9 in this embodiment. For all candidate actions in the next state;

[0061] Define the timing difference error as ;

[0062] Will have parameter vector function Update based on the aforementioned timing difference error, satisfying... , For learning rate, For the function's parameter vector The gradient is used to indicate the direction of parameter updates, and a threshold adaptive adjustment strategy is formed after continuous parameter updates. In this embodiment... We set the value to 0.001 to balance convergence speed and training stability. State vector. ,action Reward signals With the next state vector The system continuously accumulates samples and iteratively optimizes the strategy by constructing complete empirical sample triplets or quadruplets. This allows the threshold adjustment behavior to gradually converge from initial tentative adjustments to a stable and effective adaptive strategy. Through continuous parameter updates, the reinforcement learning strategy network learns how to adjust the threshold under different optical image conditions, thereby maximizing long-term rewards and achieving maximum optical detection quality.

[0063] This invention proposes a reinforcement learning-based adaptive threshold adjustment method for optical image detection. By modeling and training a reinforcement learning policy network and updating its parameters, the threshold parameters are dynamically adjusted in real time, eliminating the need for manual setting and maintaining excellent detection performance under complex optical conditions. Secondly, this invention establishes a complete closed loop from output action, threshold execution, detection evaluation to the adaptive threshold adjustment strategy, achieving adaptive adjustment of the optical image detection threshold. This allows the system to automatically select near-optimal threshold parameters under different lighting conditions, material properties, and noise environments, significantly improving the accuracy and robustness of optical detection. Thirdly, the reinforcement learning policy network modeling and parameter updating does not require a complex retraining process, enabling continuous optimization through real-time feedback in a real-world production environment, thus improving system deployment efficiency.

[0064] Figure 2 This illustration shows a simulated optical image scene constructed in an embodiment of the present invention. The image consists of a uniform background region and a central target region, with the target region having a slightly higher grayscale value than the background, used to simulate the difference in reflection of the workpiece surface under optical imaging conditions. The image is noise-free and can be used as a reference image under ideal conditions to verify the impact of different threshold adaptive adjustment strategies on the segmentation effect.

[0065] Figure 3 This is an image after adding noise interference to an ideal optical image. The image exhibits random noise and local brightness unevenness, used to simulate interference caused by lighting fluctuations, sensor noise, and surface reflections commonly seen in industrial settings. Figure 3 Used to verify the ability to resist noise and the effectiveness of the method in complex environments.

[0066] Figure 4 This is the result of segmenting a noisy optical image using the optimal threshold obtained through reinforcement learning. The segmentation image effectively distinguishes the target region from the background region, with complete boundaries and a low number of false noise detections. Figure 4 This demonstrates that the threshold adaptive adjustment strategy employed in this invention can automatically search for a threshold suitable for the current image conditions in a noisy environment, thereby achieving high segmentation quality.

[0067] Figure 5 The graph shows the cumulative reward changes of the reinforcement learning policy network of this invention over multiple training rounds. The curves exhibit an overall upward trend, indicating that during continuous interaction, the reinforcement learning policy network can gradually learn more reasonable threshold adjustment actions, resulting in continuous improvement in segmentation quality as training progresses. Meanwhile, the occasional fluctuations in the graph reflect the influence of the exploration mechanism on policy updates, consistent with the typical convergence characteristics of reinforcement learning.

[0068] Example 2: An adaptive adjustment system for optical image detection threshold based on reinforcement learning, comprising:

[0069] An optical imaging module is used to acquire the original image of the object under test;

[0070] An image preprocessing module is used to perform grayscale normalization, noise reduction, and contrast enhancement on the original image.

[0071] A state vector construction module is used to extract multidimensional features and construct a state vector.

[0072] An output action module, which is used to cause the reinforcement learning policy network to output an action;

[0073] A threshold execution module, which is used to update threshold parameters;

[0074] A detection and evaluation module, which generates a reward signal and judges the detection effect;

[0075] The reinforcement learning policy network modeling, training, and parameter update module is used to form a threshold adaptive adjustment strategy after continuous parameter updates. In this threshold adaptive adjustment strategy, the system receives a positive reward when the threshold selection leads to a detection result close to the true boundary or separates defect regions; a penalty is imposed when the threshold selection leads to edge breakage, over-segmentation, or missed detection. Through continuous reward feedback, the reinforcement learning policy network learns the optimal mapping between the threshold and image features under different optical conditions, forming the threshold adaptive adjustment strategy.

[0076] Specifically, the reinforcement learning policy network includes a state encoding layer, a feature fusion layer, and an action decision layer. The state encoding layer is used for feature extraction and compressed representation of multi-dimensional features, and includes three convolutional layers. The first convolutional layer uses 16 convolutional kernels with a kernel size of 3×3 and a stride of 1, and a non-linear activation function is applied after convolution. The second convolutional layer uses 32 convolutional kernels with a kernel size of 3×3 and a stride of 1. The third convolutional layer uses 64 convolutional kernels with a kernel size of 3×3. The feature fusion layer is used to integrate image state information from different dimensions. The feature fusion layer adopts a fully connected neural network structure and includes one or two fully connected layers. The number of hidden units is preferably set to 128 or 256 to control the network size while ensuring expressive power and meeting industrial real-time requirements. The action decision layer is used to output the estimated value or probability distribution of the threshold adjustment action. The output dimension of the action decision layer is equal to the size of the predefined action space, and each output node corresponds to a threshold adjustment action. To ensure network training stability, the action decision layer may not have an activation function or may use a linear output form. During the training phase, the system uses real-time images or historical detection data for interactive learning; during the inference phase, the reinforcement learning policy network receives the current state and immediately outputs action instructions for threshold adjustment, thereby achieving efficient online parameter tuning capabilities. The reinforcement learning policy network adopts a lightweight network structure that meets the real-time requirements of industrial environments, and optimizes the policy inference process, enabling the system to complete threshold update operations in milliseconds without affecting production cycle time.

[0077] The threshold execution module includes a threshold caching unit, an edge operator parameter update unit, and a preprocessing parameter synchronization unit. The threshold caching unit stores and smooths the threshold adjustment results and provides stable threshold parameters between adjacent detection cycles. The edge operator parameter update unit updates the response threshold or operator parameters of the edge detection operator based on the threshold adjustment results, adjusting the sensitivity and positioning accuracy of edge detection. The preprocessing parameter synchronization unit synchronizes the updated threshold parameters with relevant parameters in the image preprocessing process, ensuring consistency between image preprocessing, threshold segmentation, and edge detection.

[0078] The modules of this invention are connected sequentially in a data flow manner, achieving real-time interaction through network communication or memory sharing. The reinforcement learning strategy network modeling training and parameter update module incorporates a threshold adaptive adjustment strategy, enabling the system to automatically select the optimal segmentation threshold or edge detection parameters based on real-time image features, thereby improving the stability, accuracy, and environmental adaptability of optical detection.

[0079] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, modifications can still be made to the embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A reinforcement learning-based adaptive threshold adjustment method for optical image detection, characterized in that: Includes the following steps: S1. Acquire the original image of the object being tested; S2. Perform grayscale normalization, noise reduction, and contrast enhancement on the original image to obtain a preprocessed image; S3. Extract multidimensional features from the preprocessed image, and normalize and vectorize the multidimensional features to construct the state vector at the current moment; Multidimensional features include the global grayscale histogram. Local regional variance Gradient magnitude distribution Edge pixel density Saturation ratio of high reflectivity areas ; S4. The reinforcement learning policy network receives the state vector from step S3 and outputs the action. S5. Update the detection threshold based on the action, and apply the updated threshold parameter to the preprocessed image to obtain the detection result image; S6. Calculate the detection quality evaluation index and reward signal based on the detection result image; pass Calculate the quality evaluation index of testing , , , These are the weighting coefficients. Used to measure the accuracy and completeness of target region segmentation. Used to measure the continuity of edge detection and positioning accuracy. Used to measure the degree of false detection of noise or false edges; pass Calculate reward signal , To smooth out the penalty coefficient, For the updated threshold parameter Let be the detection vector, and t be the time t. S7. Reinforcement learning strategy network modeling, training, and parameter updating; after continuous parameter updates, a threshold adaptive adjustment strategy is formed.

2. The adaptive threshold adjustment method for optical image detection based on reinforcement learning according to claim 1, characterized in that: In step S5: based on Obtain the updated threshold parameters t is time t. For the detection vector, , , For action, , For a pre-designed set of discrete or finite continuous actions, , The global binarization threshold is used. For locally adaptive threshold window size, For the edge detection operator threshold, This represents the sharpening intensity factor; Updated threshold parameters Apply to preprocessed images Image segmentation, edge detection, and defect identification are performed to obtain the detection result image. t represents the detection time or the t-th frame of the image. These are the pixel coordinates in the image.

3. The adaptive adjustment method for optical image detection threshold based on reinforcement learning according to claim 2, characterized in that: In step S7: definition For state Next action The estimated long-term cumulative return obtained satisfies , Let be the state vector at time t. m is the state dimension. , As a discount factor, For all candidate actions in the next state; Define the timing difference error as ; Will have parameter vector function Update based on the aforementioned timing difference error, satisfying... , For learning rate, For the function's parameter vector The gradient is used to form a threshold adaptive adjustment strategy after continuous parameter updates.

4. An adaptive threshold adjustment system for optical image detection based on reinforcement learning, characterized in that: This system is used in the reinforcement learning-based adaptive threshold adjustment method for optical image detection according to any one of claims 1-3, the system comprising: An optical imaging module is used to acquire the original image of the object under test; An image preprocessing module is used to perform grayscale normalization, noise reduction, and contrast enhancement on the original image. A state vector construction module is used to extract multidimensional features and construct a state vector. An output action module, which is used to cause the reinforcement learning policy network to output an action; A threshold execution module, which is used to update threshold parameters; A detection and evaluation module, which generates a reward signal and judges the detection effect; A reinforcement learning policy network modeling, training, and parameter update module is used to form a threshold adaptive adjustment strategy after continuous parameter updates.

5. The adaptive adjustment system for optical image detection threshold based on reinforcement learning according to claim 4, characterized in that: In the threshold adaptive adjustment strategy, the system receives a positive reward when the threshold selection causes the detection result to approach the true boundary or separates the defect region; and a penalty is imposed when the threshold selection causes edge breakage, over-segmentation, or missed detection. Through continuous reward feedback, the reinforcement learning strategy network learns the optimal mapping between the threshold and image features under different optical conditions, forming the threshold adaptive adjustment strategy.

6. The reinforcement learning-based adaptive threshold adjustment system for optical image detection according to claim 5, characterized in that: The reinforcement learning policy network includes a state encoding layer, a feature fusion layer, and an action decision layer. The state encoding layer is used to extract and compress multi-dimensional features. The feature fusion layer is used to integrate image state information from different dimensions. The action decision layer is used to output an estimated value or probability distribution of the threshold-adjusted action.

7. The adaptive threshold adjustment system for optical image detection based on reinforcement learning according to claim 4, characterized in that: The threshold execution module includes a threshold caching unit, an edge operator parameter update unit, and a preprocessing parameter synchronization unit. The threshold caching unit is used to store and smooth the threshold adjustment results and provide stable threshold parameters between adjacent detection cycles. The edge operator parameter update unit is used to update the response threshold or operator parameters of the edge detection operator according to the threshold adjustment results, thereby adjusting the sensitivity and positioning accuracy of edge detection. The preprocessing parameter synchronization unit is used to synchronize the updated threshold parameters with the parameters in the image preprocessing process, so that the image preprocessing, threshold segmentation and edge detection processes are consistent.

Images