Lens visual field regulation and control method and system based on target recognition

Abstract

The invention relates to the technical field of position or direction control achieved through an optical means, in particular to a lens view regulation and control method and system based on target recognition. The method comprises the following steps: acquiring an image sequence and a reflection distribution diagram sequence for target imaging; extracting first feature data based on the image sequence, wherein the first feature data represents a change trend of a target depth position; extracting second feature data based on the reflection distribution diagram sequence, wherein the second feature data represents fluctuation characteristics of reflection light distribution; dynamically adjusting the parameter structure of the controller according to the combination state of the first feature data and the second feature data, so that the controller generates differentiated adjustment responses to different combination states; and generating an adjustment instruction based on the adjusted parameter structure, wherein the adjustment instruction is used for driving the focusable optical lens to change the focal length so as to realize lens visual field adjustment and control. According to the invention, the detection imaging quality in a flow production line can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of position or direction control realized by optical means, and in particular to a lens field of view regulation method and system based on target recognition. BACKGROUND

[0002] In the field of industrial automation production, visual inspection systems are widely used in product quality inspection, dimension measurement and defect identification, etc. A typical industrial visual inspection system usually includes an image acquisition device, a motion control device and an image processing unit. After a workpiece enters the detection field of view through a conveying mechanism such as a conveyor belt, the system needs to identify the current imaging state and control the lens to adjust to the appropriate position to obtain a clear image. In high-precision detection applications, due to the limited depth of field of the optical lens, a position control system based on optical feedback needs to be established to dynamically regulate the focal plane position according to the changes in the position and surface features of the workpiece, thereby realizing the field of view regulation in the depth direction.

[0003] In the prior art, the identification of the focal plane position is usually realized by extracting the sharpness features of the target image, such as gradient method, frequency domain analysis method and contrast detection method, etc. The sharpness signals output by these methods can be used as the feedback quantity of the control system. Based on the optical feedback signal, the control system drives the focusing mechanism to change the focal length of the lens. In addition, in order to compensate for the effects of workpiece movement and mechanical vibration, the prior art also uses image registration methods to align multiple consecutive images.

[0004] However, the prior art still has some problems, resulting in that the detection imaging quality of the existing optical position control system for regulating the field of view of the lens in the flow production line still needs to be improved. SUMMARY

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a lens field of view regulation method and system based on target recognition, which can improve the detection imaging quality in the flow production line.

[0006] In a first aspect, the present application provides a lens field of view regulation method based on target recognition, which comprises the following steps: obtaining an image sequence and a reflection distribution map sequence for target imaging; extracting first feature data based on the image sequence, the first feature data representing the variation trend of the target depth position; extracting second feature data based on the reflection distribution map sequence, the second feature data representing the fluctuation characteristics of the reflected light distribution; The parameter structure of the controller is dynamically adjusted according to the combined state of the first feature data and the second feature data, so that the controller can produce differentiated adjustment responses to different combined states. An adjustment command is generated based on the adjusted parameter structure. The adjustment command is used to drive the adjustable focus optical lens to change the focal length in order to control the lens field of view.

[0007] Optionally, the process of dynamically adjusting the parameter structure of the controller based on the combined state of the first feature data and the second feature data includes the following steps: Based on the first feature data and the second feature data, the predictability of the target depth position change is determined, and the combined state is identified, wherein the combined state includes at least: In the first combination state, the first feature data does not show a monotonic change trend and the fluctuation intensity of the second feature data is within a preset stable range; The second combination state, wherein the first feature data exhibits a monotonic change trend; The third combination state is characterized in that the first feature data exhibits non-monotonic change characteristics and the fluctuation intensity of the second feature data exceeds the preset stable range, or the change amplitude of the first feature data exceeds the preset deviation range. The first combined state corresponds to invalid fluctuations in the target depth position, requiring suppression of the focus adjustment response; the second combined state corresponds to continuous shifts in the target depth position, requiring enhancement of the focus adjustment response along the shift direction; and the third combined state corresponds to high dynamic changes in the target depth position, requiring omnidirectional enhancement of the focus adjustment response. The parameter structure of the controller is adjusted for different combinations of states so that different focal length adjustment responses are generated for the same depth deviation under different combinations of states.

[0008] Optionally, the process of extracting the second feature data based on the reflectance distribution map sequence includes the following steps: Extract the position sequence of the centroid of reflection from the reflection distribution map sequence; The second feature data is calculated based on the statistical characteristics of the position sequence to characterize the fluctuation intensity of the reflected light distribution; The statistical characteristics include displacement amplitude or trajectory entropy.

[0009] Optionally, the step of extracting the first feature data based on the image sequence includes: Obtain a sequence of target depth locations from the image sequence; Calculate the rate of change of depth position based on the depth position sequence; Determine whether the rate of change exhibits monotonicity, and if it does, determine the direction of the monotonic change; The first feature data is determined based on the monotonicity characteristics and the direction of monotonic change to characterize the changing trend of the target depth position.

[0010] Optionally, the process of adjusting the parameter structure of the controller for different combined states includes the following steps: When the first combined state is detected, the damping coefficient of the controller is increased and the dead zone range is set so that depth deviations smaller than the dead zone range do not trigger the focus adjustment response. When the second combined state is identified, directional control parameters are set according to the direction of monotonic change, wherein the damping coefficient is reduced and the response gain is increased along the direction of monotonic change, and the damping coefficient is increased in the opposite direction. When the third combined state is detected, the damping limit is removed and the maximum response gain is set; The adjustment command is generated based on the adjusted parameter structure, wherein the adjustment command includes an adjustment amount of the scanning center position along the optical axis to achieve clear imaging range control along the optical axis.

[0011] Optionally, the process of obtaining the sequence of target depth locations from the image sequence includes the following steps: Within a preset scanning range, the focal length of the adjustable optical lens is changed to acquire images corresponding to different focal lengths, and the sharpness score of each image is calculated to obtain a sharpness score sequence. Based on the sharpness score sequence and the corresponding focal length sequence, the target depth position is obtained by energy centroid calculation, wherein the energy centroid is determined by weighted summation of sharpness score and focal length; Repeat the above acquisition process over time to obtain a sequence of the target depth positions.

[0012] Optionally, the process of obtaining the target depth position through energy centroid calculation includes the following steps: Obtain an ideal focal length response model, which describes the distribution characteristics of sharpness score as a function of focal length during focal length scanning of a qualified workpiece; Calculate the ideal energy centroid based on the aforementioned ideal focal length response model; The actual energy centroid, calculated by weighted summation based on the sharpness score sequence and the corresponding focal length sequence, is compared with the ideal energy centroid to calculate the energy centroid offset. The target depth position is determined based on the energy center of gravity offset.

[0013] Optionally, the process of executing the adjustment command to drive the adjustable optical lens to change its focal length further includes the following steps: Based on the adjustment rate along the optical axis contained in the adjustment command, the change in angle of view caused by the change in lens focal length is calculated. A correction transformation matrix is ​​constructed based on the aforementioned change in viewing angle. The correction transformation matrix includes cropping compensation for the image sequence to compensate for changes in magnification and perspective caused by dynamic changes in focal length. The correction transformation matrix is ​​applied to the image sequence to generate aligned images; The diffuse reflection component and the specular reflection component are extracted from the aligned image, wherein the diffuse reflection component is used to synthesize the final output detection image, and the specular reflection component generates the reflection distribution map sequence for extracting the second feature data for the next detection.

[0014] Optionally, the process of constructing the correction transformation matrix based on the adjustment rate along the optical axis included in the adjustment command includes the following steps: Based on the adjustment rate and the time interval of image acquisition, the change in lens focal length between adjacent image acquisition times is calculated. Based on the focal length change, the radial scaling factor of the image is calculated, which is used to compensate for the change in magnification caused by the focal length change. Based on the focal length change and the adjustment rate, a shearing compensation amount is calculated, which is used to compensate for the image position shift caused by the focal length change. The modified transformation matrix is ​​constructed based on the radial scaling factor and the shear compensation amount; In the extraction of diffuse reflection and specular reflection components, the positional change of each pixel in the aligned image over time is calculated. When the positional change is less than a preset stability threshold, it is extracted as a diffuse reflection component. When the positional change exceeds the preset stability threshold, it is extracted as a specular reflection component. The specular reflection component forms the reflection distribution map sequence.

[0015] Secondly, this application provides a lens field of view control system based on target recognition, characterized in that the system includes at least one module, the at least one module being used to execute a lens field of view control method based on target recognition as described in any of the first aspects.

[0016] The technical solution provided in this application has the following advantages compared with the prior art: One of its beneficial effects and its working principle is as follows: High-magnification industrial lenses typically have a depth of field of only about 2mm. However, as a workpiece moves on a conveyor belt, its surface depth changes due to variations in its shape. Furthermore, the irregular mechanical vibrations of the conveyor belt cause millimeter-level random fluctuations in the workpiece's position in the vertical direction. A single shot cannot guarantee that the entire workpiece surface is within the clear imaging range. Existing technologies typically employ high-speed focusing mechanisms, such as electrowetting lenses, for axial scanning. Multiple images with different focal lengths are continuously acquired within a single scanning cycle to form a focal stack sequence, covering the workpiece's depth range. The core requirement of this scanning method is that the center position of the scanning area can be dynamically adjusted according to changes in the workpiece's surface depth, ensuring that the workpiece surface is always covered within the scanning range.

[0017] However, depth variations that cause scan center deviations have different sources. The mechanical vibration of the conveyor belt typically manifests as high-frequency random swinging around a reference plane (i.e., after a positional shift, it quickly returns and then shifts again). Changes in workpiece type or undulations in the workpiece surface itself (such as steps or slopes) manifest as a clear directional abrupt change or a low-frequency shift trend. This structural depth variation has a cumulative retention effect (i.e., once the position changes, it remains at the new position or continues to deviate). If a traditional controller is slow to respond to such signals, the scan center will lag behind the actual position on the workpiece surface, causing the effective depth range to separate from the workpiece surface, resulting in a period of blurring in image acquisition. Simultaneously, when conveyor belt vibration suddenly intensifies (such as when the conveyor belt crosses a bump or the equipment resonates), the depth variation exhibits large-amplitude non-periodic fluctuations, and conventional suppression strategies can cause the workpiece to fall out of the scanning range.

[0018] If the center position height is obtained based on a range sensor, the sensor must be mounted next to the lens, which creates a large angle at the macro scale, leading to blind spots or misalignment and resulting in low accuracy. Furthermore, when adjusting based on depth signals from image sequence feedback, fixed control parameters cannot simultaneously handle depth variations of the aforementioned different features.

[0019] This application extracts the trend of target depth position change from image sequences as the first feature data, and extracts the fluctuation characteristics of reflected light distribution from reflectance distribution sequence as the second feature data. Mechanical vibration of the conveyor belt causes slight angular jitter in the workpiece. Due to the amplification effect of specular reflection, this angular jitter causes a displacement of the reflective position in the image that is much larger than that of the diffuse reflection texture. Therefore, the fluctuation characteristics of reflected light distribution can characterize the intensity of mechanical vibration with high sensitivity. Workpiece type switching or surface morphology undulations represent changes in physical structure, which manifest as directional continuous displacement in the time series. Therefore, the trend of depth position change can characterize changes in workpiece structure. These two features correspond to two different physical sources: conveyor belt vibration and workpiece structural changes. Based on these two features, the predictability of target depth position change is determined, and combined states are identified. When the first feature data does not show a monotonic trend and the fluctuation intensity of the second feature data is within a preset stable range, it is determined to be an invalid fluctuation near the scan center. At this time, the damping coefficient of the controller is increased and a dead zone range is set so that depth deviations smaller than the dead zone range do not trigger adjustments to the scan center position. When the first feature data exhibits a monotonic trend, it is determined to be a continuous offset of the workpiece depth position. In this case, directional control parameters are set according to the direction of the monotonic change: the damping coefficient is reduced and the response gain is increased along the trend direction, while the damping coefficient is increased in the opposite direction. This allows the scanning center to quickly follow the depth change of the workpiece surface, while suppressing oscillations opposite to the trend to avoid oscillations. When the first feature data exhibits a non-monotonic change and the fluctuation intensity of the second feature data exceeds the preset stability range, or the change amplitude of the first feature data exceeds the preset deviation range, it is determined to be a high-dynamic condition with superimposed vibration and morphological changes. In this case, the damping limit is removed and the maximum response gain is set, thereby obtaining the maximum sensitivity range to track the workpiece position.

[0020] This variable structure control strategy, based on the predictability of depth position changes determined by dual-factor features, utilizes the high sensitivity of reflective features to vibration and the directional nature of depth trends to structural changes. This allows the center position of the scanning interval to remain stable and unresponsive to zero-mean mechanical vibrations of the conveyor belt, enabling rapid tracking of workpiece surface morphology undulations and type changes, and tracking of a wide range of sudden, severe vibrations. Without introducing additional ranging sensors, this application can distinguish depth changes from different sources based solely on image sequences and reflective distribution map sequences, avoiding the problems of frequent adjustments to the scanning center on ineffective vibrations or slow follow-up during sudden changes in workpiece depth, leading to acquisition truncation, that occur under traditional fixed-parameter control. This allows the acquisition of the focal stack sequence to better cover the workpiece surface, enabling the industrial vision inspection system to obtain continuously clear surface images in the dynamic environment of the production line.

[0021] Its second beneficial effect and its working principle are as follows: After the controller adjusts the parameters based on the two-factor feature recognition results, it generates adjustment commands to drive the electrowetting lens to rapidly change its focal length. In scenarios where the workpiece depth position continuously shifts, the controller reduces damping and increases response gain along the trend direction, enabling the lens to follow the workpiece surface at a higher speed. Under high-dynamic conditions where vibration and topographic changes are superimposed, the controller removes the damping limit and adopts maximum response gain, driving the lens to track the workpiece position at high speed.

[0022] However, during this rapid focusing process, the system still needs to continuously acquire image sequences to extract depth location information and reflective distribution characteristics. Because the lens focal length changes rapidly during image sequence acquisition, the focal lengths of the first and last frames may differ by several millimeters. This focal length difference leads to changes in magnification and perspective between different frames. Traditional image registration methods only consider the workpiece's translation along the conveyor belt direction and cannot compensate for the changes in imaging relationships caused by rapid focal length changes. The aligned image will exhibit edge streaking and local ghosting, affecting the quality of the final detected image. Simultaneously, this alignment error also affects the extraction of reflective distribution features, causing distortion of the second feature data used for control decisions in the next detection. This creates a contradiction: the faster the response speed, the worse the image alignment quality.

[0023] This application outputs adjustment trend information, including the lens's adjustment rate along the optical axis, simultaneously when generating adjustment commands. Based on this adjustment rate and the image acquisition time interval, the change in lens focal length between adjacent image acquisition moments is calculated. A radial scaling factor is calculated based on the focal length change to compensate for magnification changes caused by focal length variations. Simultaneously, a shearing compensation amount is calculated based on the focal length change and adjustment rate to compensate for imaging position shifts caused by focal length changes. A correction transformation matrix is ​​constructed based on the radial scaling factor and shearing compensation amount. This correction transformation matrix is ​​applied to the acquired image sequence to generate aligned images.

[0024] In the aligned image, the positional change of each pixel over time is calculated. If the positional change is less than a preset stability threshold, it is extracted as a diffuse reflection component; if the positional change exceeds the stability threshold, it is extracted as a specular reflection component. The diffuse reflection component corresponds to the surface features of the real object. After alignment, it remains stationary between frames and can be used to synthesize a full depth image as the final detection result output. The specular reflection component corresponds to the reflected virtual image. Because reflections do not follow rigid body motion laws, even after alignment, they will still show positional differences, forming a reflection distribution map sequence for use in the next detection to extract the second feature data.

[0025] The method provided in this application enables the image processing algorithm to perceive the real-time state of the control system and actively compensate for geometric distortions caused by rapid lens focusing. While maintaining a high-speed controller response, this application still achieves a clearer image with higher alignment accuracy, ensuring the quality of the final detected image. Simultaneously, it converts reflective features into vibration information, which is used for control decisions in the next detection cycle, forming a dynamic feedback loop from the execution layer to the decision layer. This allows the industrial vision inspection system to obtain consistently stable image quality and reliable control feedback signals even during high-speed dynamic focusing. Attached Figure Description

[0026] Figure 1 A schematic diagram illustrating an application scenario of a lens field of view control method based on target recognition provided in this application embodiment; Figure 2 One of the flowcharts for a lens field of view control method based on target recognition provided in this application embodiment; Figure 3 A second schematic flowchart illustrating a lens field of view control method based on target recognition, provided for an embodiment of this application; Figure 4 This is the third flowchart illustrating a lens field of view control method based on target recognition, provided as an embodiment of this application. Detailed Implementation

[0027] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0028] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0029] Before providing a detailed explanation of the embodiments of this application, let's first introduce the application scenarios involved in the embodiments of this application.

[0030] Figure 1 This is a schematic diagram illustrating an application scenario of the lens field of view control method based on target recognition provided in an embodiment of this application. For example... Figure 1 As shown, a vision inspection device is installed in the conveyor belt inspection scenario of an industrial automated production line. The vision inspection device includes an adjustable-focus optical lens, an image acquisition module, and a lens field-of-view control system. The lens field-of-view control system executes the lens field-of-view control method described in the following embodiments to image the workpiece moving on the conveyor belt and output a clear inspection image covering the depth range of the workpiece surface for quality inspectors or defect identification modules to determine product quality.

[0031] The lens field of view control method based on target recognition provided in this application embodiment can be loaded and executed by a lens field of view control system based on target recognition. The system includes at least one module, which is used for the lens field of view control method based on target recognition described in the following embodiments.

[0032] Reference Figures 2-4 As shown in the embodiments of this application, a lens field of view control method based on target recognition includes the following steps: S201: Acquire the image sequence and reflectance distribution map sequence for target imaging; In this step, a focusable optical lens continuously images the workpiece on the conveyor belt. The focusable optical lens uses an electrowetting lens as its focusing mechanism; the focal length is changed by adjusting the radius of curvature of the liquid lens through varying the applied voltage. Within one detection cycle, the controller drives the electrowetting lens to continuously change its focal length within a preset scanning range, and the image acquisition module simultaneously acquires image frames corresponding to different focal lengths, forming an image sequence. The reflection distribution map sequence is derived from the image processing results of the previous detection cycle.

[0033] After acquiring the image sequence, the following steps are performed to align the image sequence and extract the diffuse reflection and specular reflection components from the aligned image. The specular reflection component corresponds to the spatial distribution of reflections on the workpiece surface, and the changes of this component over time constitute the reflection distribution map sequence. Upon initial startup, the reflection distribution map sequence can be initialized as an empty sequence.

[0034] S202: Extract first feature data based on the image sequence, wherein the first feature data characterizes the changing trend of the target depth position; Specifically, the step of extracting the first feature data based on the image sequence includes: Obtain a sequence of target depth locations from the image sequence; Specifically, the process of obtaining the target depth position sequence from the image sequence includes the following steps: Within a preset scanning range, the focal length of the adjustable optical lens is changed to acquire images corresponding to different focal lengths, and the sharpness score of each image is calculated to obtain a sharpness score sequence. Within a preset scanning range, the focal length of the electrowetting lens varies according to a linear or non-linear law. Taking linear scanning as an example, let the scanning range be [ , If the scanning period is T, then the focal length f(t) changes linearly with time t. The image acquisition module acquires images at a fixed frame rate, with each frame corresponding to a focal length value. After acquisition, a sharpness score is calculated for each frame.

[0035] In this embodiment, the sharpness score is achieved using the gradient energy method. The image is convolved using the Sobel operator, and the gradient magnitudes in the horizontal and vertical directions are calculated. The sum of the squares of the gradient magnitudes across the entire image is used as the sharpness score for that frame.

[0036] Let the sharpness score of the i-th frame be . The corresponding focal length is Then, the sharpness score sequence is obtained throughout the entire scanning cycle. , ,..., } and the corresponding focal length sequence { , ,..., }

[0037] Based on the sharpness score sequence and the corresponding focal length sequence, the target depth position is obtained by energy centroid calculation, wherein the energy centroid is determined by weighted summation of sharpness score and focal length; Specifically, the process of obtaining the target depth position through energy centroid calculation includes the following steps: Obtain an ideal focal length response model, which describes the distribution characteristics of sharpness score as a function of focal length during focal length scanning of a qualified workpiece; Calculate the ideal energy centroid based on the aforementioned ideal focal length response model; The actual energy centroid, calculated by weighted summation based on the sharpness score sequence and the corresponding focal length sequence, is compared with the ideal energy centroid to calculate the energy centroid offset. The target depth position is determined based on the energy center of gravity offset.

[0038] Specifically, in the embodiments of this application, the energy center of gravity The calculation formula is: This calculation method uses the sharpness score as a weight and calculates a weighted average based on the focal length. It reflects the location of energy concentration in the sharpness distribution of the workpiece surface within the scanning range.

[0039] This application introduces an ideal focal length response model as a benchmark in energy centroid calculation. The ideal focal length response model describes the distribution characteristics of sharpness score as a function of focal length during focal length scanning of a qualified workpiece. This model is expressed using a Gaussian function: in Rate peak resolution. This represents the focal length position corresponding to the peak value. This is the distribution width parameter. This function reflects the distribution pattern of the workpiece surface under ideal conditions, exhibiting a single, clear peak.

[0040] The parameters of the ideal focal length response model are obtained through system calibration. The calibration process is performed before the production line officially starts operation. Several qualified workpiece samples from the same batch are selected; these samples have undergone quality inspection to confirm that their surface quality meets requirements. A focal length scan is performed on each sample, and the sharpness score curve during the scan is recorded. The sharpness score curves of multiple samples are aligned according to focal length, and the mean and variance of the sharpness score at each focal length position are calculated. Based on the mean curve, a Gaussian function is fitted using the least squares method to obtain parameters A. The values ​​of σ and σ.

[0041] Calculate the ideal energy centroid based on the ideal focal length response model. The calculation method is as follows: The actual energy center of gravity With the ideal energy center of gravity Compare and calculate the energy center of gravity offset: This offset reflects the degree of deviation of the actual depth position of the workpiece from the ideal position. The center focal length of the scanning range... Add offset To obtain the focal length value corresponding to the target. : This focal length value reflects the change in the target's depth position. In the following description, for simplicity, this focal length value will be used as the target depth position d.

[0042] The above acquisition process is repeated over time. Each detection cycle completes one focal length scan and energy centroid calculation, yielding a target depth position value. The m most recent consecutive detected target depth position values ​​constitute a depth position sequence. , ,..., }

[0043] Calculate the rate of change of depth position based on the depth position sequence; Specifically, the rate of change is calculated using the difference between adjacent depth positions: Obtain the rate of change sequence { , ,..., }

[0044] Determine whether the rate of change exhibits monotonicity, and if it does, determine the direction of the monotonic change; Specifically, in this embodiment, the monotonicity determination employs a sign consistency test: counting the number of positive, negative, and near-zero values ​​in the rate of change sequence. If the proportion of positive values ​​exceeds a preset threshold (e.g., 80%), it is determined to be a positive monotonic change, with the direction of monotonicity being positive. If the proportion of negative values ​​exceeds a preset threshold, it is determined to be a negative monotonic change, with the direction of monotonicity being negative. If neither the number of positive nor negative values ​​exceeds the threshold, it is determined to be a non-monotonic change.

[0045] The first feature data is determined based on the monotonicity characteristics and the direction of monotonic change to characterize the changing trend of the target depth position.

[0046] Specifically, the first feature data contains two elements: a monotonicity indicator and a direction indicator. The monotonicity indicator takes the value of 0 (monotonic) or 1 (non-monotonic), while the direction indicator takes the value of +1 (positive) or -1 (negative) in the case of monotonicity, and takes the value of 0 (invalid) in the case of non-monotonicity.

[0047] S203: Extract second feature data based on the reflected light distribution sequence, wherein the second feature data characterizes the fluctuation characteristics of the reflected light distribution; Specifically, the process of extracting the second feature data based on the reflectance distribution map sequence includes the following steps: Extract the position sequence of the centroid of reflection from the reflection distribution map sequence; Specifically, in this embodiment, the reflectivity distribution map sequence originates from the extraction result of step S303 in the previous detection cycle, wherein the spatial distribution of specular reflection components between consecutive frames constitutes the reflectivity distribution map sequence. During the first detection, the reflectivity distribution map sequence is initialized as an empty sequence, and the second feature data uses default values.

[0048] First, the reflectivity distribution map of each frame is binarized to extract the set of pixel coordinates of the reflectivity area.

[0049] The centroid of reflection is calculated using a weighted average of pixel coordinates. Let the set of pixel coordinates of the reflective region in the reflective distribution map of the j-th frame be {( , The brightness value of the corresponding pixel is )} Then the coordinates of the centroid of the reflected light ( , The calculation is as follows: The coordinates of the centroids of reflections in multiple consecutive frames of reflection distribution maps constitute a position sequence {( , ), ( , ), ..., ( , )}.

[0050] The second feature data is calculated based on the statistical characteristics of the position sequence to characterize the fluctuation intensity of the reflected light distribution; The statistical characteristics include displacement amplitude or trajectory entropy.

[0051] Specifically, in this embodiment, the displacement amplitude is used for calculation, and the calculation method is as follows: Calculate the Euclidean distance between adjacent reflective centroids to obtain the displacement sequence { , ,..., The standard deviation of the displacement sequence is calculated and used as a quantitative indicator of the fluctuation intensity, i.e., as the second characteristic data.

[0052] S204: Dynamically adjust the parameter structure of the controller according to the combined state of the first feature data and the second feature data, so that the controller can produce differentiated adjustment responses to different combined states; Specifically, the process of dynamically adjusting the controller's parameter structure based on the combined state of the first feature data and the second feature data includes the following steps: Based on the first feature data and the second feature data, the predictability of the target depth position change is determined, and the combined state is identified, wherein the combined state includes at least: In the first combination state, the first feature data does not show a monotonic change trend (the monotonicity of the first feature data is marked as 1 (non-monotonic)) and the fluctuation intensity of the second feature data is within a preset stable range; Specifically, in the embodiments of this application, the preset stable range is determined by statistically analyzing the intensity of reflective fluctuations under normal operating conditions, and is usually set as the range of the mean plus or minus two standard deviations.

[0053] The second combination state, wherein the first feature data exhibits a monotonic change trend (the monotonicity of the first feature data is marked as 0 (monotonic)). The third combination state is characterized by the first feature data exhibiting non-monotonic variation characteristics (the monotonicity of the first feature data is marked as 1 (non-monotonic)) and the fluctuation intensity of the second feature data exceeding the preset stable range, or the variation amplitude of the first feature data exceeding the preset deviation range. The preset deviation range is determined based on the effective depth of field of the scanning interval and is usually set to 50% of the depth of field range.

[0054] The first combined state corresponds to invalid fluctuations in the target depth position, requiring suppression of the focus adjustment response; the second combined state corresponds to continuous shifts in the target depth position, requiring enhancement of the focus adjustment response along the shift direction; and the third combined state corresponds to high dynamic changes in the target depth position, requiring omnidirectional enhancement of the focus adjustment response. Specifically, in this embodiment, the identification of the combined state is achieved through conditional judgment logic. First, the monotonicity indicator of the first feature data is judged. If it is 0, it is directly identified as the second combined state. If it is 1, it is further judged whether the second feature data exceeds a preset stable range or whether the depth position change exceeds a preset deviation range. If so, it is identified as the third combined state; otherwise, it is identified as the first combined state.

[0055] The parameter structure of the controller is adjusted for different combinations of states so that different focal length adjustment responses are generated for the same depth deviation under different combinations of states.

[0056] The controller employs a PID control architecture. The controller output is the adjustment amount for the scan center position. The calculation formula is: in, The depth deviation at the current moment is the energy centroid offset Δf calculated in step S202. This offset reflects the degree of deviation of the actual depth position of the workpiece from the ideal position, which is also the amount of focal length that needs to be adjusted at the center of the current scanning interval. For proportional gain, For integral gain, This is the differential gain. The parameter structure is adjusted by changing... , and The values ​​of are determined by introducing dead zone range and directional weights.

[0057] When the first combined state is detected, the damping coefficient of the controller is increased and the dead zone range is set so that depth deviations smaller than the dead zone range do not trigger the focus adjustment response. Specifically, the damping coefficient is reduced by decreasing the proportional gain. and differential gain The specific value is set to 30% to 50% of the standard parameters. The dead zone range is set to ±0.5mm of the focal length deviation. That is, when the absolute value of the depth deviation e(t) (corresponding to the focal length offset Δf) is less than 0.5mm, the controller output is zero and the adjustment of the scanning center position is not triggered.

[0058] When the second combined state is identified, directional control parameters are set according to the direction of monotonic change, wherein the damping coefficient is reduced and the response gain is increased along the direction of monotonic change, and the damping coefficient is increased in the opposite direction. Specifically, directional control is achieved by setting different gain coefficients for positive and negative deviations. Let the direction identifier of the first characteristic data be dir (with a value of 1), then the proportional gain is adjusted as follows: in Here, is the standard proportional gain, and sign(e(t)) represents the sign of the depth deviation. This parameter structure allows the controller to reduce damping and increase response gain along the monotonically changing direction, enabling the scan center to quickly follow the depth changes on the workpiece surface. Increasing damping in the opposite direction suppresses oscillations that run counter to the trend, preventing control oscillations.

[0059] When the third combined state is detected, the damping limit is removed and the maximum response gain is set; Specifically, proportional gain and differential gain All are set to twice the standard parameters, integral gain Set to 1.5 times the standard parameters. The dead zone is set to zero, and all depth deviations trigger an adjustment response. This parameter configuration allows the controller to achieve maximum response speed and range of variation.

[0060] The adjustment command is generated based on the adjusted parameter structure, wherein the adjustment command includes an adjustment amount of the scanning center position along the optical axis to achieve clear imaging range control along the optical axis.

[0061] The parameter structure adjustment is performed before the start of each detection cycle. Based on the currently identified combination state, the controller loads the corresponding gain coefficient and dead zone range from the parameter configuration table, updates the internal parameters, and then proceeds with the adjustment calculation.

[0062] Based on the adjusted parameter structure, the controller calculates the current depth deviation and generates an adjustment amount for the scan center position. Adjustment amount This indicates the amount of focal length change required to adjust the center focal length of the scanned area, with the same unit as focal length. A positive value indicates that the center focal length needs to be increased (corresponding to the scanned area moving away from the lens), while a negative value indicates that the center focal length needs to be decreased (corresponding to the scanned area moving closer to the lens).

[0063] S205: Generate adjustment instructions based on the adjusted parameter structure. The adjustment instructions are used to drive the adjustable focus optical lens to change the focal length in order to achieve lens field of view control.

[0064] Specifically, the adjustment instructions include the scan center position adjustment amount. and adjustment rate Adjustment rate The adjustment rate v(t) is the derivative of the adjustment amount with respect to time, reflecting the rate of change of the scan center position. In practice, the adjustment rate v(t) is approximately calculated by the difference in adjustment amounts between two adjacent detection cycles. Where Δt is the time interval of the detection cycle.

[0065] The adjustment command is output in the form of a data structure, containing the adjustment amount. Adjustment rate Updated center focal length of the scanning interval and the corresponding scan interval boundaries [ , The updated focal length of the scan interval center is calculated as follows: The scanning interval boundary is determined based on the preset scanning range width W: The adjustment command is sent to the electrowetting lens drive module. The drive module calculates the change in applied voltage based on the adjustment amount u(t) and adjusts the lens focal length by changing the radius of curvature of the liquid lens. In the next detection cycle, the image acquisition module scans within the updated scan interval […]. , The system acquires a focal stack sequence within the scanning range, enabling dynamic tracking along the optical axis to ensure that the workpiece surface remains within a clear imaging range.

[0066] The process of executing the adjustment command to drive the adjustable optical lens to change the focal length also includes the following steps: S301: Based on the adjustment rate along the optical axis contained in the adjustment command, calculate the change in viewing angle caused by the change in lens focal length; construct a correction transformation matrix based on the change in viewing angle, the correction transformation matrix including cropping compensation for the image sequence to compensate for changes in magnification and perspective caused by dynamic changes in focal length. Specifically, the process of constructing the correction transformation matrix based on the adjustment rate along the optical axis included in the adjustment command includes the following steps: S401: Based on the adjustment rate and the time interval of image acquisition, calculate the change in lens focal length between adjacent image acquisition times; The image acquisition module acquires images at a fixed frame rate, and the inter-frame time interval is set to... At that moment and The change in lens focal length between two captured images The calculation is as follows: During the entire detection period, the focal length changes between multiple frames are accumulated to obtain the focal length change sequence. , ,..., This sequence reflects the dynamic change of focal length during image acquisition.

[0067] S402: Based on the focal length change, calculate the radial scaling factor of the image, which is used to compensate for the change in magnification caused by the focal length change; The radial scaling factor is used to compensate for changes in magnification caused by variations in focal length. Let the focal length corresponding to the i-th frame be... The focal length corresponding to the (i+1)th frame image is The radial scaling factor between the two frames is... The calculation is as follows: Because in this embodiment of the application, Compared to Since the size is relatively small, a first-order approximation is used: Radial scaling factor Applied to the (i+1)th frame of the image, this transforms each pixel in the image by radial scaling relative to the image center: Where (x, y) are the original pixel coordinates, and (x', y') are the scaled pixel coordinates. , () represents the coordinates of the image center.

[0068] S403: Calculate the shearing compensation amount based on the focal length change and the adjustment rate. The shearing compensation amount is used to compensate for the image position shift caused by the focal length change. Clipping compensation is used to compensate for image position shift caused by lens movement along the optical axis. When the lens moves along the optical axis, the intersection point of the principal optical axis of the imaging optical path and the object surface shifts. This shift manifests as an overall translation on the image plane, and the amount of translation is related to the change in focal length and the adjustment rate.

[0069] It should be noted that the methods for achieving focal length changes differ for different types of adjustable focusing optical lenses. For electrowetting lenses, focal length changes are mainly achieved by altering the radius of curvature of the liquid lens. The entire lens does not need to physically move along the optical axis; rapid focusing can be achieved by changing the imaging optical path. This makes electrowetting lenses particularly suitable for the rapid dynamic focusing scenario described in this application. For piezoelectric ceramic driven lens groups, focal length changes are achieved through the physical displacement of the lens group along the optical axis. Despite the different mechanisms, focal length changes both lead to changes in the imaging relationship, resulting in a positional shift on the image plane.

[0070] Image cropping compensation This represents the translation of the (i+1)th frame relative to the ith frame in the x and y directions. This translation is related to the change in focal length. Proportional: in and The shearing coefficient is obtained through system calibration. The calibration process uses a calibration plate at a known location, acquires images at different focal lengths, calculates the ratio of image offset to focal length change through feature point matching, and then fits the result. and The value of .

[0071] S404: Construct the corrected transformation matrix based on the radial scaling factor and the shear compensation amount; Correction transformation matrix Radial scaling and shear compensation are combined into a single affine transformation. For the (i+1)th frame of the image, the transformation matrix is ​​corrected. Represented as: This matrix maps the pixel coordinates (x, y) of the (i+1)th frame image to the aligned coordinate system: Throughout the image sequence, taking the first frame as the baseline, the corresponding correction transformation matrix is ​​applied sequentially to each subsequent frame. The cumulative transformation matrix of the k-th frame is shown below. The product of the first k-1 correction transformation matrices: S302: Apply the correction transformation matrix to the image sequence to generate an aligned image; For each frame in the image sequence, a corresponding cumulative transformation matrix is ​​applied to map the pixel coordinates to a unified reference coordinate system. The mapping process uses bilinear interpolation. For the transformed coordinate (x', y'), if the coordinate is a non-integer value, the pixel value at that position is obtained by interpolation based on the pixel values ​​at the four surrounding integer coordinate positions.

[0072] In the aligned image sequence, the geometric relationships of each frame are unified. The true texture of the workpiece surface remains consistent in position across frames, while the reflective virtual image, due to its non-rigid body motion, still shows positional differences after alignment.

[0073] S303: Extract diffuse reflection and specular reflection components from the aligned image, wherein the diffuse reflection component is used to synthesize the final output detection image, and the specular reflection component generates the reflection distribution map sequence for extracting second feature data for the next detection.

[0074] In the extraction of diffuse reflection and specular reflection components, the positional change of each pixel in the aligned image over time is calculated. When the positional change is less than a preset stability threshold, it is extracted as a diffuse reflection component. When the positional change exceeds the preset stability threshold, it is extracted as a specular reflection component. The specular reflection component forms the reflection distribution map sequence.

[0075] Specifically, in this embodiment, for the aligned image sequence, the positional change of each pixel over time is calculated. When calculating on a pixel-by-pixel basis, for a pixel position (x, y) in the image, the grayscale value sequence of that position in each frame is extracted. , , ..., }. Calculate the standard deviation σ(x, y) of the sequence.

[0076] The standard deviation σ(x,y) reflects the degree of grayscale fluctuation of the pixel position over time, indirectly characterizing the amount of positional change.

[0077] Set preset stability threshold For each pixel position (x, y), its standard deviation σ(x, y) is compared with a preset stability threshold. Compare them. When σ(x,y) is less than When σ(x,y) is greater than or equal to 0, the pixel is considered to be in a stable position and is classified as a diffuse reflection component. At this time, the pixel is determined to be in an unstable position and is classified as a specular reflection component.

[0078] For pixels classified as diffuse components, the output value Take the average grayscale value of each frame: For pixels classified as specular reflection components, their grayscale values ​​in each frame are retained, without temporal merging. The specular reflection component value of this pixel in the i-th frame is... for: The specular reflection components of each frame of the image constitute a sequence of reflection distribution maps. , ,..., This sequence preserves the dynamic changes in reflectivity over time, and is used to extract the position sequence of the reflectivity centroid in S203 during the next detection.

[0079] Preset stability threshold The standard deviation of pixel grayscale values ​​is determined based on the noise level of the image acquisition system. In flat, non-reflective areas, the standard deviation of pixel grayscale values ​​is mainly contributed by camera noise. By acquiring non-reflective sample images and statistically analyzing the distribution of pixel standard deviations, the 95th percentile can be used as a stable threshold to effectively distinguish between noise fluctuations and actual positional changes.

[0080] Diffuse reflection component The resulting image is the final output detection image. This image integrates the sharp regions from the entire focal stack sequence, achieving a panoramic depth imaging effect. The detection image is used for subsequent quality inspection tasks such as defect identification.

[0081] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0082] In the various embodiments of this application, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments are consistent and can be referenced mutually. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships. In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. In the textual description of the embodiments of this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. In this application, "first," "second," and various numerical designations are only for ease of description and are not used to limit the scope of the embodiments of this application. For example, they are used to distinguish different messages, rather than to describe a specific order or sequence.

[0083] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. The order of the process numbers does not imply the order of execution; the execution order of each process should be determined by its function and internal logic.

[0084] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A lens field-of-view control method based on target recognition, characterized in that, The lens field of view control method based on target recognition includes the following steps: Acquire image sequences and reflection distribution map sequences for target imaging; First feature data is extracted based on the image sequence, and the first feature data represents the changing trend of the target depth position; The second feature data is extracted based on the reflected light distribution sequence, and the second feature data characterizes the fluctuation characteristics of the reflected light distribution; The parameter structure of the controller is dynamically adjusted according to the combined state of the first feature data and the second feature data, so that the controller can produce differentiated adjustment responses to different combined states. An adjustment command is generated based on the adjusted parameter structure. The adjustment command is used to drive the adjustable focus optical lens to change the focal length in order to control the lens field of view.

2. The lens field of view control method based on target recognition according to claim 1, characterized in that, The process of dynamically adjusting the controller's parameter structure based on the combined state of the first feature data and the second feature data includes the following steps: Based on the first feature data and the second feature data, the predictability of the target depth position change is determined, and the combined state is identified, wherein the combined state includes at least: In the first combination state, the first feature data does not show a monotonic change trend and the fluctuation intensity of the second feature data is within a preset stable range; The second combination state, wherein the first feature data exhibits a monotonic change trend; The third combination state is characterized in that the first feature data exhibits non-monotonic change characteristics and the fluctuation intensity of the second feature data exceeds the preset stable range, or the change amplitude of the first feature data exceeds the preset deviation range. The first combined state corresponds to invalid fluctuations in the target depth position, requiring suppression of the focus adjustment response; the second combined state corresponds to continuous shifts in the target depth position, requiring enhancement of the focus adjustment response along the shift direction; and the third combined state corresponds to high dynamic changes in the target depth position, requiring omnidirectional enhancement of the focus adjustment response. The parameter structure of the controller is adjusted for different combinations of states so that different focal length adjustment responses are generated for the same depth deviation under different combinations of states.

3. The lens field of view control method based on target recognition according to claim 1, characterized in that, The process of extracting the second feature data based on the reflectance distribution map sequence includes the following steps: Extract the position sequence of the centroid of reflection from the reflection distribution map sequence; The second feature data is calculated based on the statistical characteristics of the position sequence to characterize the fluctuation intensity of the reflected light distribution; The statistical characteristics include displacement amplitude or trajectory entropy.

4. The lens field of view control method based on target recognition according to claim 2, characterized in that, The step of extracting the first feature data based on the image sequence includes: Obtain a sequence of target depth locations from the image sequence; Calculate the rate of change of depth position based on the depth position sequence; Determine whether the rate of change exhibits monotonicity, and if it does, determine the direction of the monotonic change; The first feature data is determined based on the monotonicity characteristics and the direction of monotonic change to characterize the changing trend of the target depth position.

5. The lens field of view control method based on target recognition according to claim 4, characterized in that, The process of adjusting the parameter structure of the controller for different combinations of states includes the following steps: When the first combined state is detected, the damping coefficient of the controller is increased and the dead zone range is set so that depth deviations smaller than the dead zone range do not trigger the focus adjustment response. When the second combined state is identified, directional control parameters are set according to the direction of monotonic change, wherein the damping coefficient is reduced and the response gain is increased along the direction of monotonic change, and the damping coefficient is increased in the opposite direction. When the third combined state is detected, the damping limit is removed and the maximum response gain is set; The adjustment command is generated based on the adjusted parameter structure, wherein the adjustment command includes an adjustment amount of the scanning center position along the optical axis to achieve clear imaging range control along the optical axis.

6. The lens field of view control method based on target recognition according to claim 4, characterized in that, The process of obtaining the sequence of target depth locations from the image sequence includes the following steps: Within a preset scanning range, the focal length of the adjustable optical lens is changed to acquire images corresponding to different focal lengths, and the sharpness score of each image is calculated to obtain a sharpness score sequence. Based on the sharpness score sequence and the corresponding focal length sequence, the target depth position is obtained by energy centroid calculation, wherein the energy centroid is determined by weighted summation of sharpness score and focal length; Repeat the above acquisition process over time to obtain a sequence of the target depth positions.

7. The lens field of view control method based on target recognition according to claim 6, characterized in that, The process of obtaining the target depth position through energy centroid calculation includes the following steps: Obtain an ideal focal length response model, which describes the distribution characteristics of sharpness score as a function of focal length during focal length scanning of a qualified workpiece; Calculate the ideal energy centroid based on the aforementioned ideal focal length response model; The actual energy centroid, calculated by weighted summation based on the sharpness score sequence and the corresponding focal length sequence, is compared with the ideal energy centroid to calculate the energy centroid offset. The target depth position is determined based on the energy center of gravity offset.

8. The lens field of view control method based on target recognition according to claim 1, characterized in that, The process of executing the adjustment command to drive the adjustable optical lens to change the focal length also includes the following steps: Based on the adjustment rate along the optical axis contained in the adjustment command, the change in angle of view caused by the change in lens focal length is calculated. A correction transformation matrix is ​​constructed based on the aforementioned change in viewing angle. The correction transformation matrix includes cropping compensation for the image sequence to compensate for changes in magnification and perspective caused by dynamic changes in focal length. The correction transformation matrix is ​​applied to the image sequence to generate aligned images; The diffuse reflection component and the specular reflection component are extracted from the aligned image, wherein the diffuse reflection component is used to synthesize the final output detection image, and the specular reflection component generates the reflection distribution map sequence for extracting the second feature data for the next detection.

9. A lens field of view control method based on target recognition according to claim 8, characterized in that, The process of constructing the correction transformation matrix based on the adjustment rate along the optical axis included in the adjustment command includes the following steps: Based on the adjustment rate and the time interval of image acquisition, the change in lens focal length between adjacent image acquisition times is calculated. Based on the focal length change, the radial scaling factor of the image is calculated, which is used to compensate for the change in magnification caused by the focal length change. Based on the focal length change and the adjustment rate, a shearing compensation amount is calculated, which is used to compensate for the image position shift caused by the focal length change. The modified transformation matrix is ​​constructed based on the radial scaling factor and the shear compensation amount; In the extraction of diffuse reflection and specular reflection components, the positional change of each pixel in the aligned image over time is calculated. When the positional change is less than a preset stability threshold, it is extracted as a diffuse reflection component. When the positional change exceeds the preset stability threshold, it is extracted as a specular reflection component. The specular reflection component forms the reflection distribution map sequence.

10. A lens field-of-view control system based on target recognition, characterized in that, The system includes at least one module, which is used to execute a lens field of view control method based on target recognition as described in any one of claims 1-9.

Images