A binocular camera alignment method for a binocular active alignment device of an XR device

Abstract

The present application relates to the technical field of precision optical assembly and data processing, in particular to a binocular camera positioning method for a binocular active alignment device of an XR device. The method comprises: acquiring multi-dimensional pose coupling data flow in the adjustment process, establishing a time synchronization mechanism and preprocessing the data; constructing a pose coupling response index, evaluating the nonlinear coupling strength under the current mechanical state based on historical data and current action intention; calculating a feedforward compensation amount based on adaptive gain, superimposing a reverse position compensation instruction while issuing an angle adjustment instruction, and dynamically updating the gain coefficient according to the residual error; performing closed-loop fusion control and execution, driving the device to act after fusing the compensation amount and the original instruction. The present application effectively eliminates the oscillation phenomenon in the adjustment process by introducing a dynamic coupling index and a feedforward compensation mechanism, and significantly improves the alignment efficiency and adaptability to device aging.

Description

Technical Field

[0001] This invention relates to the field of precision optical assembly and data processing technology. More specifically, this invention relates to a binocular camera positioning method for a binocular active alignment device in XR equipment. Background Technology

[0002] In the manufacturing process of XR devices, active alignment is a crucial process to ensure the optimal optical performance of the binocular camera module. This process typically involves clamping the optomechanical module to be assembled onto the end of a six-axis robot or a six-DOF precision slide, projecting a specific image onto a target chart while the screen is illuminated, and acquiring image data in real time using an industrial camera to analyze the current image quality. Based on the analysis results, the system repeatedly adjusts the spatial pose of the optomechanical module until all indicators converge to a preset acceptable range. However, in actual micron-level precision adjustments, existing technologies face severe challenges related to multi-DOF coupling. Specifically, because the rotation center of the mechanical slide and the photosensitive center of the optomechanical module often do not coincide in physical space, resulting in an arm length effect, coupled with the stress deformation of the flexible fixture and the Abbe error of the mechanical transmission, when the system attempts to adjust the UV angle to correct the optical axis parallelism, it inevitably causes a nonlinear, minute drift in the center position on the XY plane.

[0003] To address the multi-degree-of-freedom coupling problem in the existing technologies, the common improvement methods include: using independent PID control loops to adjust each dimension separately, or using a static decoupling matrix based on fixed geometric parameters for compensation.

[0004] However, the above solution still has significant shortcomings and defects in practical applications. First, this approach has a clear logical flaw: the operation of correcting angular deviations compromises positional accuracy, forcing the system to repeatedly oscillate between adjusting angles and correcting positions, resulting in iterative oscillations and slow convergence, significantly increasing the number of cycles and severely slowing down the production cycle. Second, the fixed decoupled model cannot adapt to long-term wear and tear of the equipment, changes in lubrication conditions, and subtle differences between different batches of contoured parts, leading to the failure of the static model, inaccurate compensation, and difficulty in meeting the increasingly stringent accuracy requirements of XR equipment. Summary of the Invention

[0005] The purpose of this invention is to propose an intelligent security image recognition method based on deep learning to solve the problems of iterative oscillation, slow convergence, and static model inability to adapt to equipment wear caused by multi-degree-of-freedom coupling in the prior art; to this end, the present invention provides a solution in one aspect.

[0006] This invention provides a binocular camera positioning method for a binocular active alignment device in XR equipment, comprising:

[0007] The system acquires multi-dimensional pose coupling data streams during the adjustment process, monitors and synchronously collects the motion state of the six-axis robot and the feedback data from the vision system in real time, and performs preprocessing. It constructs a pose coupling response index, and uses the preprocessed data, combined with historical adjustment records and the current planned adjustment amount, to evaluate the nonlinear coupling strength under the current mechanical state. Based on adaptive gain, it calculates the feedforward compensation amount, and according to the pose coupling response index and the basic geometric transformation relationship, combined with a dynamically updated adaptive gain coefficient, calculates the required superimposed reverse position compensation command. It then performs closed-loop fusion control and execution, fusing the feedforward compensation amount with the original angle adjustment command to generate the final drive command and control the device to execute it until the optical parameters meet the preset requirements.

[0008] Preferably, the acquisition of the multidimensional pose coupling data stream during the adjustment process includes: establishing a time synchronization mechanism to align the command issuance time of the motion controller with the image capture time of the vision acquisition card; and constructing a sliding window to store the most recent... The historical state data for each adjustment cycle includes control command data and visual feedback data; wherein, the control command data records the angle adjustment amount issued by the system in each round, and the visual feedback data records the actual offset of the image center on the XY plane after each round of adjustment is completed.

[0009] Preferably, the control command data includes the increment of rotation around the X-axis and the increment of rotation around the Y-axis; the visual feedback data is obtained by image processing algorithms such as the centroid method or template matching.

[0010] Preferably, after acquiring the visual feedback data, the acquired sequence is subjected to a weighted moving average filter to eliminate image noise and high-frequency interference caused by mechanical vibrations.

[0011] Preferably, the constructed pose coupling response index satisfies the following expression:

[0012]

[0013] in, For the current moment The pose coupling response index; The observation window length for historical data; For summation loop variables; For the first time in history The Euclidean position drift caused by wheel adjustment; For the first time in history The modulus of the angle adjustment command issued by the wheel; This is the minimum action threshold constant; For the current moment The planned angle adjustment command module length; The standard deviation of the angle adjustment within the historical window; It is an exponential function with the natural constant e as its base.

[0014] Through the above technical solution, the formula assigns higher weights to larger historical errors by using the square root of the sum of squares, and introduces consideration of the current adjustment magnitude and historical volatility through an exponential function. This design enables the exponent to respond non-linearly to changes in the coupled state of the system, providing a stronger gain signal, especially when facing sudden large adjustments.

[0015] Preferably, the step of calculating the feedforward compensation amount based on adaptive gain satisfies the following expression:

[0016]

[0017] in, For the current moment The XY-axis feedforward position compensation vectors that need to be superimposed; For adaptive prediction gain coefficients; The pose coupling response index; The basic geometric transformation matrix; This is the angle command vector to be issued in the current plan.

[0018] Through the above technical solution, this formula uses the coupling response exponent as a dynamic weight for the geometric transformation, combined with an adaptive gain coefficient, to achieve fine-tuning of the compensation amount. The negative sign ensures that the compensation direction is opposite to the drift direction, thus effectively offsetting the coupling error.

[0019] Preferably, the basic geometric transformation matrix is ​​determined by the physical distance from the optomechanical rotation center to the imaging plane.

[0020] Preferably, the update logic of the adaptive gain coefficient satisfies the following expression:

[0021]

[0022] in, For adaptive gain coefficients; This is the gain coefficient from the previous round; The learning rate is a constant; This represents the residual error after the previous round of adjustments. This is the maximum allowable error limit for the system.

[0023] Through the above technical solution, the update mechanism gives the system the ability to learn continuously. As the equipment runs longer, even if the slide wears or the lubrication condition changes, the algorithm can automatically adjust the gain coefficient through residual error feedback, thereby maintaining a high-precision decoupling effect and reducing the dependence on manual calibration and maintenance.

[0024] Preferably, the closed-loop fusion control and execution includes: superimposing the X-axis compensation component of the feedforward compensation amount onto the target position X adjustment amount to generate the final X-axis command; superimposing the Y-axis compensation component of the feedforward compensation amount onto the target position Y adjustment amount to generate the final Y-axis command; keeping the originally calculated angle adjustment command unchanged; driving the robot to execute the above final command, and collecting the image calculation error again, storing the data in the history window to enter the next cycle.

[0025] Preferably, the length of the sliding window The value ranges from 5 to 10.

[0026] The beneficial effects of this invention are as follows: By real-time acquisition of the pose control command stream and visual feedback error stream during the adjustment process, a dynamic pose coupling response index is constructed to evaluate the nonlinear coupling strength under the current mechanical state; furthermore, by combining an adaptive gain feedforward compensation model, a reverse position compensation command is automatically superimposed while issuing the angle adjustment command. This method effectively solves the iterative oscillation and model failure problems faced by traditional PID control or static decoupling matrices, greatly improving the convergence speed of the AA process, shortening the conventional 5-8 iterations to 2-3 iterations, while enhancing the equipment's adaptability to aging and environmental changes, and achieving precise control of nonlinear errors. Attached Figure Description

[0027] Figure 1 This schematically illustrates the steps of the binocular camera positioning method for a binocular active alignment device in an XR device according to this embodiment;

[0028] Figure 2 This is a comparison diagram of the deviation convergence between the method of this invention and the prior art in the active alignment process of a binocular camera. Detailed Implementation

[0029] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0030] like Figure 1 As shown in this embodiment, a binocular camera positioning method for a binocular active alignment device in an XR device includes the following steps:

[0031] Step S1: Obtain the multi-dimensional pose coupling data stream during the adjustment process, monitor and synchronously collect the motion state of the six-axis robot and the feedback data of the vision system in real time, and perform preprocessing.

[0032] During the coarse and fine adjustment stages of the AA process, the system monitors and synchronously collects the motion state of the six-axis robot and the feedback data from the vision system in real time. Specifically, a time synchronization mechanism is established to align the command issuance time of the motion controller with the image capture time of the vision acquisition card. A data structure of length [missing information] is constructed. A sliding window used to store the most recent Historical state data for each adjustment cycle. For example, the length of the sliding window... The value ranges from 5 to 10.

[0033] The collected data items include control command data and visual feedback data. The control command data records the angle adjustments planned and issued by the system in each round, including the increments of rotation around the X-axis and rotation around the Y-axis, denoted as follows: and .

[0034] Visual feedback data is the actual offset of the image center in the XY plane calculated by an image processing algorithm using the centroid method or template matching after each round of adjustment. It is denoted as... and .

[0035] To eliminate image noise and high-frequency interference caused by mechanical vibrations, the acquired data was processed... and The sequence undergoes weighted moving average filtering to ensure that the data used in subsequent calculations reflects the true physical displacement trend. For example, if the currently collected raw data... The sequence is [2.1, 2.0, 1.9, 2.2, 2.0] micrometers. By setting weighting coefficients and applying weighted filtering, a smoother value of 2.04 micrometers may be obtained, thus avoiding the random noise of a single sample from misleading subsequent models.

[0036] With the above settings, by establishing a strict time synchronization mechanism and sliding window filtering, it can be ensured that the acquired control commands and visual feedback correspond strictly in the time domain, and high-frequency noise interference can be effectively filtered out, providing a reliable and clean data foundation for the subsequent construction of an accurate coupled response index.

[0037] Step S2: Construct the pose coupling response index. Using the preprocessed data, combined with historical adjustment records and the current planned adjustment amount, evaluate the nonlinear coupling strength under the current mechanical state.

[0038] The coupling effect of a mechanical system is not linearly constant, but rather dynamically changes with the magnitude of the adjustment, the extension posture of the robotic arm, and the current load state. To accurately assess the impact of the angle adjustment on positional accuracy at the current moment, this step constructs a pose coupling response index.

[0039] Let the current time be The expression for the pose coupling response exponent is as follows:

[0040]

[0041] in, For the current moment The pose coupling response index; The observation window length for historical data, rounded to an integer, for example, The value of is 5; The loop variable is used for summation, representing the historical iterations of the backtracking process; For the first time in history The Euclidean position drift caused by wheel adjustment is calculated as follows: ; For the first time in history The modulus of the angle adjustment command issued by the wheel is calculated as follows: ; This is the minimum action threshold constant, used to prevent the denominator from being zero. For example, it can be 0.001 degrees. For the current moment The planned angle adjustment command module length; The standard deviation of the angle adjustment within the historical window; It is an exponential function with the natural constant e as its base.

[0042] Thus, by constructing a pose coupling response index that includes historical coupling rate statistics and current motion intention prediction, the nonlinear coupling strength of the mechanical system can be dynamically and sensitively evaluated, especially the potential risks brought about by large-scale motion adjustments, providing a scientific quantitative indicator for achieving precise decoupling control.

[0043] Step S3: Calculate the feedforward compensation amount based on the adaptive gain. According to the pose coupling response index and the basic geometric transformation relationship, combined with the dynamically updated adaptive gain coefficient, calculate the reverse position compensation command that needs to be superimposed.

[0044] After obtaining the coupling strength, it is not enough to rely solely on PID control with feedback lag; the deviation must also be anticipated and compensated before the command is issued. This step utilizes the principle of geometric projection combined with the dynamic coupling index calculated in the previous step to calculate the reverse compensation command that needs to be superimposed on the XY axis.

[0045] Define the current time XY-axis feedforward position compensation vectors to be superimposed It satisfies the following expression:

[0046]

[0047] in, For the current moment The XY-axis feedforward position compensation vectors that need to be superimposed. It is a two-dimensional column vector containing two elements. , representing the additional compensation displacement required along the X-axis and Y-axis, respectively; The adaptive prediction gain coefficient is a scalar value. The pose coupling response index; The basic geometric transformation matrix is ​​determined by the physical distance from the optomechanical rotation center to the imaging plane; The current planned angle command vector contains... The negative sign in the expression indicates that the compensation direction is opposite to the predicted drift direction, thus having a counteracting effect.

[0048] In order for the system to have learning capabilities and adapt to the wear and tear of different devices, Iterative updates will be performed according to the following rules:

[0049]

[0050] in, For adaptive gain coefficients; This is the gain coefficient from the previous round; This is the learning rate constant, which, for example, can take a value of 0.1; This represents the residual error after the previous round of adjustments. This is the maximum allowable error limit for the system.

[0051] In this way, by integrating geometric principles, dynamic coupling index and gain coefficient with self-learning capability, highly accurate feedforward compensation can be calculated. This not only achieves the decoupling effect of adjusting angle without changing position, but also continuously optimizes parameters as the equipment operates, effectively coping with the uncertainties brought about by equipment wear and environmental changes.

[0052] Step S4: Perform closed-loop fusion control and execution, fuse the feedforward compensation amount with the original angle adjustment command, generate the final drive command and control the device to execute it until the optical parameters meet the preset requirements.

[0053] The position compensation amount calculated in step S3 is fused with the original motion control command to generate the final drive command sent to the six-axis robot. Specifically, the X-axis compensation component in the feedforward compensation amount is superimposed on the target position X adjustment amount to generate the final X-axis command; the Y-axis compensation component in the feedforward compensation amount is superimposed on the target position Y adjustment amount to generate the final Y-axis command; the U, V, Z, and T-axis commands retain their original calculated values.

[0054] After executing the compound command, the robot acquires images again, recalculates the error, stores the data in the history window, and enters the next cycle until all optical parameters meet the specifications.

[0055] like Figure 2 The figure shows a comparison of deviation convergence in the active alignment process of a binocular camera generated based on the method of this invention. This is a two-dimensional line graph, with the horizontal axis representing the "adjustment execution rounds" and the vertical axis representing the "center position deviation" of the binocular camera. In this graph, the center position deviation value of 5.0 is the "production qualification standard line." The data trend curve of the "existing conventional alignment method" exhibits significant sawtooth-like oscillations, with the deviation value decreasing slowly and showing a significant rebound (increased value) in the 3rd and 5th rounds, indicating that the positional accuracy was severely affected during angle adjustment, only barely dropping to near the qualification line around the 9th round. In contrast, the data trend curve of the "adaptive decoupling method of this invention" shows a precipitous drop in the 2nd round, with the deviation value rapidly decreasing from a high level to near the qualification line, and then smoothly entering below the qualification line (value less than 5.0) in the 3rd round. Subsequent rounds remain highly stable, with no rebound oscillations throughout, clearly demonstrating the significant effect of this invention in eliminating coupling oscillations and shortening the adjustment rounds.

[0056] In this way, by organically integrating and executing the feedforward compensation command with the original control command, the closed-loop control of active alignment is completed. This not only effectively eliminates coupling interference during multi-degree-of-freedom adjustment, but also significantly improves the production cycle through rapid convergence characteristics, ensuring the high-precision assembly quality of the binocular camera module of the XR equipment.

[0057] In the description of this specification, "multiple" means at least two, such as two, three or more, etc., unless otherwise expressly and specifically defined.

[0058] While various embodiments of the invention have been shown and described in this specification, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention.

Claims

1. A method for adjusting the position of a binocular camera in a binocular active alignment device for XR equipment, characterized in that, include: The system acquires multidimensional pose coupling data streams during the adjustment process, monitors and synchronously collects the motion state of the six-axis robot and the feedback data from the vision system in real time, and performs preprocessing. A pose coupling response index is constructed. Using preprocessed data, combined with historical adjustment records and the current planned adjustment amount, the nonlinear coupling strength under the current mechanical state is evaluated. The pose coupling response index is constructed to satisfy the following expression: For the current moment The pose coupling response index; The observation window length for historical data; For summation loop variables; For the first time in history The Euclidean position drift caused by wheel adjustment; For the first time in history The modulus of the angle adjustment command issued by the wheel; This is the minimum action threshold constant; For the current moment The planned angle adjustment command module length; The standard deviation of the angle adjustment within the historical window; It is an exponential function with the natural constant e as its base; The feedforward compensation amount is calculated based on adaptive gain. According to the pose coupling response index and the relationship of basic geometric transformation, combined with the dynamically updated adaptive gain coefficient, the reverse position compensation command that needs to be superimposed is calculated. Closed-loop fusion control and execution are performed, which fuses the feedforward compensation amount with the original angle adjustment command to generate the final drive command and control the device to execute it until the optical parameters meet the preset requirements; Perform closed-loop fusion control and execution, including: The X-axis compensation component in the feedforward compensation is superimposed on the target position X adjustment to generate the final X-axis command. The Y-axis compensation component in the feedforward compensation amount is superimposed on the target position Y adjustment amount to generate the final Y-axis command; Keep the original angle adjustment command unchanged; The robot executes the final instructions and collects images again to calculate the error. The data is then stored in the history window to enter the next cycle.

2. The binocular camera positioning method for a binocular active alignment device in an XR device according to claim 1, characterized in that, The acquisition of the multidimensional pose coupling data stream during the adjustment process includes: Establish a time synchronization mechanism to align the command issuance time of the motion controller with the image capture time of the vision acquisition card; Build a sliding window to store the most recently used windows. Historical status data for each adjustment cycle, including control command data and visual feedback data; The control command data records the angle adjustment amount issued by the system in each round, and the visual feedback data records the actual offset of the image center on the XY plane after each round of adjustment is completed.

3. The binocular camera positioning method for a binocular active alignment device in an XR device according to claim 2, characterized in that, The control command data includes the increment of rotation around the X-axis and the increment of rotation around the Y-axis; the visual feedback data is obtained by image processing algorithms such as the centroid method or template matching.

4. The binocular camera positioning method for a binocular active alignment device in XR equipment according to claim 2, characterized in that, After acquiring the visual feedback data, the collected sequence is subjected to a weighted moving average filter to eliminate image noise and high-frequency interference caused by mechanical vibrations.

5. The binocular camera positioning method for a binocular active alignment device in an XR device according to claim 1, characterized in that, The feedforward compensation amount calculated based on adaptive gain satisfies the following expression: in, For the current moment The XY-axis feedforward position compensation vectors that need to be superimposed; For adaptive prediction gain coefficients; The pose coupling response index; The basic geometric transformation matrix; This is the angle command vector to be issued in the current plan.

6. The binocular camera positioning method for a binocular active alignment device in an XR device according to claim 5, characterized in that, The fundamental geometric transformation matrix is ​​determined by the physical distance from the optomechanical rotation center to the imaging plane.

7. The binocular camera positioning method for a binocular active alignment device in an XR device according to claim 5, characterized in that, The update logic for the adaptive gain coefficients satisfies the following expression: in, For adaptive gain coefficients; This is the gain coefficient from the previous round; The learning rate is a constant; This represents the residual error after the previous round of adjustments. This is the maximum allowable error limit for the system.

8. The binocular camera positioning method for a binocular active alignment device in an XR device according to claim 2, characterized in that, The length of the sliding window The value ranges from 5 to 10.

Images