Image processing method for AR / VR active alignment

Abstract

The invention belongs to the technical field of image processing, and particularly relates to an image processing method for AR / VR active alignment, and the method comprises the steps: obtaining an optical collection image containing a composite test pattern, wherein the RGB three channels respectively bear a high-frequency Siemens star map used for detecting MTF and optical axis inclination, a sparse spot grid used for detecting distortion and a field angle, and a gray color block used for detecting brightness uniformity and color deviation; performing channel decomposition and spatial correction on the image based on pre-calibrated spectrum geometric parameters to obtain an independent characteristic spectrum; solving multi-dimensional optical parameters in parallel and constructing a global quality score; and finally, the six-axis robot is driven to execute closed-loop iteration adjustment based on the score until the optimal pose is achieved. Through the spectrum multiplexing and space coding technology, single-frame full-parameter detection is achieved, and the problems that traditional serial detection is low in efficiency and the precision is affected by mechanical vibration are solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology. More specifically, this invention relates to an image processing method for active alignment in AR / VR. Background Technology

[0002] In the assembly and production of AR / VR devices, in order to ensure the final imaging quality, it is usually necessary to use a six-axis robot to hold the optomechanical module and adjust its spatial pose relative to the optical elements in real time according to the imaging quality feedback from the image sensor until the optimal state is achieved.

[0003] To achieve the aforementioned precise adjustments, existing technologies typically employ image analysis-based feedback control algorithms. The main approach involves using a serial detection mode, where the optical engine under test projects different functional test patterns sequentially. For example, a black-and-white striped pattern is first projected to calculate sharpness using a modulation transfer function algorithm; then a checkerboard pattern is projected to calculate geometric distortion; and finally, a solid color image is projected to detect brightness uniformity and color deviation. The system constructs an evaluation function based on these step-by-step acquired optical indicators, driving the robot to perform iterative adjustments.

[0004] However, this serial step-by-step image processing method has significant drawbacks in practical applications: on the one hand, the switching and stabilization between different test patterns requires a long buffer time, resulting in excessively long alignment time and limiting the production cycle; on the other hand, during multiple step-by-step acquisitions, the mechanical platform inevitably experiences slight vibrations, causing key geometric features such as the center of sharpness and the center of distortion acquired at different times to fail to align precisely in the same coordinate system, thereby reducing the accuracy of multi-dimensional index fusion calculation and the final binocular image combining accuracy. Summary of the Invention

[0005] To address the aforementioned technical problem of poor active alignment performance, this invention provides an image processing method for active alignment in AR / VR, comprising: An optical image containing a composite test pattern is acquired. This composite test pattern comprises RGB projection source images carrying different test patterns in three orthogonal color channels (R, G, and B). The R channel carries a high-frequency Siemens star pattern for detecting MTF and optical axis tilt; the G channel carries a sparse speckle grid for detecting distortion and field of view; and the B channel carries regional stepped grayscale patches for detecting brightness uniformity and color deviation. Based on the optical image and pre-calibrated spectral geometric parameters, channel decomposition and spatial correction are performed to obtain independent feature maps for the RGB three channels. The correction includes spectral demixing using a pre-calibrated color crosstalk matrix and geometric alignment according to a preset color difference model. Based on the independent feature maps of the RGB three channels, the optomechanical tilt angle, modulation transfer function, geometric distortion, field of view, brightness uniformity, and color deviation are calculated in parallel. A global quality score is constructed based on the calculated multidimensional optical parameters, and a six-axis robot is driven to perform closed-loop iterative adjustments until the optimal imaging pose is achieved.

[0006] This invention achieves full parameter coverage in a single frame by carrying MTF detection, distortion detection, and color detection patterns in three orthogonal RGB color channels respectively. This spectral multiplexing and parallel processing technology not only eliminates the buffer time required for switching test patterns in traditional methods, improving production cycle time, but also avoids the problem of inconsistent coordinates between the MTF center and distortion center caused by mechanical micro-vibrations during multiple acquisitions, ensuring accurate fusion and closed-loop control of multi-dimensional optical indicators at the same physical moment and in the same coordinate system.

[0007] Preferably, the method for constructing the composite test pattern further includes: Spatial coding is introduced at specific locations in a sparse speckle grid, replacing standard specks with preset special topological graphics as absolute field-of-view anchor points.

[0008] Preferably, the calculation of the optical-mechanical tilt angle includes: Extract the blurred kernel contour of the high-frequency Siemens star map; perform ellipse fitting on the blurred kernel contour to obtain the major axis, minor axis and rotation angle of the fitted ellipse; the amplitude of the optomechanical tilt angle is proportional to the natural logarithm of the ratio of the major axis to the minor axis of the fitted ellipse, and its direction angle is orthogonal to the rotation angle of the major axis of the fitted ellipse.

[0009] This invention utilizes the astigmatism principle of Siemens star map in defocused state. By fitting the elliptical features of the blur kernel contour, it directly linearly maps the degree of blur of the image to the physical tilt angle and direction of the optomechanical system, simplifying the detection process and improving alignment efficiency.

[0010] Preferably, the solution of the modulation transfer function includes: A target spatial frequency is preset, and a sampling circle corresponding to the target spatial frequency is constructed; a brightness sequence is extracted along the sampling circle; the brightness sequence is divided into multiple independent modulation units, and the average modulation contrast of all modulation units is calculated as the modulation transfer function.

[0011] This invention constructs a sampling circle corresponding to the sensor's half-Nyquist frequency on a Siemens star map and divides the brightness sequence into independent modulation units to calculate contrast, which can reflect the true performance of the optical system at the limit resolution and provide a highly sensitive sharpness feedback index for active alignment.

[0012] Preferably, the solution of the geometric distortion includes: The centroid coordinates of all spots in the sparse speckle grid are detected; a physical coordinate system is established based on the absolute field of view anchor point, and the actual imaging height of the 1.0 field of view anchor point is obtained; the geometric distortion is equal to the normalized deviation between the actual imaging height and the theoretical imaging height.

[0013] This invention combines a physical coordinate system established by an absolute field-of-view anchor point to compare the actual imaging height of the spot with the theoretical design height, thereby calculating the normalized geometric distortion rate. This enables the system to distinguish between the inherent distortion of the optical system and assembly errors, providing accurate geometric performance data for subsequent weighted scoring.

[0014] Preferably, obtaining the pre-calibrated color crosstalk matrix includes: The optical engine under test is controlled to project pure red, pure green and pure blue images in sequence; a binocular bionic camera is used to acquire images corresponding to each pure color image, and the average grayscale response value of the central region of the image on the camera's original RGB channels is calculated; the three-channel response values ​​corresponding to each pure color image are normalized and used as a column of the color crosstalk matrix.

[0015] Preferably, the geometric alignment based on the preset color difference model includes: Using the independent feature map of channel G as the geometric reference, radial scaling is performed on the independent feature maps of channels R and B, wherein the radial scaling satisfies the expression: ,in, To correct the distance from the pixel to the center, To correct the distance before, , These are the second-order radial distortion correction coefficients and the fourth-order radial distortion correction coefficients.

[0016] Preferably, the establishment of the global quality score includes: The modulation transfer function is used as a positive gain term, and the optomechanical tilt angle, geometric distortion, and brightness non-uniformity are used as negative penalty terms, and then a linear weighted sum is performed.

[0017] Preferably, the global quality score satisfies the expression: ; In the formula, This represents the global quality score under the current optomechanical pose. Represents the normalized modulation transfer function; Indicates the normalized optical engine tilt angle; This represents the normalized TV distortion rate; Indicates normalized brightness uniformity; , , , They represent , , , The weight coefficient of the item; The sign representing the magnitude of a vector; Represents the absolute value symbol.

[0018] Preferably, the driving of the six-axis robot to perform closed-loop iterative adjustments includes: Calculate the partial derivatives of the global quality score with respect to the six degrees of freedom of the six-axis robot, and construct the pose adjustment vector using an adaptive gain matrix; control the six-axis robot to make micro-adjustments according to the direction and step size of the pose adjustment vector; the adaptive gain matrix consists of position step size coefficients and attitude step size coefficients.

[0019] The beneficial effects of this invention are as follows: By carrying MTF, distortion and color test patterns in the RGB three channels respectively, and combining channel demixing and geometric correction technology, this invention can simultaneously acquire multi-dimensional optical indicators in a single exposure, overcoming the problems of low efficiency of traditional serial detection and inconsistent coordinates of multiple indicators caused by mechanical vibration. By constructing a global quality score to drive robot closed-loop iteration, the efficiency and accuracy of AR / VR optical engine active alignment are improved. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating an image processing method for active alignment in AR / VR according to the present invention; Figure 2 This is a schematic illustration of optically acquired images; Figure 3 This is a schematic diagram illustrating the iterative change curve of the global quality score. Detailed Implementation

[0021] This invention discloses an image processing method for active alignment in AR / VR, referring to... Figure 1This includes steps S1-S4: S1: Based on the multi-dimensional detection requirements of the active alignment process and the spectral characteristics of the display unit, a full-parameter composite test pattern is constructed, and the optomechanical projection is controlled to obtain optically acquired images.

[0022] It should be noted that in the AA (Active Alignment) process of AR / VR optomechanical modules, the six-axis robot needs to adjust the optomechanical pose in real time according to the image quality. Traditional detection schemes use a serial mode, projecting multiple single-function patterns in sequence, such as MTF stripe patterns for detecting sharpness, checkerboard patterns for detecting distortion, and RGB color block patterns for detecting color. In this way, the switching between different patterns requires a certain buffer time, usually 15 to 30 seconds, which limits the production cycle; and the unavoidable mechanical micro-vibrations during multiple acquisitions can cause key geometric features such as the MTF center and the distortion center to not be accurately aligned in the same coordinate system, affecting the accuracy of binocular image merging. Therefore, this invention uses the independent driving capability of the RGB channels of the display unit, and through spectral multiplexing and spatial coding technology, fuses the features of all detection schemes into a single frame image, achieving full parameter coverage with a single projection.

[0023] Specifically, based on the multi-dimensional detection requirements of the active alignment process and the spectral characteristics of the display unit, a full-parameter composite test pattern is constructed, and optomechanical projection is controlled to acquire optical images, including: An RGB projection source image is generated, with its three orthogonal color channels carrying test patterns with different frequency domain characteristics. The R channel is a high-frequency Siemens star map, carrying MTF (Modulation Transfer Function) and Tilt information (Optical Axis Tilt). The G channel is a sparse speckle grid, carrying distortion and FOV (Field of View) information. The B channel is a regional stepped grayscale patch distributed in the gap between the R channel pattern and the G channel pattern, used to detect the brightness uniformity and color deviation of the entire field of view, avoiding background interference to high-frequency feature extraction.

[0024] Preferably, at specific locations in the sparse speckle grid, such as the center point, the 0.5 field of view, and the 1.0 field of view, spatial coding is introduced to replace the standard dots with special topological graphics, such as crosses, as absolute field of view anchor points, replacing the physical multi-field calibration process.

[0025] It should be noted that the high-frequency Siemens star map consists of black and white stripes radiating outward from the center, containing high-frequency edge information in all directions, and is highly sensitive to defocusing, and is used to achieve static titrating detection.

[0026] The optical engine under test is controlled to light up, projecting the RGB projection source image; after the light is transmitted through the optical system, it is collected by a binocular bionic camera to obtain an optical image. The optical system includes lenses and optical waveguides.

[0027] It should be noted that, as Figure 2 This is an optically acquired image, obtained by transmitting the fully parametric composite test pattern projected by the optomechanical system under test through an optical system. The image contains mixed information from the three RGB channels, and is visually presented as an overlay of a Siemens star map, a sparse speckled grid, and regional color patches.

[0028] Thus, optical images containing full-dimensional optical testing information were acquired.

[0029] S2: Based on the optically acquired images and pre-calibrated spectral geometric parameters, perform channel decomposition and spatial correction to obtain independent feature maps of the RGB three channels.

[0030] It should be noted that AR / VR optical systems, such as the Pancake folding optical path, generally suffer from chromatic aberration, resulting in different magnifications of light of different wavelengths on the imaging plane. Furthermore, display pixels exhibit spectral leakage, meaning that green pixels emit a small amount of red light. Therefore, directly separating the RGB channels leads to spatial misalignment between the red MTF texture and the green distortion points, resulting in impure signals and affecting algorithm accuracy. Therefore, this invention establishes a spectral geometric mapping model before feature extraction. First, spectral crosstalk is eliminated through matrix transformation, and then the geometric coordinates are aligned through a radial scaling model. Based on this, this invention constructs a spectral geometric decomposition model.

[0031] Specifically, based on optically acquired images and pre-calibrated spectral geometric parameters, channel decomposition and spatial correction are performed to obtain independent feature maps, including: Perform spectral demixing to obtain a clean spectral signal.

[0032] It should be noted that in order to recover the true channel signal from the mixed sensor response, the inverse transform of the color correction matrix needs to be applied. The RGB values ​​acquired by the camera sensor are the result of the convolution of the display spectrum and the Bayer filter response, which involves aliasing. Assuming that the signal aliasing is linear, the mixed signal can be demixed into independent signals using the inverse of a pre-calibrated color crosstalk matrix.

[0033] The optically acquired images are denoted as , , , , yes The RGB three-channel pixel value vector, It is the transpose symbol.

[0034] Control the optical engine under test to project pure red images sequentially. Pure green screen And a pure blue screen For each projection, images are acquired using a binocular bionic camera, and the average grayscale response value of the central image region on the camera's original RGB channels is calculated; a color crosstalk matrix is ​​then constructed. The first column represents the normalized response values ​​of the camera's RGB channels when projecting a pure red image. The second column shows the normalized response values ​​when projecting a pure green image. The third column represents the normalized response value when projecting a pure blue image. , It should be noted that the color crosstalk matrix is ​​used to eliminate signal leakage between channels.

[0035] Spectral unmixing satisfies the expression: ; In the formula, Indicates an optically demixed image; Represents the color crosstalk matrix; Indicates the dot product symbol; This indicates an optically acquired image.

[0036] In the formula, By using linear algebraic transformation, physical crosstalk such as red photons from the display screen exciting green pixels in the camera is mathematically eliminated, ensuring that the MTF calculation in the red channel will not be affected by the background interference of the distorted grid pattern in the green channel.

[0037] Will Recorded as , , , yes The RGB three-channel pixel value vector, for , , Performing LCA geometric correction yields clean and aligned RGB three-channel independent feature maps, denoted as follows: , , For example, using a green channel. Using the geometric reference, and based on the chromatic aberration curve of the optical design, for red... and blue Channel radial scaling This ensures that its geometric features are strictly aligned spatially with the green corridor, whereby... To correct the distance from the pixel to the center, To correct the distance before, , These are the second-order radial distortion correction coefficients and the fourth-order radial distortion correction coefficients, for example. Set as This indicates that a contraction correction is needed. Set as , indicating edge high-order fine-tuning.

[0038] Thus, three spatially aligned and spectrally pure independent feature maps were obtained. , , .

[0039] S3: Based on the independent feature maps of the RGB three channels, astigmatism analysis, absolute coordinate addressing and photometric statistics are performed respectively to obtain multi-dimensional optical parameters such as optomechanical tilt angle, distortion rate and brightness uniformity.

[0040] It should be noted that in the active alignment process of AR / VR optomechanical modules, traditional parameter calculation methods often rely on step-by-step physical measurements. For example, detecting optomechanical tilt depends on mechanical scanning of a physical collimator or acquisition of multiple frames of defocused images. This method is not only time-consuming but also cannot meet production cycle requirements. Furthermore, due to the large field of view of AR / VR optomechanical modules, distortion measurements at the edges of the field of view often suffer from calculation errors because an accurate central reference cannot be found. This invention utilizes the physical characteristics of independent feature maps in the RGB three channels and employs a parallel computation strategy. Using the high-frequency Siemens star map of the red channel, based on the principle of optical astigmatism—that is, when the optical system is tilted, the meridional and sagittal rays in the off-axis field of view separate at their focal points, causing the circular blurry spot at the center of the star map to degenerate into an ellipse—single-frame static tilt detection is achieved. The absolute field of view anchor point of the green channel is used to establish an absolute physical coordinate system with zero geometric error. The regional grayscale information of the blue channel is used to evaluate the photometric characteristics of the system.

[0041] Specifically, based on the independent feature map of the red channel, the optomechanical tilt angle and modulation transfer function are calculated, including: right Edge detection is performed to extract the blurred kernel contour of the high-frequency Siemens star map at the center of the image; the blurred kernel contour is then fitted with an ellipse using the least squares method to obtain the major and minor axis parameters of the fitted ellipse. For example, the edge detection uses the Canny operator.

[0042] It should be noted that the accuracy of the optical-mechanical tilt angle calculation is highly dependent on the tilt sensitivity coefficient, an inherent physical quantity determined by the lens's F-number and aberration characteristics.

[0043] The tilt sensitivity coefficient is obtained in advance through an optical simulation calibration process: Based on the optical design file of the optomechanical device under test, such as Zemax or CodeV data, an ideal optical simulation model is constructed; discrete known tilt angle variables are introduced into the model, for example, in steps of 0.1°, set from 0° to 2.0°; for each tilt angle, the point spread function generated at the center of the imaging surface is simulated, and the major-minor axis ratio of the PSF spot is measured. The natural logarithm of the major-minor axis ratio is used as the abscissa, and the known tilt angle is used as the ordinate. Linear regression analysis is performed, and the slope of the regression line is extracted. This slope is defined as the tilt sensitivity coefficient.

[0044] It should be noted that when the optical axis is perpendicular to the imaging plane, the defocused blur spot is a perfect circle; when the optical axis is tilted, astigmatism causes the blur spot to stretch into an ellipse. Considering that the ratio of the major and minor axes of the ellipse is positively correlated with the magnitude of the tilt angle, and the rotation angle of the major axis of the ellipse is orthogonal to the tilt direction angle, this invention establishes an expression for calculating the optical axis tilt angle vector.

[0045] The optical-mechanical tilt angle satisfies the expression: ; In the formula, Indicates the tilt angle of the optical engine; This indicates the magnitude of the tilt of the optical axis of the optical engine relative to the ideal normal; , Indicates the major and minor axes of the fitted ellipse; Indicates the azimuth angle of the tilt direction; This represents the rotation angle of the major axis of the fitted ellipse relative to the horizontal axis of the image; This represents the tilt sensitivity coefficient; This represents the natural logarithm function.

[0046] In the formula, Linearly mapping the degree of astigmatism to physical angles ensures that when The physical fact that the tilt angle is 0 at that time; Single-frame static tilt detection is achieved through astigmatism characteristics, eliminating the need for mechanical scanning.

[0047] It should be noted that the Siemens star chart features alternating black and white stripes radiating outwards from the center. The physical width of the stripes on the circumference increases with the radius, implying that the spatial frequency decreases as the radius increases. Therefore, each radius value on the image corresponds to a specific spatial frequency. To evaluate the performance of the optical system at the limit of resolution, it is necessary to accurately locate the specific radius position corresponding to the target spatial frequency.

[0048] Based on the physical properties of the camera sensor, the sampling radius is calculated as follows: multiply the target spatial frequency by twice pi (referred to as the first product), and divide the total number of black and white fringe pairs in the Siemens star map by the first product to obtain the sampling radius. The target spatial frequency is the half-Nyquist frequency of the sensor.

[0049] A circular brightness sequence is constructed with the center of the Siemens star map as the center and the sampling radius as the radius. The circular brightness sequence is divided into K independent modulation units using the zero-crossing detection method. Each modulation unit contains a complete "bright-dark" change cycle.

[0050] The modulation transfer function satisfies the expression: ; In the formula, Represents the modulation transfer function; This represents the total number of complete modulation units identified from the circumferential brightness sequence; , The values ​​represent the peak and trough values ​​of the luminance waveform within the i-th modulation unit.

[0051] Thus, the optomechanical tilt angle and modulation transfer function were obtained.

[0052] Preferably, based on the green channel independent feature map, the geometric distortion and field of view are calculated, including: right The sparse speckled mesh in the image is binarized and connected component analysis is performed to identify the centroid coordinates of all specks.

[0053] Perform absolute coordinate addressing and use a template matching algorithm to search for absolute field-of-view anchor points in the sparse speckle mesh; set the centroid coordinates of the identified anchor points at their center positions as follows. The physical zero point of the pixel coordinate system; obtain the anchor point position at the 1.0 field of view, denoted as the field of view angle.

[0054] It should be noted that distortion reflects the degree of deviation between the actual imaging height and the ideal optical imaging height.

[0055] Therefore, the TV distortion rate satisfies the expression: ; In the formula, Indicates the distortion rate; This represents the Euclidean distance from the centroid of the field anchor point located at the 1.0 field of view to the physical zero point in the independent feature map of the green channel; This indicates the theoretical imaging height.

[0056] In the formula, This represents the normalized relative deviation ratio of the actual imaging height to the ideal imaging height. This indicates that the relative deviation ratio is converted into a percentage.

[0057] It should be noted that, The acquisition includes multiplying the ratio of the design effective focal length specified in the optical engine specification to the pixel size of the image sensor and the tangent function value of the field of view to obtain the theoretical imaging height.

[0058] At this point, the distortion rate and field of view were obtained.

[0059] Preferably, based on the independent feature map of the blue channel, the brightness uniformity and color deviation are calculated, including: Extract using a pre-set binarization mask. The average gray value of each gray block in the stepped gray area distributed in the gaps is statistically analyzed to calculate the brightness uniformity of the entire field of view; at the same time, the difference between the gray area and the standard color card value is calculated to calculate the RGB color deviation.

[0060] It should be noted that, since the B channel pattern is located in the R / G channel gap, and combined with spectral demixing, the color data calculated here is not affected by high-frequency stripes, ensuring the accuracy of color calibration.

[0061] Thus, the brightness uniformity and color deviation were obtained.

[0062] S4: Based on the multidimensional optical parameters, a global weighted evaluation function is constructed to drive the six-axis robot to perform closed-loop iterative adjustment until the optimal imaging pose is achieved.

[0063] It should be noted that in the AA process of AR / VR optomechanical modules, there are often physical couplings and constraints between various optical parameters. For example, excessively pursuing the sharpness of the central field of view may lead to a sharp increase in astigmatism in the peripheral field of view, or sacrificing brightness uniformity to correct geometric distortion. Therefore, traditional single-parameter feedback adjustment can easily cause the system to get stuck in a local optimum. This invention introduces a multi-objective fusion strategy, normalizing multi-dimensional parameters into a comprehensive score to guide the robot in finding the globally optimal balance point.

[0064] Specifically, based on the aforementioned multidimensional optical parameters, a globally weighted evaluation function is constructed to drive the six-axis robot to perform closed-loop iterative adjustments, including: It should be noted that for an ideal optical system, its imaging quality is jointly determined by "positive gain attributes" and "negative loss attributes." The modulation transfer function represents sharpness, which is a positive attribute that should be maximized; while optomechanical tilt, distortion, and brightness unevenness represent aberrations and defects, which are negative attributes that should be minimized. To transform this multi-objective optimization problem into a single-objective optimization task that a six-axis robot can perform, a linear superposition model needs to be established, treating positive attributes as bonuses and negative attributes as deductions. Considering that different optical products have different sensitivities to these indicators—for example, VR devices prioritize center sharpness, while waveguide AR devices value geometric distortion more—weighting coefficients are introduced. By adjusting the magnitude of these weights, the relative importance of each physical quantity in the evaluation system is defined.

[0065] Record the current optical engine pose as , representing the three-dimensional coordinates and three-dimensional rotation angle of the six-axis robot at the current moment.

[0066] The global quality score satisfies the expression: ; In the formula, This represents the global quality score under the current optomechanical pose. Represents the normalized modulation transfer function; Indicates the normalized optical engine tilt angle; This represents the normalized TV distortion rate; Indicates normalized brightness uniformity; , , , They represent , , , The weight coefficient of the item; The sign representing the magnitude of a vector; This represents the absolute value symbol. It should be noted that... This is a positive gain term. This is a negative penalty item. This is a negative penalty item. This is a negative penalty item.

[0067] In the formula, , , , All values ​​are greater than 0. This is based on the pre-set sensitivity of each indicator according to the product specifications. For example, for VR display devices that emphasize center image quality, given their high requirements for sharpness and optical axis parallelism, and the characteristic that distortion can be corrected secondaryly through software algorithms, Set as , Set as , Set it to 0.1. Set it to 0.1.

[0068] It should be noted that a six-axis robot has six degrees of freedom. This, along with the global quality score, forms a high-dimensional nonlinear surface. The goal of the AA process is to find the global highest point on this surface. To efficiently locate this highest point and avoid blind random trials, this invention employs the gradient ascent method to calculate... The partial derivative with respect to each degree of freedom of motion has the physical meaning of sensitivity, indicating the direction and distance that will cause the fastest increase in score. Based on this sensitivity, pose adjustment commands that include direction and step size are constructed.

[0069] The pose adjustment vector satisfies the expression: ; ; In the formula, This represents the single-step feed vector of a six-axis robot. Represents the adaptive gain matrix; Indicates the transpose symbol; Represents a diagonal matrix; This represents the position step size coefficient. This represents the attitude step size coefficient.

[0070] It should be noted that, Including spatial location and rotation angle The adjustment amount; the adaptive gain matrix is ​​used to control the adjustment step size, taking a large value in the early stage of the iteration to achieve fast convergence, and a small value in the later stage to avoid oscillation; Used to adjust spatial displacement The convergence speed, Used to adjust the rotation angle The convergence speed. By setting them separately. and It can specifically balance the adjustment range of translation and rotation, eliminate convergence problems caused by inconsistent dimensions, and ensure that the robot smoothly approaches the optimal pose in six-dimensional space.

[0071] In the formula, This represents the global quality score, where each item indicates the rate of change of the optical quality score when the robot makes a small displacement in the corresponding degree of freedom, indicating the ability to... The fastest growing movement direction.

[0072] Execute closed-loop iteration and termination judgment: Determine whether the global quality score under the current optomechanical pose meets the termination condition. ,in The target threshold is set as a preset threshold, or the change in score over three consecutive iterations. ,in If the convergence threshold is met, the AA process is considered successful; otherwise, pose adjustment is performed, controlling the six-axis robot to follow the specified path. Perform fine-tuning; after adjustment, return to reproject and acquire images, then sequentially acquire RGB independent feature maps and multidimensional optical parameters, and recalculate. And proceed to the next round of judgment.

[0073] It should be noted that, as Figure 3 This diagram illustrates the iterative change curve of the global quality score. The horizontal axis represents the number of adjustment iterations, and the vertical axis represents the normalized comprehensive score. The curve shows a clear upward trend, indicating that the six-axis robot continuously corrects the optomechanical position based on the pose adjustment vector calculated by the gradient ascent method, maximizing the positive gain attribute and minimizing the negative loss attribute, ultimately converging to the global optimal equilibrium point.

[0074] Thus, the active alignment and testing of the optomechanical module based on the full-parameter composite test pattern was completed.

[0075] While various embodiments of the invention have been shown and described in this specification, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention.

Claims

1. An image processing method for AR / VR active alignment, characterized in that, The method comprises the following steps: acquiring an optical acquisition image containing a composite test image, the composite test image containing RGB projection source images respectively carrying different test patterns in three orthogonal color channels R, G and B, wherein the R channel carries a high-frequency Siemens star pattern for detecting MTF and optical axis tilt, the G channel carries a sparse spot grid for detecting distortion and field of view angle, and the B channel carries a regional step grayscale color block for detecting brightness uniformity and color deviation; performing channel decomposition and spatial correction based on the optical acquisition image and pre-labeled spectral geometric parameters to obtain independent feature maps of the RGB three channels, wherein the correction includes spectral unmixing using a pre-labeled color crosstalk matrix and geometric alignment according to a pre-set color difference model; respectively based on the independent feature maps of the RGB three channels, parallel solving of optical and mechanical tilt angle, modulation transfer function, geometric distortion, field of view angle, brightness uniformity and color deviation; and based on the solved multi-dimensional optical parameters, constructing a global quality score and driving a six-axis robot to perform closed-loop iterative adjustment until an optimal imaging pose is reached.

2. The image processing method for AR / VR active alignment according to claim 1, wherein, The method for constructing the composite test image further comprises: introducing spatial coding at specific positions of the sparse spot grid, replacing standard spots with a pre-set special topological pattern as an absolute field of view anchor.

3. The image processing method for AR / VR active alignment according to claim 1, wherein, The method for solving the optical and mechanical tilt angle comprises: extracting a blur kernel profile of the high-frequency Siemens star pattern; performing ellipse fitting on the blur kernel profile to obtain a long axis, a short axis and a rotation angle of the fitted ellipse; the amplitude of the optical and mechanical tilt angle is directly proportional to the natural logarithm of the ratio of the long axis to the short axis of the fitted ellipse, and the direction angle is orthogonal to the rotation angle of the long axis of the fitted ellipse.

4. The image processing method for AR / VR active alignment according to claim 1, wherein, The method for solving the modulation transfer function comprises: pre-setting a target spatial frequency and constructing a sampling circumference corresponding to the target spatial frequency; extracting a brightness sequence along the sampling circumference; dividing the brightness sequence into multiple independent modulation units and calculating the average modulation contrast of all modulation units as the modulation transfer function.

5. The image processing method for AR / VR active alignment according to claim 2, wherein, The method for solving the geometric distortion comprises: detecting the centroid coordinates of all spots in the sparse spot grid; establishing a physical coordinate system based on the absolute field of view anchor and obtaining the actual imaging height of the 1.0 field of view anchor; the geometric distortion is equal to the normalized deviation between the actual imaging height and the theoretical imaging height.

6. The image processing method for AR / VR active alignment according to claim 1, wherein, The method for obtaining the pre-labeled color crosstalk matrix comprises: controlling the to-be-tested optical machine to project pure red, pure green and pure blue pictures in sequence; using a binocular biomimetic camera to acquire images corresponding to each pure color picture respectively and counting the average grayscale response values of the central region of the images on the original RGB channels of the camera; normalizing the three-channel response values corresponding to each pure color picture to obtain a column of the color crosstalk matrix.

7. The image processing method for AR / VR active alignment according to claim 1, wherein, The method for performing geometric alignment according to the pre-set color difference model comprises: With the G channel independent feature map as a geometric reference, radial scaling is performed on the independent feature maps of the R channel and the B channel, and the radial scaling satisfies an expression: wherein, is the distance from the corrected pixel point to the center, is the distance before correction, , is a second-order radial distortion correction coefficient, and a fourth-order radial distortion correction coefficient.

8. The image processing method for AR / VR active alignment according to claim 1, wherein, The method for constructing the global quality score comprises: linearly weighting and summing the modulation transfer function as a positive gain term and the optical and mechanical tilt angle, geometric distortion and brightness non-uniformity as negative penalty terms.

9. The image processing method for AR / VR active alignment according to claim 1, wherein, The global quality score satisfies the expression: ； wherein, represents the global quality score under the current optical-mechanical pose; represents the normalized modulation transfer function; represents the normalized optical-mechanical tilt angle; represents the normalized TV distortion rate; represents the normalized brightness uniformity; , , , respectively represent , , , weighting coefficients of the terms; represents the modulus of a vector; represents the absolute value symbol.

10. The image processing method for AR / VR active alignment according to claim 1, wherein, The method for driving the six-axis robot to perform closed-loop iterative adjustment comprises: The partial derivative of the global quality score with respect to six motion degrees of freedom of the six-axis robot is calculated, and an adaptive gain matrix is constructed to build a pose adjustment vector; and the six-axis robot is controlled to make a fine adjustment in the direction and step length of the pose adjustment vector; the adaptive gain matrix is composed of a position step length coefficient and an attitude step length coefficient.

Images