Active alignment method, device and electronic equipment of optical module

Abstract

The present disclosure relates to an active alignment method and device of an optical module. The method can include: in response to an alignment start instruction, sending a target feature image to the optical module to drive the optical module to project a target imaging light beam based on the target feature image; obtaining the target imaging light beam based on an image acquisition device to obtain a target feedback image; obtaining a preset reference object corresponding to the target feature image; superimposing and displaying the target feedback image and the preset reference object to obtain a visual alignment view; and forming an image-based visual alignment guiding mechanism in the active alignment device according to the technical solution provided by the present disclosure, which helps to provide a visual alignment reference and adjustment guidance for the adjustment of the relative pose, thereby improving the consistency of the alignment operation result, shortening the alignment operation time, reducing the number of repeated trial adjustments, and reducing the dependence of the alignment operation on human experience.

Description

Technical Field

[0001] This disclosure relates to the field of optical module assembly and alignment technology, and in particular to an active alignment method, apparatus and electronic device for optical modules. Background Technology

[0002] In the manufacturing and assembly process of AR (Augmented Reality) head-mounted display devices, the display optical components of reflective folded optical paths typically require imaging state adjustment during the assembly stage to ensure that the geometric position, orientation, and binocular consistency of the virtual image reach an acceptable state. Taking the Birdbath architecture as an example, this type of optical path usually includes beam splitters and reflectors, and the impact of assembly deviations on the imaging state has coupled characteristics.

[0003] In the relevant assembly and calibration process, a common practice is to output a specific pattern on the display to be assembled, and then acquire a virtual image at the designed observation position using an imaging acquisition device. This image is then combined with displacement and angle adjustment mechanisms to perform multi-dimensional adjustments to the relative positions and angles between components. Since the adjustment dimensions include translation and rotation, and the imaging evaluation target may simultaneously involve geometric position and binocular consistency, the calibration process often exhibits iterative characteristics.

[0004] In the Birdbath architecture, if there is an angular deviation between the central axis of the display and the principal optical axis of the reflective device, the virtual image plane may exhibit trapezoidal distortion or center shift. In binocular assembly scenarios, the relative deviation of the optical axes of the left and right eyes may cause binocular vertical parallax, thereby affecting the fusion comfort of wearing and viewing. In addition, since the beam splitter has a certain thickness, reflections from different surfaces may introduce secondary reflected images, forming visible ghosting.

[0005] In existing production lines, one type of solution uses automated active alignment equipment for calibration. This type of solution usually relies on image quality evaluation indicators, such as MTF (Modulation Transfer Function), for search-based or iterative adjustments. The equipment investment and maintenance requirements are relatively high, and there may be trade-offs between the evaluation indicators and the geometric alignment target. The other type of solution uses manual visual calibration. Operators need to judge whether the image meets the requirements based on visual observation. This method is highly dependent on experience, and it is difficult to quantify and verify the rotational errors and relative consistency between the left and right eyes using the same standard. Summary of the Invention

[0006] This disclosure provides an active alignment method, apparatus, and electronic device for optical modules, to at least solve the problem in related technologies of how to provide visual guidance for the assembly and alignment process of optical modules based on image acquisition results and reduce the reliance on manual experience in the alignment process. The technical solution of this disclosure is as follows: According to a first aspect of the embodiments of this specification, an active alignment method for an optical module is provided, applied to an active alignment device, the active alignment device including an optical module and an image acquisition device; the image acquisition device is disposed on the light projection path of the optical module; including: In response to the alignment start command, a target feature image is sent to the optical module to drive the optical module to project a target imaging beam based on the target feature image; Based on the image acquisition device, the target imaging beam is acquired to obtain a target feedback image; the target feedback image represents an optical virtual image corresponding to the target feature image; Obtain a preset reference object corresponding to the target feature image; the preset reference object represents the imaging position of the target feature image after transmission through the light projection path of the optical module in an ideal optical assembly state; The target feedback image is overlaid with the preset reference object to obtain a visual alignment view; Based on the visualized alignment view, pose adjustment instructions are generated to guide the adjustment of the relative pose between the optical module and the image acquisition device.

[0007] According to a second aspect of the embodiments of this specification, an active alignment device for an optical module is provided, characterized in that it includes an alignment control module, an optical module, an image acquisition device, an image display terminal, and a pose adjustment mechanism; The alignment control module is used to respond to the alignment start command by sending a target feature image to the optical module; obtaining a preset reference object corresponding to the target feature image; the preset reference object represents the imaging position of the target feature image after transmission through the light projection path of the optical module in an ideal optical assembly state; and generating a pose adjustment command based on the visualized alignment view. The optical module is used to project a target imaging beam based on the target feature image; The image acquisition device is used to acquire the target imaging beam and obtain a target feedback image; the target feedback image represents an optical virtual image corresponding to the target feature image. The image display terminal is used to overlay the target feedback image with the preset reference object to obtain the visual alignment view; The pose adjustment mechanism is used to adjust the relative pose between the optical module and the image acquisition device in response to the pose adjustment command.

[0008] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method as described in any one of the first aspects above.

[0009] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects: By sending the target feature image to the optical module in response to the alignment start command, and obtaining the target feedback image based on the target imaging beam acquired by the image acquisition device, a visual representation of the optical virtual image corresponding to the target feature image can be obtained during the alignment process, thereby providing a basis for the alignment status judgment. By acquiring the preset reference object corresponding to the target feature image, and the preset reference object representing the imaging position under ideal optical assembly state, an alignment benchmark can be provided for alignment determination, thereby improving the consistency of alignment determination aperture. By overlaying the target feedback image with a preset reference object, a visual alignment view is obtained. The alignment reference and the target feedback image can be presented in the same view, thereby improving the recognition efficiency of imaging position deviation. By generating pose adjustment instructions based on the visual alignment view to guide the adjustment of relative pose, clear guidance can be provided for the relative pose adjustment between the optical module and the image acquisition device, thereby reducing the reliance on human experience in alignment operations and improving the consistency of alignment results. By setting an image acquisition device on the light projection path of the optical module, alignment guidance can be performed based on the imaging feedback on the light projection path, thereby improving the consistency between the alignment operation and the actual imaging path.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this disclosure, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0012] Figure 1 This is a schematic diagram illustrating an application environment according to an exemplary embodiment.

[0013] Figure 2 This is a flowchart illustrating an active alignment method for an optical module according to an exemplary embodiment.

[0014] Figure 3 This is a schematic diagram illustrating the overlay of a visualization alignment view layer in a crosshair target scene according to an exemplary embodiment.

[0015] Figure 4 This is a schematic diagram illustrating the overlay of a visual alignment view layer in a checkerboard scene according to an exemplary embodiment.

[0016] Figure 5 This is a schematic diagram illustrating target projection, image acquisition, and motion control of an active alignment device according to an exemplary embodiment.

[0017] Figure 6 This is a block diagram of an active alignment device for an optical module according to an exemplary embodiment.

[0018] Figure 7 This is a block diagram illustrating an electronic device for active alignment of an optical module according to an exemplary embodiment. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0020] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0021] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating an application environment according to an exemplary embodiment. For example... Figure 1 As shown, the application environment may include an active alignment device 01, which may include an optical module 101 and an image acquisition device 102. The image acquisition device 102 may be disposed on the light projection path of the optical module 101. For example, the optical module 101 may be a Birdbath AR glasses module; the optical module 101 may include a Micro-OLED (Micro Organic Light-Emitting Diode) and an optical lens.

[0022] In an optional embodiment, the active alignment device 01 may further include a pose adjustment mechanism 103. The optical module 101 may be fixedly mounted on the pose adjustment mechanism 103 to move in tandem with it. Exemplarily, the pose adjustment mechanism 103 may be a 6-DOF stage (Six Degrees of Freedom Stage), which supports X / Y / Z translation and Pitch / Yaw / Roll rotation. Exemplarily, the pose adjustment mechanism 103 may be used to fix a microdisplay or optical prism.

[0023] In an optional embodiment, the image acquisition device 102 can be an Eye Camera (Bionic Camera). The Eye Camera can be placed at the Eye Point (designed eye point), and the Eye Camera can be configured with an entrance pupil diameter parameter, which can be 3mm to 4mm. The Eye Camera can be configured with a lens FOV (Field of View) parameter, which can be greater than the eyeglass FOV, which can be greater than 50°.

[0024] In an optional embodiment, the active alignment device 01 may further include an image display terminal 104. The image display terminal 104 may be communicatively connected to the image acquisition device 102 to display a target feedback image; the image display terminal 104 may also be communicatively connected to a data storage unit for providing a preset reference object to display the preset reference object. Exemplarily, the image display terminal 104 may display a camera preview window; the interactive interface of the image display terminal 104 may include manual adjustment button controls.

[0025] In addition, it should be noted that, Figure 1 The example shown is merely one application environment of the active alignment method of the learning module provided in this disclosure.

[0026] It should be noted that the following diagram illustrates one possible sequence of steps, and it is not strictly required to follow this order. Some steps can be performed in parallel without interdependence. The user information (including but not limited to user device information, user personal information, user behavior information, etc.) and data (including but not limited to data used for display, training data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0027] The following describes a specific embodiment of an active alignment method for an optical module disclosed in this invention. Figure 2 This is a flowchart illustrating an active alignment method for an optical module according to an exemplary embodiment. Figure 2As shown, it may include the following steps.

[0028] In step S201, in response to the alignment start command, a target feature image is sent to the optical module to drive the optical module to project a target imaging beam based on the target feature image.

[0029] In the embodiments of this specification, the alignment start command can be used to characterize the start trigger condition of the active alignment process. The triggering of the alignment start command is used to initiate the processing of sending the target feature image to the optical module.

[0030] Optical modules can be used to provide light projection paths and form imaging outputs based on target feature images.

[0031] The target feature image can be used to characterize the image content that needs to be projected by the optical module. The target feature image is used to drive the optical module to form an imaging output corresponding to the target feature image.

[0032] The target imaging beam can be used to characterize the imaging light output projected by the optical module, and the target imaging beam carries imaging information corresponding to the target feature image.

[0033] In one possible implementation, in response to an alignment start command, the image content of the target feature image can be determined, and the target feature image can be sent to the optical module in the form of image data. The target feature image may include pixel matrix data for display driving and display parameters associated with the pixel matrix data. Sending the target feature image to the optical module may include writing the target feature image into a target frame buffer, encapsulating the target feature image into an image frame data unit, and transmitting the image frame data unit to the optical module through a preset transmission interface. Based on the target feature image, the optical module can be driven to output imaging light corresponding to the target feature image, and the imaging light can be transmitted along the light projection path of the optical module to form a target imaging beam. Furthermore, the target feature image may be repeatedly sent according to a preset refresh strategy so that the target imaging beam remains in a projection state during alignment.

[0034] In one example, the target feature image can be a cross pattern image, comprising horizontal and vertical line segments, the intersection of which represents the center of the cross pattern. Exemplarily, the target feature image can be a monochrome high-contrast pattern to improve the recognizability of the cross pattern image within the target imaging beam. Exemplarily, sending the target feature image to the optical module can include sending image frame data containing a pixel matrix. The image frame data can include a frame header field and a frame body field. The frame body field carries the pixel matrix data, and the frame header field carries a resolution identifier and a frame sequence number identifier associated with the pixel matrix data.

[0035] Optionally, the alignment start command can be at least one of a button trigger signal, a trigger event signal, or an external control signal. Optionally, the target feature image can be determined by a preset image template, or the target feature image can be generated by a preset pattern rule. Optionally, the linewidth parameter, brightness parameter, or grayscale parameter of the target feature image can be configured to adapt to the imaging characteristics of the optical module and the alignment observation conditions. Optionally, the target feature image can be sent to the optical module in a single-frame transmission and display mode, or it can be sent to the optical module in a multi-frame loop transmission mode.

[0036] In one optional implementation, the target feature image can take the form of a target image; the target image can include a central cross or concentric circles to form a locatable linear or annular edge in the target feedback image. For example, the target feature image can be a white cross-shaped target pattern; the central pixel of the white cross-shaped target can correspond to the geometric center of the Micro-OLED to establish a correspondence between the center of the target feature image and the geometric center of the microdisplay inside the optical module.

[0037] In another alternative implementation, the target feature image can be an array calibration image, which can be used to form extractable array geometric features in the target feedback image. For example, the array calibration image can be a checkerboard image or a grid image, which can include feature units distributed in a row-column array, forming multiple corner feature points or multiple grid edges in the target feedback image to support the visualization and interpretation of translational and rotational deviations during alignment.

[0038] In practical applications, by sending the target feature image to the optical module after the alignment start command is triggered, and projecting the target imaging beam based on the target feature image, it helps to form a target imaging light output carrying the imaging information of the target feature image on the light projection path, thereby providing a projection basis with a clear image content correspondence for the active alignment process.

[0039] In step S203, the target imaging beam is acquired based on the image acquisition device to obtain the target feedback image; the target feedback image represents the optical virtual image corresponding to the target feature image.

[0040] In the embodiments of this specification, the image acquisition device can be an imaging acquisition apparatus used to acquire imaging information on the light projection path of the optical module. The image acquisition device can be disposed on the light projection path of the optical module so that the image acquisition device can receive the target imaging beam.

[0041] The target imaging beam can be the imaging light output formed by the optical module on the light projection path based on the target feature image. The target feedback image can be the image data obtained after the image acquisition device acquires the target imaging beam. The target feedback image is used to characterize the optical virtual image corresponding to the target feature image.

[0042] In one possible implementation, while the target imaging beam is in the projection state, the target imaging beam can be acquired using an image acquisition device, and a target feedback image can be output. This acquisition process may include imaging the optical signal of the target imaging beam and forming image data, so that the target feedback image can reflect the current imaging state of the optical virtual image. Furthermore, the target feedback image can be obtained in single-frame mode, or it can be obtained continuously in multi-frame mode to form a target feedback image sequence that updates over time.

[0043] In one example, the image acquisition device can be a camera, positioned on the light projection path of the optical module to capture an optical virtual image formed by the target imaging beam; the target feedback image can be a frame from the video stream output by the camera. For example, when the target feature image is a cross pattern image, the target feedback image can present a virtual image pattern corresponding to the cross pattern image to demonstrate the correspondence between the target feature image and the target feedback image.

[0044] Optionally, when acquiring the target imaging beam using an image acquisition device, the exposure or gain parameters of the imaging acquisition can be configured to ensure that the image contrast of the target feedback image meets the recognition requirements. Optionally, the target feedback image can be cropped or scaled to ensure that the display range of the target feedback image covers the main imaging area of ​​the optical virtual image.

[0045] In practical applications, acquiring the target imaging beam and obtaining the target feedback image through an image acquisition device helps to obtain image results for characterizing optical virtual images, thus providing a visual representation basis for subsequent processing of optical virtual images.

[0046] In one possible implementation, the image acquisition device is equipped with a photosensitive component; the image acquisition device is controlled to receive a target imaging beam located at a preset eye point position; based on the photosensitive component, the received target imaging beam is photoelectrically converted to generate a digital image signal; the digital image signal is analyzed to obtain a target feedback image.

[0047] In the embodiments described in this specification, the photosensitive component can be disposed in the image acquisition device for photoelectric conversion of the received target imaging beam.

[0048] The preset eye point position can be used to characterize the observation position of the optical module projecting a complete optical virtual image. The preset eye point position can also serve as an observation position constraint when the image acquisition device receives the target imaging beam.

[0049] Digital image signals can include grayscale or luminance data corresponding to pixel locations. Analyzing digital image signals can be used to convert them into image data of a target feedback image, so that the target feedback image can be used to characterize an optical virtual image corresponding to the target feature image.

[0050] In one possible implementation, the image acquisition device can be controlled to receive a target imaging beam located at a preset eyepoint position. The preset eyepoint position can be defined by the assembly reference of the optical module, or it can be determined by the calibration results of an active alignment device, allowing the preset eyepoint position to characterize the observation position where the optical module projects a complete optical virtual image. Furthermore, the image acquisition device can be positioned at the preset eyepoint position using a position fixing structure or a position adjustment structure, so that when the image acquisition device receives the target imaging beam at the preset eyepoint position, it can cover the imaging area of ​​the complete optical virtual image.

[0051] In one possible implementation, the received target imaging beam can be photoelectrically converted by a photosensitive component to generate a digital image signal. The digital image signal may include pixel sample data organized by rows and columns, and may also include timestamp information or frame sequence information associated with the pixel sample data to support multi-frame acquisition of the target imaging beam. Furthermore, the digital image signal can be parsed. The parsing process may include unpacking the pixel sample data, rearranging the pixels, and organizing the parsing results into an image frame structure to obtain the target feedback image.

[0052] For example, the image acquisition device can be a bionic camera, which is placed at a designed eyepoint position, which is used to characterize the observation position of the optical module projecting a complete optical virtual image; the entrance pupil diameter of the bionic camera can be 3-4mm; the FOV (Field of View) of the bionic camera lens can be greater than that of the glasses, and the lens FOV can typically be greater than 50° to improve the coverage of the complete optical virtual image.

[0053] For example, the digital image signal can be a grayscale data stream organized by a pixel matrix, and parsing the digital image signal can include converting the grayscale data stream into a pixel matrix representation of the target feedback image so that subsequent processing can use the target feedback image to characterize the optical virtual image.

[0054] Optionally, when controlling the image acquisition device to receive the target imaging beam, exposure parameters or gain parameters can be configured to ensure that the grayscale dynamic range of the digital image signal meets the resolution requirements. Optionally, after resolving the digital image signal to obtain the target feedback image, the target feedback image can be cropped or scaled to ensure that the effective display area of ​​the target feedback image covers the main imaging area of ​​the optical virtual image.

[0055] In practical applications, by receiving the target imaging beam at a preset eyepoint position and generating a target feedback image, it is helpful to ensure that the target feedback image covers the complete optical virtual image, thereby improving the consistency of the target feedback image in representing the optical virtual image and providing a more stable input basis for subsequent alignment correlation processing.

[0056] In one possible implementation, after acquiring the target imaging beam and obtaining the target feedback image using an image acquisition device, the target scanning area is determined in the target feedback image; the grayscale gradient value of each pixel within the target scanning area is calculated to obtain brightness distribution information; based on the brightness distribution information, the main brightness peak is determined; the secondary brightness peaks within a preset correlation range of the main brightness peak are determined; the ratio of the brightness values ​​between the secondary brightness peaks and the main brightness peak is calculated to obtain the peak intensity ratio; and if the peak intensity ratio is greater than a preset ghosting threshold, optical anomaly information is generated.

[0057] In the embodiments of this specification, the target scanning area may be a region determined from the target feedback image, and the target scanning area is used to cover the region in the target feedback image where the pixel grayscale value changes abruptly.

[0058] A pixel can be a sampling location in the target feedback image, and each pixel corresponds one-to-one with a pixel grayscale value. A preset scanning direction can be used to determine the direction of change in pixel grayscale values; the preset scanning direction is used to define the direction for determining abrupt changes. Abrupt changes can characterize the change in pixel grayscale value from high to low along the preset scanning direction when the gradient exceeds a preset gradient threshold; the preset gradient threshold can be used to define the threshold for determining abrupt changes.

[0059] The grayscale gradient value can be a value obtained by calculating the gradient of the pixel grayscale value of each pixel in the target scanning area along a preset scanning direction. The grayscale gradient value is used to characterize the magnitude of the change in pixel grayscale value.

[0060] Brightness distribution information can characterize the mapping relationship curve between each pixel position and the corresponding gray-level gradient value within the target scanning area. Brightness distribution information is used to reflect the curve shape of gray-level gradient value as pixel position changes.

[0061] The primary brightness peak can be the main peak in the brightness distribution information, and it is used to characterize the peak response corresponding to the main jump in the target feedback image. The secondary brightness peak can be a peak within a preset correlation range of the primary brightness peak, and it is used to characterize the secondary peak response adjacent to the primary brightness peak.

[0062] The peak intensity ratio can be the ratio of the brightness values ​​between the secondary brightness peak and the primary brightness peak. The peak intensity ratio is used to quantify the intensity ratio of the secondary brightness peak relative to the primary brightness peak.

[0063] The preset ghosting threshold can be a threshold parameter used to determine optical ghosting. Optical anomaly information can be generated when the peak intensity ratio is greater than the preset ghosting threshold. Optical anomaly information indicates that the optical module has optical ghosting that does not meet the preset imaging quality standard. The preset imaging quality standard is used to limit the imaging quality requirements on which the determination of optical ghosting is based.

[0064] In one possible implementation, the target scanning region can be determined in the target feedback image. Determining the target scanning region may include: retrieving locations in the target feedback image where pixel grayscale values ​​decrease from high to low along a preset scanning direction; identifying the set of locations where the gradient change exceeds a preset gradient threshold as a set of transition locations; and then determining the target scanning region based on the set of transition locations, such that the target scanning region covers the pixel region corresponding to the set of transition locations.

[0065] In one possible implementation, the grayscale gradient values ​​of each pixel within the target scanning area can be calculated to obtain brightness distribution information. The calculation of grayscale gradient values ​​may include: performing differential or convolution calculations on the pixel grayscale values ​​of each pixel within the target scanning area along a preset scanning direction to obtain the grayscale gradient value of each pixel; and organizing the grayscale gradient values ​​according to pixel position to obtain brightness distribution information. The brightness distribution information can be represented as a curve showing the mapping relationship between pixel position and grayscale gradient value, so as to subsequently determine the primary and secondary brightness peaks.

[0066] In one possible implementation, a primary brightness peak can be determined based on brightness distribution information, and secondary brightness peaks within a preset association range of the primary brightness peak can also be determined. Determining the primary brightness peak may include: retrieving peak points from the brightness distribution information and selecting the peak point with the largest brightness value as the primary brightness peak. The preset association range can be a pixel position window centered on the peak position of the primary brightness peak, used to limit the search range for secondary brightness peaks; other peak points within the preset association range besides the primary brightness peak can be determined as candidates for secondary brightness peaks, and the peak point with the largest brightness value among these candidates is selected as the secondary brightness peak.

[0067] In one possible implementation, the ratio of the brightness values ​​between the secondary brightness peak and the primary brightness peak can be calculated to obtain a peak intensity ratio. Optical anomaly information is generated when the peak intensity ratio is greater than a preset ghosting threshold. The calculation of the peak intensity ratio may include dividing the brightness value corresponding to the secondary brightness peak by the brightness value corresponding to the primary brightness peak. The generation of optical anomaly information may include comparing the peak intensity ratio with a preset ghosting threshold, and outputting optical anomaly information when the comparison result indicates that the peak intensity ratio is greater than the preset ghosting threshold. This allows the optical anomaly information to characterize the presence of optical ghosting in the optical module that does not meet a preset imaging quality standard.

[0068] For example, the target scanning area can be set around the high-contrast edge in the target feedback image, the preset scanning direction can be selected as the direction through the high-contrast edge, the preset gradient threshold can be used to filter out grayscale changes caused by noise, and the preset ghosting threshold can be used to distinguish whether the intensity ratio between the main brightness peak and the secondary brightness peak meets the preset imaging quality standard; in one example, the preset ghosting threshold can be 0.02 to indicate that optical ghosting exists when the brightness value of the secondary brightness peak exceeds 2% of the brightness value of the main brightness peak.

[0069] For example, the brightness distribution information can be represented in the form of a Profile, which can be used to present a curve showing the change of grayscale gradient values ​​with pixel position; the main brightness peak can correspond to the main peak in the Profile, and the secondary brightness peak can correspond to the secondary peak in the Profile. For example, when the peak intensity ratio is greater than a preset ghosting threshold, the optical anomaly information can be output as a red flashing prompt, which may include the words "Ghosting Detected" to indicate the presence of optical ghosting.

[0070] Optionally, the window width of the preset association range can be configured to adapt to the imaging magnification of different optical modules or the edge width of different target feature images. Optionally, when determining the main brightness peak and the secondary brightness peak, a peak point spacing constraint can be introduced to suppress spurious peak points caused by noise near the same main brightness peak.

[0071] In practical applications, the target scanning area is determined based on the pixel gray value jump in the target feedback image, and the main brightness peak, secondary brightness peak and peak intensity ratio are obtained based on the brightness distribution information. When the peak intensity ratio is greater than the preset ghosting threshold, optical anomaly information is generated, which helps to quantify and judge optical ghosting, thereby helping to reduce the possibility of optical ghosting not being identified and improving the consistency of recognition of optical ghosting states that do not meet the preset imaging quality standards.

[0072] In step S205, a preset reference object corresponding to the target feature image is obtained.

[0073] In the embodiments of this specification, a preset reference object can be used to characterize the imaging position under an ideal optical assembly state. The preset reference object and the target feature image have a corresponding relationship, which is used to define the association between the imaging position characterized by the preset reference object and the target feature image. An ideal optical assembly state can be used to characterize the state of the optical module under preset assembly reference conditions. The imaging position after light transmission through the optical module's projection path under the ideal optical assembly state can be used to characterize the position representation of the target feature image in the observation plane after imaging by the optical module. The imaging position can be represented using at least one of pixel coordinates, coordinate sets, bounding boxes, curve sets, or region masks. The representation of the imaging position is used to maintain the accessibility and reusability of the preset reference object.

[0074] In one possible implementation, obtaining the preset reference object corresponding to the target feature image may include: determining the image identification information of the target feature image, and retrieving matching entries from a preset reference object set based on the image identification information to obtain the preset reference object corresponding to the target feature image. The preset reference object set can be a preset stored reference object library, and each reference object entry in the preset reference object set can be associated with a target feature image type, thereby forming a correspondence between the target feature image and the preset reference object. Furthermore, the preset reference objects can be categorized and stored according to the resolution identification information of the target feature image, or they can be categorized and stored according to the pattern type identification information of the target feature image to improve the stability of the retrieval and matching.

[0075] In an optional implementation, when the target feature image is a crosshair pattern image, the preset reference object can be a reference scribing pattern. An example of the preset reference object is that the reference scribing pattern can include a reference crosshair and reference center coordinates associated with the reference crosshair. The reference center coordinates are used to characterize the imaging center position of the crosshair pattern image after transmission through the light projection path of the optical module in an ideal optical assembly state. Exemplarily, the reference scribing pattern can also include a bounding box pattern, which characterizes the imaging boundary position of the crosshair pattern image after transmission through the light projection path of the optical module in an ideal optical assembly state, thereby enabling the preset reference object to simultaneously characterize both the center position and the boundary position.

[0076] In one example, the preset reference object can be generated based on ideal optical axis coordinates, which can be taken as the center of the sensor of the image acquisition device (C_x, C_y), and used as the center reference coordinates of the preset reference object. For example, the preset reference object can be a green crosshair frame Reference Reticle (Reference Reticle, standard indicator screen); the size and position of the green crosshair frame can correspond to the image boundary of a standard qualified product to characterize the imaging position range under ideal optical assembly conditions.

[0077] In another example, the preset reference object can be a virtual reference system corresponding to the array calibration image, and the virtual reference system can be an ideal color grid. The ideal color grid can characterize the imaging position range of the array calibration image after transmission through the light projection path of the optical module in an ideal optical assembly state. For example, the center reference coordinates of the ideal color grid can be taken as the sensor center (C_x, C_y) of the image acquisition device, so that the position reference of the ideal color grid in the visualization alignment view is consistent with the ideal optical axis coordinates.

[0078] Optionally, the preset reference object can be generated based on a calibration sample. The calibration sample can be an optical module sample in an ideal optical assembly state. After receiving the target feature image and projecting imaging light, the calibration sample can form a calibration imaging result. The imaging position corresponding to the target feature image in the calibration imaging result can be extracted and recorded as the preset reference object. Optionally, the preset reference object can include multiple hierarchical versions, which are used to correspond to target feature images of different resolutions or different scaling ratios, so that the same target feature image type has a corresponding preset reference object under different display configurations.

[0079] In practical applications, by obtaining a preset reference object corresponding to the target feature image and making the preset reference object represent the imaging position under the ideal optical assembly state, it helps to provide a reusable ideal position representation for the imaging position of the target feature image, thereby improving the consistency of the understanding of the ideal imaging position between different alignment processes.

[0080] In step S207, the target feedback image is overlaid with a preset reference object to obtain a visual alignment view.

[0081] In the embodiments of this specification, the visual alignment view can be the display result after overlay display. Overlay display can be used to characterize the display method of simultaneously presenting the target feedback image and the preset reference object in the same display space. Overlay display is used to make the target feedback image and the preset reference object comparable in spatial position. The visual alignment view is used to simultaneously include the imaging content of the target feedback image and the imaging position represented by the preset reference object.

[0082] In one possible implementation, the overlay display may include: determining a alignment view coordinate system for display, displaying the target feedback image according to the alignment view coordinate system, and simultaneously drawing and displaying a preset reference object according to the alignment view coordinate system, thereby obtaining a visual alignment view. Furthermore, when the image size parameters of the target feedback image are inconsistent with the coordinate representation parameters of the preset reference object, the preset reference object can be scaled or positionally adapted based on a preset coordinate transformation relationship, so that the display position of the preset reference object in the visual alignment view corresponds to the imaging position represented by the preset reference object. Furthermore, when the target feedback image is updated, the overlay display can be updated synchronously with the target feedback image to maintain the representational capability of the visual alignment view of the current optical virtual image imaging state.

[0083] For example, the preset reference object can be a reference scribing pattern, which may include a reference mark used to characterize the coordinates of the reference center. Overlay display may include displaying the outline of the reference scribing pattern overlaid on the target feedback image, so that the reference scribing pattern and the target feedback image are simultaneously visible in the same view, thereby forming a visual alignment view. For example, when the target feedback image contains a pattern center position corresponding to the target feature image, the visual alignment view can be used to present the relative positional relationship between the pattern center position and the reference mark.

[0084] Optionally, the overlay display can be configured to display a local or global area of ​​a preset reference object in the visual alignment view to adapt to the imaging coverage of different optical modules. Optionally, the overlay display can be configured to generate visual alignment views at different magnifications to make the visual alignment view more clearly present local imaging position deviations.

[0085] In practical applications, by overlaying the target feedback image with a preset reference object to obtain a visual alignment view, it is helpful to establish a visual comparison relationship between the target feedback image and the preset reference object in the same view, thereby improving the consistency of alignment state recognition and providing a visual input basis for the subsequent generation of pose adjustment commands.

[0086] In one possible implementation, the active alignment device further includes an image display terminal; mapping the target feedback image to a background display layer and mapping a preset reference object to a foreground reference layer; obtaining preset fusion parameters; and based on the preset fusion parameters, overlaying and rendering the foreground reference layer onto the background display layer to obtain a visual alignment view.

[0087] In the embodiments of this specification, the image display terminal can be a display device for presenting a visual alignment view. The image display terminal can establish a data connection with an active alignment device to receive image data and reference data for rendering. For example, the image display terminal can display a camera preview window, which can be used to present a visual alignment view after the target feedback image and a preset reference object are superimposed.

[0088] The background display layer can be a display layer obtained by mapping the target feedback image, and the background display layer is used to carry the pixel content of the target feedback image; the foreground reference layer can be a display layer obtained by mapping a preset reference object, and the foreground reference layer is used to carry the reference mark content of the preset reference object.

[0089] Preset blending parameters can be a set of parameters used to control the display effect after the foreground reference layer and the background display layer are superimposed. Preset blending parameters include at least one of transparency parameters or complementary color parameters. Preset blending parameters are used to ensure that the foreground reference layer and the background display layer are still visible on the image display terminal after being superimposed.

[0090] In one alternative implementation, the overlay display mode can be triggered by an interactive control of the image display terminal, for example, by responding to a manual adjustment button _Click to trigger the overlay display mode in order to present a visual alignment view on the image display terminal.

[0091] In one possible implementation, mapping the target feedback image to a background display layer may include: organizing the target feedback image into displayable pixel buffer data, and using the pixel buffer data as the image content of the background display layer, so that the background display layer forms a continuously updated background image on the image display terminal. Furthermore, the background display layer may adopt the same resolution or scaling parameters as the target feedback image to maintain the pixel correspondence between the background display layer and the target feedback image.

[0092] In one possible implementation, mapping a preset reference object to a foreground reference layer may include: organizing the preset reference object into a set of drawable graphic elements, and drawing the set of graphic elements as the display content of the foreground reference layer, so that the foreground reference layer forms an overlay of reference marks on the image display terminal. Furthermore, the foreground reference layer may use the same coordinate system parameters as the background display layer to maintain the spatial alignment between the foreground reference layer and the background display layer.

[0093] In one possible implementation, preset blending parameters can be obtained. These preset blending parameters can come from a preset configuration file or a preset parameter table. The transparency parameter can be used to characterize the transparency level of the foreground reference layer, ensuring that the background display layer remains visible when overlaid on top of it. The complementary color parameter can be used to characterize the color differentiation strategy between the foreground reference layer and the background display layer, ensuring that the foreground reference layer remains recognizable when overlaid on top of it.

[0094] In one possible implementation, overlaying the foreground reference layer onto the background display layer based on preset blending parameters may include: performing a blending calculation on the foreground reference layer and the background display layer at the pixel level or at the primitive level during the rendering process, so that the blended pixel result is output as a visual alignment view. Furthermore, when the transparency parameter is used as the preset blending parameter, the blending calculation can be a weighted composite of the foreground reference layer and the background display layer based on the transparency parameter; when the complementary color parameter is used as the preset blending parameter, the blending calculation can be mapping the color values ​​of the foreground reference layer based on the complementary color parameter to enhance the contrast between the foreground reference layer and the background display layer.

[0095] For example, when the target feature image is an array calibration image, the target feedback image can be an array feedback image, which can present a grid imaging form with distortion or rotational deviation; the preset reference object can be an ideal color grid. After overlaying the array feedback image with the ideal color grid, the grid misalignment can be presented in the visualization alignment view. The grid misalignment can be used to characterize the superposition of deviation in the translation direction and deviation in the rotation direction in the same view.

[0096] Optionally, the image display terminal can be a monitor or a touch screen; the background display layer can be a video frame sequence formed by the target feedback image; the foreground reference layer can be a reference scribing pattern corresponding to a preset reference object; the transparency parameter can be used to make the reference scribing pattern superimposed on the video frame sequence in a semi-transparent manner, so that the target feedback image and the preset reference object can be seen simultaneously on the image display terminal.

[0097] Optionally, the preset fusion parameters can be adjusted according to the brightness changes of the target feedback image to maintain the visibility of the foreground reference layer under different brightness backgrounds. Optionally, the foreground reference layer can contain multiple reference marker sub-layers, and the multiple reference marker sub-layers can each use different transparency parameters or different complementary color parameters to adapt to the display characteristics of different target feature images.

[0098] In practical applications, by mapping the target feedback image to a background display layer and the preset reference object to a foreground reference layer, and by overlaying and rendering the foreground reference layer on top of the background display layer based on preset fusion parameters to obtain a visual alignment view, it is helpful to keep the target feedback image and the preset reference object visible at the same time on the image display terminal, thereby improving the readability of the alignment view and improving the consistency of the alignment operation judgment.

[0099] In one example, with Figure 3 For example, Figure 3 This is a schematic diagram illustrating the overlay of visualization alignment view layers for a crosshair target scene according to an exemplary embodiment. The bottom layer can be a background display layer, which corresponds to the bottom layer: the camera's actual captured image (white crosshair target). The bottom layer: the camera's actual captured image (white crosshair target) can represent the current imaging form of the optical virtual image represented by the target feedback image. The top layer can be a foreground reference layer, which corresponds to the top layer: a virtual reference system (color-dashed standard box). The top layer: the virtual reference system (color-dashed standard box) can represent the imaging position range under the ideal optical assembly state represented by the preset reference object.

[0100] Preset blending parameters can include at least one of transparency or complementary color parameters, which can be used to ensure that the foreground reference layer and the background display layer remain visible simultaneously after being overlaid. Based on the preset blending parameters, the top layer: virtual reference system (color dashed standard box) can be overlaid and rendered on the bottom layer: camera-captured image (white crosshair target), thus obtaining the interactive interface: visual overlay view. In the interactive interface: visual overlay, the white crosshair target and the color dashed standard box can be presented simultaneously, and the deviation D can serve as a visual indicator of the deviation of the white crosshair target relative to the color dashed standard box, so that the visual alignment view can show the deviation in the translation direction.

[0101] In another example, with Figure 4 For example, Figure 4 This is a schematic diagram illustrating the overlay of visualization alignment view layers for a checkerboard scene according to an exemplary embodiment. The bottom layer can be a background display layer, corresponding to the bottom layer: real-world image (with distorted / rotated grids). The bottom layer: real-world image (with distorted / rotated grids) can characterize the imaging form of the grid virtual image in the array feedback image, where "with distorted / rotated grids" reflects the distortion characteristics and rotation deviation of the grid pattern under optical module imaging conditions. The top layer can be a foreground reference layer, corresponding to the top layer: virtual reference system (ideal color grid). The top layer: virtual reference system (ideal color grids) can characterize the grid imaging position of a preset reference object in an ideal optical assembly state.

[0102] Preset blending parameters can include at least one of transparency or complementary color parameters. These parameters ensure that the top layer (virtual reference system, ideal color grid) and the bottom layer (real-world image, distorted / rotated grid) remain visible after being overlaid. Based on these preset blending parameters, the top layer (virtual reference system, ideal color grid) can be overlaid and rendered on top of the bottom layer (real-world image, distorted / rotated grid), resulting in the interactive interface: visual overlay (displaying rotation + translation deviation). In the interactive interface: visual overlay (displaying rotation + translation deviation), grid misalignment serves as a visual representation of the deviation after overlay. Grid misalignment can simultaneously present deviations in both rotation and translation directions, enabling the visual alignment view to present the overlay interpretation result of "displaying rotation + translation deviation."

[0103] In step S209, a pose adjustment command is generated based on the visual alignment view to guide the adjustment of the relative pose between the optical module and the image acquisition device.

[0104] In the embodiments of this specification, the pose adjustment command can be used to characterize adjustment guidance information for relative pose. The pose adjustment command can include command content for indicating the adjustment direction, indicating the adjustment magnitude level, or indicating the adjustment status. The relative pose can be used to characterize the relative spatial relationship between the optical module and the image acquisition device, and the adjustment of the relative pose can be used to characterize the process of changing the relative spatial relationship. The visual alignment view can be used to characterize the display result after the target feedback image and the preset reference object are superimposed. The pose adjustment command generated based on the visual alignment view is used to define the correspondence between the pose adjustment command and the visual alignment view.

[0105] In one possible implementation, an alignment guidance state can be determined based on a visual alignment view, and pose adjustment instructions can be generated based on this state. The alignment guidance state can characterize the relative relationship category between the target feedback image and a preset reference object in the visual alignment view; the alignment guidance state can include at least one of a deviation state or an alignment state. The pose adjustment instructions can output different guidance content based on the alignment guidance state, so that the pose adjustment instructions can be used to guide the direction or sequence of relative pose adjustment.

[0106] Furthermore, pose adjustment instructions can be displayed in conjunction with a Human Machine Interface (HMI), and can also be displayed in conjunction with a User Interface (UI) to present guidance information in a visual alignment view. The presentation form of pose adjustment instructions may include at least one of text prompts, graphic prompts, or symbol prompts. Graphic prompts may include at least one of arrow prompts, border-highlighted prompts, or flashing marker prompts.

[0107] For example, the visual alignment view can be displayed in the graphical user interface (GUI) of the image display terminal; pose adjustment instructions can be displayed in the visual alignment view as arrow prompts, which indicate the direction of relative pose adjustment; pose adjustment instructions can also output prompt sounds on the image display terminal to indicate that the adjustment status corresponding to the pose adjustment instruction has changed. For example, pose adjustment instructions can include two types of status prompts: "Continue Adjustment" and "Complete." "Continue Adjustment" indicates that the relative pose is still in the adjustment process, while "Complete" indicates that the relative pose is in a state where adjustment can be stopped.

[0108] Optionally, the pose adjustment command can be configured as a tiered prompt format, which represents different levels of intensity for different degrees of deviation. The tiered prompt format may include at least one of different flashing frequencies, different prompt tone rhythms, or different text level labels. Optionally, the pose adjustment command can be configured to be output only when the visual alignment view meets preset display conditions, thereby reducing the probability of unnecessary prompts obscuring the visual alignment view.

[0109] In practical applications, generating pose adjustment instructions based on the visual alignment view and using them to guide the adjustment of relative pose helps to transform the alignment state in the visual alignment view into executable guidance information, thereby improving the clarity of the alignment operation instructions and the consistency of the alignment process.

[0110] In one possible implementation, the actual center coordinates of the target feedback image and the reference center coordinates of the preset reference object are obtained based on the visual alignment view; the positional deviation value between the actual center coordinates and the reference center coordinates is calculated; and if the positional deviation value is greater than a preset length, a first adjustment command is generated.

[0111] In the embodiments of this specification, the actual center coordinates can be coordinates determined based on a visual alignment view, and the actual center coordinates can be used to characterize the center position of the optical virtual image in the target feedback image. The reference center coordinates can be coordinates determined based on a visual alignment view, and the reference center coordinates can be used to characterize the center position of the imaging position represented by a preset reference object.

[0112] The position deviation value can be the difference between the actual center coordinates and the reference center coordinates. The position deviation value can be used to characterize the degree of deviation of the actual center coordinates from the reference center coordinates.

[0113] The preset length can be a threshold parameter used for comparison with the position deviation value, and the preset length can be represented as a pixel length threshold. The first adjustment command can be a command generated when the position deviation value is greater than the preset length. The first adjustment command indicates that the relative translation position between the optical module and the image acquisition device needs to be adjusted to reduce the position deviation value.

[0114] In one possible implementation, obtaining the actual center coordinates and reference center coordinates from the visual alignment view may include: locating the center feature point corresponding to the target feedback image in the visual alignment view, and determining the coordinates corresponding to the center feature point as the actual center coordinates; locating the reference center point corresponding to a preset reference object in the visual alignment view, and determining the coordinates corresponding to the reference center point as the reference center coordinates. Calculating the position deviation value may include: performing coordinate difference between the actual center coordinates and the reference center coordinates to obtain a two-dimensional coordinate difference component, and obtaining the position deviation value based on the two-dimensional coordinate difference component. Generating a first adjustment command may include: outputting a first adjustment command when the position deviation value is greater than a preset length; the first adjustment command may further carry the direction information of the two-dimensional coordinate difference component to characterize the adjustment direction of the relative translation position.

[0115] In one example, the position deviation value can be represented in the form of Euclidean distance D, which can be calculated from the difference between the actual center coordinates (x, y) and the reference center coordinates (C_x, C_y). Euclidean distance D can be equal to the square root of the sum of the squares of the differences between x and C_x and the squares of the differences between y and C_y. The calculated result of Euclidean distance D can be compared with a preset length to trigger the generation of a first adjustment command. For example, the preset length can be 5 pixels as a threshold. If the Euclidean distance D is less than the preset length, the indicator box can be switched from yellow to green, and a beeping sound can be output to indicate that the position deviation value has entered the threshold range. If the Euclidean distance D is greater than or equal to the preset length, the yellow state can be maintained, and a prompt to continue adjusting the relative translation position can be given.

[0116] In one optional implementation, when the target feature image is a cross pattern image, the center feature point can be the intersection position of the cross pattern image; when the preset reference object is a reference cross mark, the reference center point can be the intersection position of the reference cross mark. Exemplarily, the position deviation value can be calculated based on the magnitude of the two-dimensional coordinate difference components. Exemplarily, the first adjustment command can include direction prompt information and deviation prompt information, whereby the direction prompt information characterizes the adjustment direction of the relative translation position, and the deviation prompt information characterizes the comparison result between the position deviation value and the preset length.

[0117] In practical applications, the first adjustment command can be used to indicate the direction of adjustment of the relative translation position, and the first adjustment command can be used to indicate whether the position deviation value is greater than the preset length, so that the adjustment process of the relative translation position has a consistent judgment standard.

[0118] In one possible implementation, the target feature image is an array calibration image; the target feedback image is an array feedback image; based on the visualized alignment view, the feature coordinates of multiple corner points in the array feedback image are obtained; based on the feature coordinates of multiple corner points, an optical distortion model of the optical module is constructed; based on the optical distortion model, the optical distortion center of the optical module is calculated; and the optical distortion center is determined as the actual center coordinates.

[0119] In the embodiments of this specification, the array calibration image can be a form of target feature image, and the array calibration image can include feature units distributed in a row and column array. The array feedback image can be a form of target feedback image, and the array feedback image is used to characterize the optical virtual image corresponding to the array calibration image.

[0120] Corner feature coordinates can be the coordinates corresponding to the corner positions in the array feedback image. Multiple corner feature coordinates can be used to characterize the array geometric feature distribution in the array feedback image.

[0121] Optical distortion models can be used to characterize the distortion characteristics of optical modules. These models can be constructed based on the coordinates of multiple corner points. The optical distortion center, calculated from the optical distortion model, represents the center of the optical distortion distribution. Defining the optical distortion center as the actual center coordinates defines the scope of the actual center coordinates as the optical distortion center.

[0122] In one possible implementation, after setting the target feature image as the array calibration image, an array feedback image corresponding to the array calibration image can be obtained. Obtaining multiple corner feature coordinates in the array feedback image based on the visualized alignment view can include: detecting the intersection positions of array feature units in the array feedback image to obtain multiple corner feature coordinates. Constructing an optical distortion model based on multiple corner feature coordinates can include: establishing a correspondence between the multiple corner feature coordinates and the ideal corner distribution of the array calibration image, and fitting optical distortion model parameters based on the correspondence to form an optical distortion model. Calculating the optical distortion center based on the optical distortion model can include: determining the optical distortion center based on the optical distortion model parameters, and determining the optical distortion center as the actual center coordinates.

[0123] In one optional implementation, the array calibration image can be a checkerboard image, comprising an array of alternating black and white squares; the coordinates of multiple corner points can be the coordinates of the intersections of the black and white squares in the array feedback image. Exemplarily, the optical distortion model can be a radial distortion model, which can include distortion coefficients k1 and k2. Exemplarily, the optical distortion center can be denoted as COD (Center of Distortion), and the optical distortion center can be presented as a center marker in the array feedback image to visually locate the corresponding position of the optical distortion center in a visual alignment view.

[0124] Optionally, the optical distortion center can be denoted as COD and drawn and displayed as a center mark in the array feedback image. In the visualization alignment view, the center mark corresponding to COD can be presented simultaneously with the reference cross center in the preset reference object to guide the alignment of the center mark corresponding to COD with the reference cross center, thereby switching the alignment aperture of the actual center coordinates from the geometric center of the pattern to the optical distortion center.

[0125] For example, the array feedback image can be a virtual image of a checkerboard obtained by imaging a checkerboard image, and the coordinates of multiple corner points can be the position coordinates of the intersection of multiple black and white squares in the virtual image of the checkerboard. In the visualization alignment view, the optical distortion center can be drawn and displayed in the form of a center mark, and the optical distortion center can be used as the actual center coordinate to participate in the determination of the position deviation value, so that the calculation caliber of the position deviation value is consistent with the distortion characteristics reflected by the distribution of checkerboard corner points.

[0126] In practical applications, determining the optical distortion center as the actual center coordinates can provide a standard for determining the actual center coordinates in array calibration image scenarios, thereby ensuring that the calculation standard for position deviation values ​​is consistent with the optical distortion model.

[0127] In one possible implementation, based on the visualized alignment view, the feature axis information of the target feedback image and the reference axis information of the preset reference object are determined; the attitude deviation value between the feature axis information and the reference axis information is calculated; if the attitude deviation value is greater than a preset angle, a second adjustment command is generated; if the position deviation value is less than or equal to a preset length and the attitude deviation value is less than or equal to a preset angle, a third adjustment command is generated.

[0128] In the embodiments of this specification, the feature axis information can be information determined based on a visual alignment view, and the feature axis information can be used to characterize the pattern direction reference corresponding to the target feature image in the target feedback image; the reference axis information can be information determined based on a visual alignment view, and the reference axis information can be used to characterize the direction reference represented by the preset reference object.

[0129] The attitude deviation value can be the difference between the feature axis information and the reference axis information. The attitude deviation value can be used to characterize the degree of rotational deviation of the target feedback image relative to the preset reference object.

[0130] The preset angle can be a threshold parameter used for comparison with the attitude deviation value.

[0131] The second adjustment command can be generated when the attitude deviation value is greater than the preset angle. The second adjustment command indicates that the relative rotation angle between the optical module and the image acquisition device needs to be adjusted to reduce the attitude deviation value.

[0132] The third adjustment command can be used to characterize that the optical module is in a fully aligned state relative to the image acquisition device. The fully aligned state can be defined as a state where the positional deviation is less than or equal to a preset length and the attitude deviation is less than or equal to a preset angle.

[0133] In one possible implementation, determining the feature axis information based on the visual alignment view may include: locating a linear feature corresponding to the target feature image in the target feedback image, and performing direction fitting on the linear feature to obtain the feature axis information. Determining the reference axis information based on the visual alignment view may include: locating a reference linear marker in a preset reference object, and performing direction resolution on the reference linear marker to obtain the reference axis information. Calculating the attitude deviation value may include: performing a difference calculation based on the direction parameters corresponding to the feature axis information and the direction parameters corresponding to the reference axis information to obtain the attitude deviation value. Generating a second adjustment command may include: outputting a second adjustment command when the attitude deviation value is greater than a preset angle, and including adjustment direction information for the relative rotation angle in the second adjustment command to guide the attitude deviation value to decrease. Generating a third adjustment command may include: outputting a third adjustment command when the position deviation value is less than or equal to a preset length and the attitude deviation value is less than or equal to a preset angle to indicate the alignment completion status.

[0134] For example, the visual alignment view can simultaneously present a white cross and a green reference line, where the white cross is a pattern representation in the target feedback image, and the green reference line is a reference mark representation of the preset reference object; the feature axis information can be selected as the horizontal direction of the white cross, and the reference axis information can be selected as the direction of the green reference line; the attitude deviation value can characterize the angle deviation between the horizontal line of the white cross and the green reference line; when the attitude deviation value is greater than the preset angle, a second adjustment command can be output to prompt continued adjustment of the relative rotation angle, thereby achieving rotation alignment (Roll Alignment) and suppressing roll error (Roll Error).

[0135] For example, when the position deviation value meets the preset length and the posture deviation value meets the preset angle, the indicator box in the visual alignment view can switch from yellow to green and output a beeping sound to correspond to the alignment completion state represented by the third adjustment command.

[0136] Optionally, the preset angle can be configured based on the width of the linear features in the target feature image, so that a stricter preset angle caliber is used when the linear features are thinner, and a more lenient preset angle caliber is used when the linear features are thicker. Optionally, the feature axis information can be determined by the statistical results of the directions of multiple linear features to reduce the impact of local noise on the attitude deviation value.

[0137] In practical applications, by determining the feature axis information and reference axis information based on the visual alignment view and calculating the attitude deviation value, a second adjustment command is generated when the attitude deviation value is greater than a preset angle to guide the adjustment of the relative rotation angle, and a third adjustment command is generated when the position deviation value and attitude deviation value simultaneously enter the threshold range. This helps to transform the judgment of rotation alignment from subjective visual judgment to quantifiable threshold judgment and provides a consistent output caliber for the alignment completion state.

[0138] In one possible implementation, the active alignment device further includes a pose adjustment mechanism; an optical module is fixedly mounted on the pose adjustment mechanism to move in tandem with it; pose adjustment parameters are determined based on a first adjustment command or a second adjustment command; a translation adjustment component and a rotation adjustment component are determined based on the pose adjustment parameters; the pose adjustment mechanism is driven to move based on the translation adjustment component so that the position deviation value is less than or equal to a preset length; and the pose adjustment mechanism is driven to rotate based on the rotation adjustment component so that the posture deviation value is less than or equal to a preset angle.

[0139] In the embodiments of this specification, the pose adjustment mechanism can be a mechanism for providing relative pose adjustment capability, and the pose adjustment mechanism can include an adjustment structure for realizing displacement and rotation; the optical module is fixedly mounted on the pose adjustment mechanism to follow the pose adjustment mechanism in linkage to define the assembly relationship between the optical module and the pose adjustment mechanism.

[0140] The pose adjustment parameters can be a set of parameters determined based on the first adjustment command or the second adjustment command. The pose adjustment parameters can be used to characterize the adjustment target of the relative pose.

[0141] Translation adjustment components can be the translation part of the pose adjustment parameters, and are used to characterize the adjustment direction and amount of the relative translation position; rotation adjustment components can be the rotation part of the pose adjustment parameters, and are used to characterize the adjustment direction and amount of the relative rotation angle.

[0142] In one possible implementation, determining the pose adjustment parameters based on the first or second adjustment command may include: parsing the first adjustment command into translation direction information and translation amount information, and combining the translation direction information and translation amount information into the translation component of the pose adjustment parameters; parsing the second adjustment command into rotation direction information and rotation amount information, and combining the rotation direction information and rotation amount information into the rotation component of the pose adjustment parameters. Determining the translation adjustment component and rotation adjustment component based on the pose adjustment parameters may include: performing component decomposition on the pose adjustment parameters to obtain the translation adjustment component and the rotation adjustment component. Driving the pose adjustment mechanism to perform displacement based on the translation adjustment component may include: according to the direction and adjustment amount represented by the translation adjustment component, causing the pose adjustment mechanism to produce a corresponding displacement, so that the position deviation value gradually decreases until the position deviation value is less than or equal to a preset length. Driving the pose adjustment mechanism to perform rotation based on the rotation adjustment component may include: according to the direction and adjustment amount represented by the rotation adjustment component, causing the pose adjustment mechanism to produce a corresponding rotation, so that the attitude deviation value gradually decreases until the attitude deviation value is less than or equal to a preset angle.

[0143] In one example, the pose adjustment mechanism can be implemented using a 6-axis adjustment table. The 6-axis adjustment table supports X / Y / Z translation and Pitch / Yaw / Roll rotation, and 6-DOF (Six Degrees of Freedom) can be used as a way to identify the degree of freedom capability. Exemplarily, the adjustment process of the pose adjustment mechanism can include a coarse adjustment stage and a fine adjustment stage. The coarse adjustment stage can be used to complete larger displacement or rotation adjustments, while the fine adjustment stage can be used to complete smaller displacement or rotation adjustments. Exemplarily, the pose adjustment mechanism can use manual rotation of the adjustment table to achieve gradual adjustment of displacement or rotation, so that the position deviation and attitude deviation values ​​gradually converge to a threshold range.

[0144] Optionally, the pose adjustment parameters may include a configuration parameter for the order of translation and rotation adjustment components, which indicates whether the translation adjustment component or the rotation adjustment component should be executed first. Optionally, the translation adjustment component may be decomposed into components along two orthogonal directions for staged displacement adjustment.

[0145] In practical applications, by determining the pose adjustment parameters based on the first or second adjustment command, and separating them into translational and rotational adjustment components, and then driving the pose adjustment mechanism to perform displacement and rotation respectively, the deviation state reflected in the visual position view can be transformed into an executable mechanism adjustment amount, thereby providing a consistent adjustment path for the position deviation value and attitude deviation value to enter the threshold range.

[0146] For example, with Figure 4 For example, Figure 4 This is a schematic diagram illustrating target projection, image acquisition, and motion control of an active alignment device according to an exemplary embodiment. The optical module can be implemented using a Birdbath optical module, which can be fixedly mounted on a 6-DOF stage as a DUT (Device Under Test) to move in tandem with the 6-DOF stage.

[0147] After alignment is initiated, the target feature image can be transmitted as a projection target to the Birdbath optical module (DUT) via HDMI (High Definition Multimedia Interface) or DP (DisplayPort) to drive the Birdbath optical module (DUT) to form a target imaging beam in the light projection path. The optical virtual image corresponding to the target imaging beam can be acquired by the bionic camera EyeCam. The image acquisition data output by the bionic camera EyeCam can be transmitted to the overlay screen via USB (Universal Serial Bus) 3.0 or GigE (Gigabit Ethernet) to obtain the target feedback image and form a visual alignment view. The pose adjustment command obtained based on the visual alignment view can be output as motion control command to the 6-axis precision stage (6-DOF Stage) to guide the 6-axis precision stage to drive the Birdbath optical module (DUT) to move or rotate, thereby realizing a closed loop of relative pose adjustment between the optical module and the image acquisition device.

[0148] In one possible implementation, the optical module is a binocular optical module, comprising a first optical module and a second optical module. The binocular optical module can be fixedly mounted on the same pose adjustment mechanism to move in tandem with it. The target feedback image can include multiple target feedback images corresponding to different imaging channels of the binocular optical module. These multiple target feedback images can be used to characterize the optical virtual images formed by the target feature images in different imaging channels. The visualization alignment view can include multiple overlay display areas, which can be used to present the overlay display results of the target feedback images of each imaging channel and a preset reference object. Pose adjustment commands can be generated based on the deviations of the multiple overlay display areas to guide the adjustment of the relative pose between the binocular optical module and the image acquisition device.

[0149] In one possible implementation, corresponding positional deviation values ​​or attitude deviation values ​​can be obtained based on multiple overlaid display areas, and a comprehensive deviation amount can be determined based on these multiple positional deviation values ​​or attitude deviation values. The comprehensive deviation amount can be used to characterize the overall deviation degree of the binocular optical module under the same relative pose. A pose adjustment command can be generated based on the comprehensive deviation amount to guide the pose adjustment mechanism to shift or rotate the binocular optical module, thereby reducing the comprehensive deviation amount. Optionally, if the positional deviation value or attitude deviation value corresponding to any imaging channel meets a threshold condition while the deviation corresponding to another imaging channel does not, the pose adjustment command can continue to be output based on the comprehensive deviation amount, so that the deviations of the binocular imaging channels tend to converge under the same relative pose.

[0150] For example, multiple superimposed display areas can be displayed side by side or in partitions on the same image display terminal; the overall deviation can be obtained by combining multiple position deviation values ​​and multiple posture deviation values; the posture adjustment command can include command content that simultaneously contains translation adjustment components and rotation adjustment components to correspond to the displacement and rotation actions of the posture adjustment mechanism.

[0151] In practical applications, by generating pose adjustment commands based on the deviations of multiple superimposed display areas and guiding the pose adjustment mechanism to perform coordinated adjustments to the binocular optical module, an executable adjustment closed loop can be formed under rigid connection constraints, providing a consistent compromise adjustment basis for the deviation state of the binocular imaging channel. Specifically, when the first optical module is in the aligned state, a first feedback image corresponding to the first optical module is acquired; the first feedback image is mirrored to generate a dynamic reference image; a second feedback image corresponding to the second optical module is acquired; the relative position deviation between the second feedback image and the dynamic reference object is calculated; and a coordinated adjustment command is generated based on the relative position deviation to drive the second optical module to perform symmetrical adjustments following the mirrored pose of the first optical module.

[0152] In the embodiments of this specification, the binocular optical module can be an optical module that includes a first optical module and a second optical module. The first optical module is used to characterize the optical module corresponding to the first imaging channel in the binocular optical module, and the second optical module is used to characterize the optical module corresponding to the second imaging channel in the binocular optical module.

[0153] The first feedback image can be the target feedback image corresponding to the first optical module, and the second feedback image can be the target feedback image corresponding to the second optical module.

[0154] The dynamic reference image can be the image obtained by mirroring and flipping the first feedback image. The dynamic reference object can be reference object data used in the calculation of relative position deviation, and the dynamic reference object can be represented by the dynamic reference image.

[0155] The relative position deviation can be the amount of relative position difference between the second feedback image and the dynamic reference object.

[0156] The linkage adjustment command can be a command generated based on the relative position deviation. The linkage adjustment command is used to guide the second optical module to make symmetrical adjustments following the mirror pose of the first optical module.

[0157] In one possible implementation, when the first optical module is in the alignment completed state, a first feedback image corresponding to the first optical module can be acquired. Mirroring the first feedback image can include performing a pixel coordinate mapping transformation on the first feedback image based on a preset mirror axis to obtain a dynamic reference image. Acquiring a second feedback image corresponding to the second optical module can include acquiring the second feedback image corresponding to the second optical module during the alignment process. Calculating the relative positional deviation between the second feedback image and the dynamic reference object can include matching the second feedback image and the dynamic reference object in a unified coordinate system to obtain the relative positional deviation. Generating a linkage adjustment command based on the relative positional deviation can include outputting a linkage adjustment command when the relative positional deviation meets a preset trigger condition. The linkage adjustment command indicates that the second optical module needs to perform symmetrical adjustments following the mirror pose of the first optical module.

[0158] For example, the first optical module can be a left-eye module, and the second optical module can be a right-eye module; the dynamic reference image can be an image obtained by horizontally mirroring the first feedback image; the dynamic reference object can use the dynamic reference image as a representation carrier. For example, the dynamic reference image can be superimposed on the second feedback image in a semi-transparent manner so that the relative relationship between the dynamic reference image and the second feedback image can be observed simultaneously in the same display view, and the relative position deviation can be calculated accordingly; the relative position deviation can be used to characterize the deviation component in the vertical direction, thereby being used for alignment determination and symmetry adjustment guidance related to Vertical Alignment.

[0159] Optionally, the mirror flipping process can be associated with the display orientation configuration of the target feature image. This configuration characterizes the symmetry between the first and second optical modules in the display direction, ensuring consistent comparability between the dynamic reference image and the second feedback image. Optionally, the linkage adjustment command can be output as a combination of directional and deviation information. The directional information characterizes the symmetry adjustment direction, while the deviation information characterizes the magnitude or component of the relative positional deviation.

[0160] In practical applications, by generating a dynamic reference image after the first optical module is in the alignment state and calculating the relative position deviation based on the dynamic reference object, and then generating a linkage adjustment command based on the relative position deviation to guide the second optical module to make symmetrical adjustments, it helps to transform the binocular consistency alignment process into a quantifiable deviation judgment based on image comparison, thereby helping to reduce the parallax risk in the vertical direction of binoculars and improve the consistency of binocular imaging. Figure 6 This is a block diagram illustrating an active alignment device for an optical module according to an exemplary embodiment. (Refer to...) Figure 6 The device may include a positioning control module 601, an optical module 603, an image acquisition device 605, an image display terminal 607, and a pose adjustment mechanism 609. The alignment control module 5601 is used to respond to the alignment start command by sending a target feature image to the optical module 5603; obtaining a preset reference object corresponding to the target feature image; the preset reference object represents the imaging position of the target feature image after transmission through the light projection path of the optical module 5603 in an ideal optical assembly state; and generating a pose adjustment command based on the visualized alignment view. Optical module 5603 is used to project a target imaging beam based on a target feature image; Image acquisition equipment is used to acquire the target imaging beam and obtain the target feedback image; the target feedback image represents the optical virtual image corresponding to the target feature image; The image display terminal 5607 is used to overlay the target feedback image with a preset reference object to obtain a visual alignment view; The pose adjustment mechanism 5609 is used to adjust the relative pose between the optical module 5603 and the image acquisition device in response to a pose adjustment command.

[0161] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here. Figure 7 This is a block diagram illustrating an electronic device for active alignment of an optical module according to an exemplary embodiment. The electronic device may be a host display terminal device, and its internal structure diagram may be as follows: Figure 7As shown, the electronic device includes a processor, memory, and peripheral interfaces connected via a system bus. The memory may include non-volatile storage media and internal memory. The non-volatile storage media may store the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs. Exemplarily, the non-volatile storage media may include ROM (Read-Only Memory), and the internal memory may include RAM (Random Access Memory). Communication connections can be established with computing storage devices and data storage devices via the peripheral interfaces to meet the needs of scenarios where the same video source data is transmitted in parallel to different hardware interfaces. Exemplarily, the peripheral interfaces may include interfaces based on the computer expansion bus standard and interfaces based on the serial storage interface standard. Further, the interface based on the computer expansion bus standard may be a PCIe interface; the interface based on the serial storage interface standard may be a SATA interface. In one example, the total bandwidth of the 4 lanes of PCIe 3.0 x4 may be 32Gbps; in one example, the bandwidth of the SATA 3.0 interface may be 6Gbps. The computer program, running on the processor, can implement the active alignment method of the optical module provided in this disclosure embodiment. The system is connected via a system bus, comprising a processor, memory, communication interface, display interface, and input devices. The processor can be one or more processing units, providing image processing and control computing capabilities. The memory can include volatile and non-volatile memory, and can be used to store computer programs, target feature images, preset reference objects, and target feedback images during the alignment process. The communication interface can be used for data interaction with the optical module, and can include a USB interface or a GigE interface. The display interface can be used to drive the display components of the image display terminal for display output, and can include an HDMI interface or a DP interface. The input devices can be used to generate alignment start commands, and can include button input devices, mouse input devices, or touch input devices. The memory can store computer programs, which, when executed by the processor, can implement an active alignment method for the optical module. Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the electronic device to which the present disclosure is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements. It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments of this disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0162] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method embodiments.

[0163] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer storage medium, such as a read-only memory, a disk, or an optical disk.

[0164] The above description is only a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. An active alignment method for an optical module, characterized in that, It is applied to an active alignment device, which includes an optical module and an image acquisition device; The image acquisition device is positioned on the light projection path of the optical module; the method includes: In response to the alignment start command, a target feature image is sent to the optical module to drive the optical module to project a target imaging beam based on the target feature image; Based on the image acquisition device, the target imaging beam is acquired to obtain a target feedback image; the target feedback image represents an optical virtual image corresponding to the target feature image; Obtain a preset reference object corresponding to the target feature image; the preset reference object represents the imaging position of the target feature image after transmission through the light projection path of the optical module in an ideal optical assembly state; The target feedback image is overlaid with the preset reference object to obtain a visual alignment view; Based on the visualized alignment view, pose adjustment instructions are generated to guide the adjustment of the relative pose between the optical module and the image acquisition device.

2. The method according to claim 1, characterized in that, The image acquisition device is equipped with a photosensitive component; the step of acquiring the target imaging beam based on the image acquisition device to obtain a target feedback image includes: The image acquisition device is controlled to receive the target imaging beam at a preset eyepoint position; the preset eyepoint position represents the observation position where the optical module projects a complete optical virtual image; Based on the photosensitive component, the received target imaging beam is photoelectrically converted to generate a digital image signal; The digital image signal is analyzed to obtain the target feedback image.

3. The method according to claim 1, characterized in that, The active alignment device further includes an image display terminal; it overlays the target feedback image with the preset reference object to obtain a visual alignment view, including: The target feedback image is mapped as a background display layer, and the preset reference object is mapped as a foreground reference layer; Obtain preset blending parameters; the preset blending parameters include at least one of transparency parameters or complementary color parameters; the preset blending parameters are used to ensure that the foreground reference layer and the background display layer are still simultaneously visible on the image display terminal after being superimposed. Based on the preset fusion parameters, the foreground reference layer is overlaid and rendered on the background display layer to obtain the visual alignment view.

4. The method according to claim 1, characterized in that, The step of generating pose adjustment instructions based on the visualized alignment view includes: Based on the visual alignment view, obtain the actual center coordinates of the target feedback image and the reference center coordinates of the preset reference object; Calculate the positional deviation between the actual center coordinates and the reference center coordinates; If the position deviation value is greater than a preset length, a first adjustment command is generated; the first adjustment command indicates that the relative translation position between the optical module and the image acquisition device needs to be adjusted to reduce the position deviation value.

5. The method according to claim 4, characterized in that, The method further includes: Based on the visual alignment view, determine the feature axis information of the target feedback image and the reference axis information of the preset reference object; Calculate the attitude deviation value between the feature axis information and the reference axis information; If the attitude deviation value is greater than a preset angle, a second adjustment command is generated; the second adjustment command indicates that the relative rotation angle between the optical module and the image acquisition device needs to be adjusted to reduce the attitude deviation value. When the position deviation value is less than or equal to the preset length and the attitude deviation value is less than or equal to the preset angle, a third adjustment command is generated; the third adjustment command indicates that the optical module is in a state of alignment completion relative to the image acquisition device.

6. The method according to claim 5, characterized in that, The active alignment device further includes a pose adjustment mechanism; the optical module is fixedly mounted on the pose adjustment mechanism to move in tandem with it; the method further includes: Determine the pose adjustment parameters based on the first adjustment command or the second adjustment command; Based on the pose adjustment parameters, determine the translation adjustment component and the rotation adjustment component; Based on the translation adjustment component, the pose adjustment mechanism is driven to perform displacement so that the position deviation value is less than or equal to the preset length; Based on the rotation adjustment component, the posture adjustment mechanism is driven to rotate so that the posture deviation value is less than or equal to the preset angle.

7. The method according to claim 1, characterized in that, After the step of acquiring the target imaging beam based on the image acquisition device and obtaining the target feedback image, the method includes: A target scanning region is determined in the target feedback image; the target scanning region covers the area in the target feedback image where the pixel grayscale value changes abruptly; the abrupt change represents a gradient change in the pixel grayscale value from high to low along a preset scanning direction that exceeds a preset gradient threshold; Calculate the grayscale gradient value of each pixel within the target scanning area to obtain brightness distribution information; the brightness distribution information represents the mapping relationship curve between the pixel position and the corresponding grayscale gradient value within the target scanning area. Based on the brightness distribution information, the main brightness peak is determined; Determine the secondary brightness peaks within the preset correlation range of the primary brightness peak; Calculate the ratio of the brightness values ​​between the secondary brightness peak and the primary brightness peak to obtain the peak intensity ratio. When the peak intensity ratio is greater than a preset ghosting threshold, optical anomaly information is generated; the optical anomaly information indicates that the optical module has optical ghosting that does not meet the preset imaging quality standard.

8. The method according to claim 4, characterized in that, The target feature image is an array calibration image; The target feedback image is an array feedback image; obtaining the actual center coordinates of the target feedback image based on the visualized alignment view includes: Based on the visualized alignment view, obtain the feature coordinates of multiple corner points in the array feedback image; Based on the coordinates of the multiple corner points, an optical distortion model of the optical module is constructed; Based on the optical distortion model, the optical distortion center of the optical module is calculated. The optical distortion center is determined as the actual center coordinates.

9. An active alignment device for an optical module, characterized in that, The device includes an alignment control module, an optical module, an image acquisition device, an image display terminal, and a pose adjustment mechanism. The alignment control module is used to respond to the alignment start command by sending a target feature image to the optical module and obtaining a preset reference object corresponding to the target feature image; The preset reference object represents the imaging position of the target feature image after it has been transmitted through the light projection path of the optical module under ideal optical assembly conditions; Generate pose adjustment commands based on the visual alignment view; The optical module is used to project a target imaging beam based on the target feature image; The image acquisition device is used to acquire the target imaging beam and obtain a target feedback image; the target feedback image represents an optical virtual image corresponding to the target feature image. The image display terminal is used to overlay the target feedback image with the preset reference object to obtain the visual alignment view; The pose adjustment mechanism is used to adjust the relative pose between the optical module and the image acquisition device in response to the pose adjustment command.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the active alignment method of the optical module as described in any one of claims 1 to 8.

Images