A projection calibration method and system

Abstract

The application discloses a projection calibration method and system, which respectively configures sensors and actuators for multiple projection modules, realizes independent detection and independent adjustment of the optical axes of the projection modules, and solves the problem that the optical axes of the multiple modules cannot be independently adjusted due to the mechanical fixing structure and the integral focusing mechanism; the image information of the projection area corresponding to each projection module is obtained by the sensors, and the optical axis deviation information of each module is determined based on the image information, so that the optical axis deviation detection at the module level is realized, the detection object directly corresponds to each independent projection module, and the problem that the optical axis deviations of the modules are coupled with each other and difficult to distinguish in the integral detection is avoided.

Description

Technical Field

[0001] This application relates to the field of projection technology, and in particular to a projection calibration method and system. Background Technology

[0002] With the development of automotive intelligence, the function of vehicle lights has been upgraded from basic lighting to high-precision interaction and information display. In order to achieve large-size, high-resolution projection, a multi-Micro-LED module splicing scheme is adopted, with each module serving as an independent display unit, splicing together to form a complete projection image.

[0003] When splicing multiple Micro-LED modules, the optical axis deviation between modules must be extremely small to ensure precise pixel stacking. Currently, mechanical fixing structures are typically used to position each module, and focusing is achieved through an integrated focusing mechanism. However, since mechanical fixing structures lack active adjustment freedom, integrated focusing cannot independently adjust each module. When temperature changes or vibrations cause optical axis drift in the modules, it may lead to problems such as misalignment, blurred edges, or failure to form a complete image in the spliced ​​image. Summary of the Invention

[0004] In view of this, this application provides a projection calibration method and system, the specific scheme of which is as follows:

[0005] A projection calibration method, comprising:

[0006] Image information of the projection area corresponding to each of the multiple projection modules is obtained through sensors;

[0007] Based on the image information, the optical axis deviation information of each projection module is determined respectively;

[0008] Based on the optical axis deviation information of each projection module, an adjustment command corresponding to each projection module is generated, and the adjustment command corresponding to each projection module is output to the actuator corresponding to the projection module so that the actuator can adjust the pose of the projection module.

[0009] The image information of the projection area corresponding to the projection module after the pose adjustment is verified, and the verification results are obtained.

[0010] In response to the verification result indicating that the optical axis of the projection module after pose adjustment meets the calibration requirements, the current calibration is determined to be complete.

[0011] Furthermore, the step of generating adjustment commands corresponding to each projection module based on the optical axis deviation information of each projection module, and outputting the adjustment commands corresponding to each projection module to the actuator corresponding to the projection module, includes:

[0012] Based on the optical axis deviation information of each projection module, the pose adjustment amount of each degree of freedom in the multi-degree of freedom corresponding to each projection module is determined, and an adjustment command containing the pose adjustment amount of the multi-degree of freedom corresponding to each projection module is generated.

[0013] Based on the adjustment instructions of each projection module, the actuator corresponding to the projection module is controlled so that the actuator can perform multi-degree-of-freedom pose adjustment.

[0014] Furthermore, the step of controlling the actuator corresponding to each projection module based on the adjustment command of each projection module includes:

[0015] Based on the adjustment instructions for each projection module, the motor drive module in the actuator is controlled to perform micron-level adjustments to the projection module;

[0016] The electrostriction module in the actuator is controlled to make nanoscale adjustments to the projection module.

[0017] Furthermore, the control of the electrostriction module in the actuator to perform nanometer-level adjustments to the projection module includes:

[0018] Based on the adjustment instructions of each projection module, the driving component in the electrostriction module is controlled to generate electric field-driven deformation to output nanoscale displacement.

[0019] Based on the inverse model of the hysteresis characteristics of the drive component, nonlinear compensation is performed on the input signal of the drive component to counteract the displacement deviation caused by the hysteresis characteristics of the drive component.

[0020] The nanoscale displacement is converted into a micrometer-level displacement by the mechanical amplification effect of the flexible hinge in the electrostrictive module, thereby achieving the adjustment of the pose of the projection module.

[0021] Furthermore, the step of generating adjustment instructions corresponding to each projection module based on the optical axis deviation information of each projection module includes:

[0022] Obtain environmental sensing information, which includes at least one of the following: temperature data and vibration acceleration data;

[0023] The environmental perception information is input into the prediction model, and the prediction model is used to determine the predicted value of the optical axis offset for the target time period caused by environmental disturbance. The target time period is the time period after the current time.

[0024] Feedforward compensation information is generated based on the optical axis offset prediction value;

[0025] Based on the optical axis deviation information of each projection module and the feedforward compensation information, adjustment instructions corresponding to each projection module are generated.

[0026] Furthermore, it also includes:

[0027] In response to the rate of change of the environmental perception information exceeding a target threshold, the environmental perception information is input into the prediction model to determine the predicted value of the optical axis offset.

[0028] Furthermore, it also includes:

[0029] In response to the verification result indicating that the optical axis of the projection module after the pose adjustment does not meet the calibration requirements, the optical axis deviation information is re-determined based on the image information of the projection module after the pose adjustment, and the adjustment command is re-generated based on the regenerated optical axis deviation information so that the actuator readjusts the pose of the projection module.

[0030] A projection calibration system, comprising:

[0031] Multiple projection modules, each projection module projects separately to form a projection area, and the projection areas formed by the projections of multiple projection modules are stitched together to form a projection image;

[0032] Multiple sensors are used, with a sensor installed at the location of each projection module. The sensor is used to obtain image information of the projection area corresponding to the projection module at its location.

[0033] Multiple actuators are provided, with an actuator installed at the location of each projection module. The actuators are used to adjust the pose of the projection module.

[0034] The controller is configured to determine the optical axis deviation information of each projection module based on the image information, generate adjustment commands corresponding to each projection module based on the optical axis deviation information of each projection module, and output the adjustment commands corresponding to each projection module to the actuator corresponding to the projection module, so that the actuator adjusts the pose of the projection module, verifies the image information of the projection area corresponding to the adjusted projection module, and obtains the verification result; in response to the verification result indicating that the optical axis of the adjusted projection module meets the calibration requirements, the controller determines that the current calibration is complete.

[0035] Furthermore, the actuator is a multi-degree-of-freedom adjustable structure.

[0036] The actuator includes: a motor drive module for micrometer-level adjustment of the projection module, and an electrostriction module for nanometer-level adjustment of the projection module;

[0037] The electrostrictive module includes: a drive component that generates an electric field to drive deformation to output nanoscale displacement, and a flexible hinge that uses mechanical amplification to convert the nanoscale displacement into micrometer-level displacement.

[0038] Furthermore, it also includes:

[0039] An environmental perception sensor is used to obtain environmental perception information so that the controller can input the environmental perception information into a prediction model. The prediction model determines the optical axis offset prediction value for a target time period caused by environmental disturbances, where the target time period is the time period after the current time. Feedforward compensation information is generated based on the optical axis offset prediction value. Adjustment instructions corresponding to each projection module are generated based on the optical axis deviation information of each projection module and the feedforward compensation information.

[0040] As can be seen from the above technical solutions, the projection calibration method and system disclosed in this application, by configuring sensors and actuators for multiple projection modules respectively, realizes independent detection and adjustment of the optical axis of each projection module, solving the problem that the optical axis of multiple modules cannot be adjusted independently due to the use of mechanical fixed structures and integrated focusing mechanisms; by obtaining image information of the projection area corresponding to each projection module through sensors, and determining the optical axis deviation information of each module based on the image information, module-level optical axis deviation detection is realized. The detection object directly corresponds to each independent projection module, avoiding the mutual coupling and difficulty in distinguishing the optical axis deviations of each module in the integrated detection. The problem was addressed by generating adjustment commands for each projection module and outputting them to the corresponding actuators. This enabled the independent adjustment of the optical axes of each module, allowing for targeted pose adjustments based on their respective optical axis deviations. This avoided issues such as misalignment, blurred edges, or inability to form a complete image that might occur with overall focusing. Furthermore, by verifying the adjusted projection modules' images and obtaining verification results, and confirming calibration completion based on these results, a closed-loop confirmation of the calibration effect was achieved. This ensured that the calibration process only ended after the optical axes of each projection module met the calibration requirements, guaranteeing the image quality of multi-module spliced ​​projection. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of a projection calibration method disclosed in an embodiment of this application;

[0043] Figure 2 This is a schematic diagram illustrating an embodiment of the present application with and without optical axis deviation;

[0044] Figure 3 This is a flowchart of a projection calibration method disclosed in an embodiment of this application;

[0045] Figure 4 This is a schematic diagram of a projection module including multiple driving units disclosed in an embodiment of this application;

[0046] Figure 5 This is a flowchart of a projection calibration method disclosed in an embodiment of this application;

[0047] Figure 6 This is a schematic diagram of the structure of a projection calibration system disclosed in an embodiment of this application;

[0048] Figure 7 The following is a schematic diagram of the focusing logic flow of a projection calibration system disclosed in an embodiment of this application. Detailed Implementation

[0049] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0050] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0051] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0052] This application discloses a projection calibration method, the flowchart of which is shown below. Figure 1 As shown, it includes:

[0053] Step S11: Obtain image information of the projection area corresponding to each of the multiple projection modules through the sensor;

[0054] Step S12: Determine the optical axis deviation information of each projection module based on the image information;

[0055] Step S13: Generate adjustment instructions for each projection module based on the optical axis deviation information of each projection module, and output the adjustment instructions for each projection module to the actuator corresponding to the projection module so that the actuator can adjust the pose of the projection module.

[0056] Step S14: Verify the image information of the projection area corresponding to the projection module after the pose adjustment, and obtain the verification result;

[0057] Step S15: In response to the verification result indicating that the optical axis of the projection module after pose adjustment meets the calibration requirements, determine that the current calibration is complete.

[0058] This embodiment discloses a projection calibration method applied to a projection calibration system. The projection calibration system can be a calibration system corresponding to a projection structure, or a calibration system in a distributed micro-display array projection structure. The calibration system of the distributed micro-display array projection structure can be a subsystem in a vehicle lighting projection system based on multiple Micro-LED units. In this case, the projection calibration system can be the main control subsystem of the vehicle lighting projection system.

[0059] This embodiment uses a projection calibration system with a distributed micro-display array projection structure as an example. The projection calibration system includes multiple projection modules, which can be Micro-LED projection modules. Each projection module is an independent display unit and can independently display a complete projection image. Alternatively, the projection images displayed by each projection module can be optically spliced ​​to form a complete road projection image, thereby achieving large-size, high-resolution projection.

[0060] The projection calibration system also includes multiple sensors, with a corresponding sensor installed at the location of each projection module, so that the image information of the projection area of ​​each projection module can be detected through the sensors.

[0061] The sensor can be a high-resolution global calibration camera, such as a high-precision CCD vision sensor, which is set in the projection optical path of each projection module or near the projection area to collect the projected image projected by the corresponding projection module, analyze the projected image to determine the image information of the projected image, which can be data such as image contrast and sharpness, or information such as the geometric features and brightness distribution of the projected image.

[0062] To improve the accuracy of image acquisition, a narrow-band pass filter can be installed at the front end of the sensor to filter out interfering wavelengths in ambient light, ensuring the clarity of the acquired calibration pattern image. The narrow-band pass filter only allows light signals of the laser calibration wavelength to pass through, thus filtering out stray ambient light interference. Alternatively, the calibration pattern projected by each projection module can be encoded using structured light. Structured light encoding includes dual-frequency phase demodulation encoding, which improves the robustness and accuracy of feature point detection in the image by fusing low-frequency coarse positioning with high-frequency fine positioning.

[0063] Specifically, when the system starts up, each projection module projects a preset calibration pattern, which can be a cross pattern, a checkerboard pattern, or a solid color block pattern, etc. Each sensor synchronously collects image information of the corresponding projection area and transmits the collected image information to the controller.

[0064] The controller receives image information transmitted from various sensors, analyzes and processes the image information, and extracts feature parameters such as the position of feature points, edge positions, and overall sharpness of the projected images from each projection module. Then, it compares the extracted feature parameters with preset reference parameters to calculate the deviation of the current optical axis of each projection module from the ideal optical axis position, i.e., the optical axis deviation information. The optical axis deviation information can include at least the direction and amount of the optical axis offset from the ideal position for each projection module. In practical applications, the optical axis deviation information can manifest as horizontal alignment error and vertical index error. These deviations directly reflect the positional offset of the projected patterns from each projection module and the degree of misalignment of the splicing boundaries.

[0065] For example, it can detect the lateral and vertical offsets of the center point of the cross pattern projected by each projection module relative to its theoretical position, as well as the pixel overlap or gap width at the splicing boundary of the patterns projected by adjacent modules. Figure 2 The diagram shown illustrates the presence and absence of optical axis deviation. Figure 2 The upper half represents the situation where there is optical axis deviation. In this case, the center point of the cross pattern deviates from its theoretical position, resulting in an unclear projected light pattern. Figure 2 The lower half represents the case where there is no optical axis deviation. In this case, the center point of the cross pattern coincides with the theoretical position, and the projected light pattern is clear.

[0066] Specifically, image sharpness evaluation functions can be used to analyze the acquired images to quantify the deviation status of each projection module. For example, evaluation functions such as gradient operators, Laplacian operators, variance functions, or entropy functions can be used to calculate the image sharpness index of the projection area of ​​each projection module. The higher the image sharpness, the closer the optical axis focusing state of the module is to the ideal state.

[0067] Based on the optical axis deviation information of each projection module, the controller calculates the required pose adjustment amount for each projection module using a preset algorithm, and generates adjustment instructions containing the pose adjustment amount. The adjustment instructions include the target displacement or target angle, as well as control parameters such as adjustment speed and adjustment direction.

[0068] Each projection module is equipped with an actuator. The actuator receives adjustment commands from the controller and drives the corresponding projection module to adjust its posture according to the adjustment commands. The actuator includes an electric drive component, which can convert electrical signals into mechanical displacement or angle changes to achieve active adjustment of the optical axis position of the projection module.

[0069] Adjustment commands for each projection module are generated and output independently. Each actuator independently receives and executes the corresponding adjustment command, thereby adjusting the pose of each projection module and adjusting the optical axis of each projection module. The adjustments to each projection module can be performed simultaneously to ensure efficiency, or, to ensure optimal display quality between projected images from different modules, the different projection modules can be adjusted in a pre-defined order.

[0070] After the actuator completes the pose adjustment, each projection module projects the calibration pattern again. The sensor re-acquires the image information of the adjusted projection area. The controller analyzes the adjusted image information, extracts feature parameters and compares them with the reference parameters to determine whether the optical axis position of each projection module meets the calibration requirements, so as to obtain the verification result.

[0071] The verification results include whether the optical axes of each projection module meet the calibration requirements, i.e., whether the optical axis deviation of each projection module is within the preset allowable range, and whether the splicing quality of the projected image meets the requirements. If the verification results indicate that the optical axes meet the calibration requirements, and the verification results of each projection module indicate that the calibration requirements are met, then the current calibration can be determined to be complete. At this time, each projection module can maintain its current position and display normally.

[0072] The projection calibration method disclosed in this embodiment achieves independent detection and adjustment of the optical axis of each projection module by configuring sensors and actuators for each module. This solves the problem of the inability to independently adjust the optical axis of multiple modules caused by mechanical fixing structures and integrated focusing mechanisms. By obtaining image information of the projection area corresponding to each projection module through sensors, and determining the optical axis deviation information of each module based on the image information, module-level optical axis deviation detection is achieved. The detection object directly corresponds to each independent projection module, avoiding the problem of mutual coupling and difficulty in distinguishing the optical axis deviations of each module in integrated detection. Each projection module generates adjustment commands and outputs them to the corresponding actuators, enabling separate adjustment of the optical axes of each module. The adjustment process of each module is independent of each other, and targeted pose adjustments can be made according to the deviation of their respective optical axes. This avoids problems such as misalignment of spliced ​​images, blurred edges, or inability to form a complete image that may occur in overall focusing. By verifying the images of the projection modules after pose adjustment, the verification results are obtained, and the calibration is determined to be complete based on the verification results. This achieves closed-loop confirmation of the calibration effect, ensuring that the calibration process ends only after the optical axes of each projection module meet the calibration requirements, thus guaranteeing the image quality of multi-module spliced ​​projection.

[0073] This embodiment discloses a projection calibration method, the flowchart of which is shown below. Figure 3 As shown, it includes:

[0074] Step S31: Obtain image information of the projection area corresponding to each of the multiple projection modules through the sensor;

[0075] Step S32: Determine the optical axis deviation information of each projection module based on the image information;

[0076] Step S33: Based on the optical axis deviation information of each projection module, determine the pose adjustment amount of each degree of freedom in the multi-degree of freedom corresponding to each projection module, and generate the pose adjustment command containing the multi-degree of freedom corresponding to each projection module.

[0077] Step S34: Control the actuator corresponding to each projection module based on the adjustment command of each projection module, so that the actuator can perform multi-degree-of-freedom pose adjustment;

[0078] Step S35: Verify the image information of the projection area corresponding to the projection module after the pose adjustment, and obtain the verification result;

[0079] Step S36: In response to the verification result indicating that the optical axis of the projection module after the pose adjustment meets the calibration requirements, determine that the current calibration is complete.

[0080] Image information of the projection area corresponding to each of the multiple projection modules is obtained by sensors to determine the optical axis deviation of each module. Based on this deviation, adjustment commands are generated for each module and output to the corresponding actuator. The actuator adjusts the pose of the projection module, and the image information of the projection area after adjustment is verified to obtain a result. When the verification result indicates that the optical axis of the adjusted module meets the calibration requirements, the calibration is considered complete. By configuring sensors and actuators separately for each projection module, independent detection and adjustment of the optical axis of each module are achieved. By generating adjustment commands for each module and outputting them to the corresponding actuator, the optical axis of each module can be adjusted independently. The adjustment process for each module is independent, allowing for targeted pose adjustments based on its own optical axis deviation, thus ensuring optimal image display.

[0081] Specifically, based on the optical axis deviation information of each projection module, adjustment commands are generated for each projection module. These adjustment commands are then output to the actuators corresponding to the projection modules. This process involves: determining the pose adjustment amount for each degree of freedom in the multi-degree-of-freedom operation for each projection module based on its optical axis deviation information; generating adjustment commands for each projection module that include the pose adjustment amounts for the multi-degree-of-freedom operation; and controlling the actuators corresponding to each projection module based on these adjustment commands, so that the actuators can perform multi-degree-of-freedom pose adjustments.

[0082] After obtaining image information of the projection area corresponding to each projection module through sensors, the optical axis deviation information of each projection module is determined through image analysis and processing. Based on this, the pose adjustment amount of each projection module in each degree of freedom is determined according to the optical axis deviation information of each projection module.

[0083] The sensor can be a sub-nanometer grating sensor or a high-precision encoder. The resolution of the sub-nanometer grating sensor is not less than 10nm. It is used to monitor the actual displacement in each degree of freedom in real time. Based on the real-time monitoring data of the sensor, the controller dynamically compensates for the displacement deviation caused by temperature drift, mechanical deformation and external interference through PID algorithm. The controller corrects and adjusts the command according to the PID calculation result so that the actual pose approaches the target pose.

[0084] The optical axis deviation of each projection module may consist of composite offsets in multiple directions. For example, a projection module may simultaneously exhibit translational deviation along the X-axis (i.e., left-right offset), translational deviation along the Z-axis (i.e., forward-backward offset, i.e., defocus), and rotational deviation around the Y-axis (i.e., pitch). Therefore, it is necessary to decouple and decompose the optical axis deviation information into independent adjustment amounts in each degree of freedom direction. Specifically, the optical axis deviation information of each projection module can be determined through kinematic calculations as the required adjustment amounts for each projection module in six degrees of freedom directions. The six degrees of freedom may include: translational degree of freedom along the X-axis, translational degree of freedom along the Y-axis, translational degree of freedom along the Z-axis, rotational degree of freedom around the X-axis, rotational degree of freedom around the Y-axis, and rotational degree of freedom around the Z-axis.

[0085] Taking a projection module as an example, suppose its optical axis deviation information is as follows: the horizontal direction is offset to the right by Δx (corresponding to the X-axis translation deviation), the projected image has a pitch angle Δα relative to the ideal horizontal plane (corresponding to the Y-axis rotation deviation), and the decrease in image clarity indicates a defocus offset Δz (corresponding to the Z-axis translation deviation). Based on this deviation information, the pose adjustment amount of the projection module in the X-axis direction is determined to be -Δx (translation to the left), the pose adjustment amount in the Y-axis direction is -Δα (reverse rotation to correct pitch), and the pose adjustment amount in the Z-axis direction is -Δz (movement back and forth to restore the focal plane).

[0086] Based on this, adjustment instructions containing the aforementioned multi-degree-of-freedom pose adjustment amounts are generated for each projection module. These adjustment instructions include the adjustment direction and amount information for each projection module in each degree of freedom direction, encapsulated in a data structure, and then sent to the corresponding actuator. Specifically, if a projection module has no offset in a certain direction, then there is no pose adjustment amount in that direction, or the pose adjustment amount in that direction is 0.

[0087] Each projection module is equipped with an actuator. The output of the actuator is mechanically connected to the corresponding projection module. When the actuator receives an adjustment command, it drives the corresponding projection module to generate displacement in the corresponding degree of freedom direction according to the adjustment amount and direction in each degree of freedom direction in the adjustment command.

[0088] The actuator is configured to independently drive the corresponding projection module in multiple degrees of freedom directions. Taking a six-degree-of-freedom actuator as an example, it includes an X-axis drive unit that translates along the X-axis, a Y-axis drive unit that translates along the Y-axis, a Z-axis drive unit that translates along the Z-axis, an RX-axis drive unit that rotates around the X-axis, an RY-axis drive unit that rotates around the Y-axis, and an RZ-axis drive unit that rotates around the Z-axis. Each drive unit is connected to the corresponding projection module and can operate independently under the command of the controller. Figure 4The diagram shows a projection module comprising multiple driving units, including: an X-axis driving unit 41, an RX-axis driving unit 42, a Y-axis driving unit 43, an RY-axis driving unit 44, a Z-axis driving unit 45, and an RZ-axis driving unit 46.

[0089] When controlling the actuator to perform multi-degree-of-freedom adjustments, the adjustments in each degree-of-freedom direction can be performed simultaneously or sequentially according to a preset priority. To improve adjustment efficiency, the adjustments in each degree-of-freedom direction can be controlled to be performed synchronously. In this case, the controller will simultaneously send instructions containing all six degrees of freedom adjustment amounts to each drive unit of the actuator, and each drive unit will act simultaneously to complete the pose correction of the projection module in a short time. Alternatively, to avoid mutual interference between adjustments in different degrees of freedom directions, adjustments in different degrees of freedom directions can be performed separately according to a preset order, such as translation followed by rotation, or coarse adjustment followed by fine adjustment, etc.

[0090] For example, the adjustment sequence for different degrees of freedom can be: adjust along the horizontal direction of the X-axis → adjust along the rotation direction of the Z-axis → adjust along the horizontal direction of the Y-axis → adjust along the rotation direction of the Y-axis → adjust along the horizontal direction of the Z-axis → adjust along the rotation direction of the X-axis.

[0091] When the controller transmits the adjustment commands corresponding to each projection module to the corresponding actuator, it can transmit them synchronously through a Time-Sensitive Network (TSN). The TSN adopts the IEEE 802.1AS time synchronization protocol to ensure that the transmission delay of the control commands corresponding to multiple projection modules is less than 1ms, so that the pose adjustment actions of each projection module are consistent in timing and the optical axis is synchronized.

[0092] The projection calibration method disclosed in this embodiment decomposes the optical axis deviation information into deviation components of multiple degrees of freedom and calculates the pose adjustment amount independently for each degree of freedom. This achieves fine decomposition and targeted compensation of the optical axis deviation of the projection module. Compared with overall coarse adjustment, multi-degree-of-freedom independent adjustment can include the offset and tilt of the optical axis in any direction within the compensation range, effectively solving the deviation of the optical axis in different directions and significantly improving the alignment accuracy of the optical axis of each projection module. By generating adjustment commands containing multi-degree-of-freedom pose adjustment amounts and outputting them to the actuators corresponding to each projection module, independent control and precise issuance of pose adjustment for each projection module are achieved. Each projection module executes multi-degree-of-freedom pose adjustment according to its corresponding adjustment command, avoiding mutual coupling of adjustment actions between multiple projection modules. This ensures that each projection module can perform optimal pose configuration according to its own optical axis state, improving the efficiency and accuracy of calibration.

[0093] Furthermore, in the projection calibration method disclosed in this embodiment, controlling the actuator corresponding to each projection module based on the adjustment command of each projection module can be specifically as follows:

[0094] Based on the adjustment instructions for each projection module, the motor drive module in the control actuator performs micron-level adjustments to the projection module; the electrostriction module in the control actuator performs nanon-level adjustments to the projection module.

[0095] Each actuator can consist of a motor drive module and an electrostriction module. The controller can decompose the pose adjustment amount into a coarse adjustment component suitable for the motor drive module and a fine adjustment component suitable for the electrostriction module, according to the magnitude and accuracy requirements of the pose adjustment amount of each degree of freedom. First, the motor drive module performs micron-level adjustment to achieve coarse adjustment, and then the electrostriction module performs nanon-level adjustment for the remaining part to achieve fine adjustment.

[0096] The driving method of the motor drive module can be a stepper motor with mechanical transmission mechanisms such as gears and lead screws, or a servo motor with precision lead screws or ball screws. The motor drive module has a large stroke driving capability and can drive the projection module to produce displacement within a large range.

[0097] The motor drive module can achieve micron-level positioning accuracy and is used to perform preliminary correction of the pose of the projection module. Specifically, based on the pose adjustment amount in each degree of freedom direction in the adjustment command, the motor drive module can be controlled to generate movement in the corresponding degree of freedom direction, driving the projection module to move along the corresponding direction, so as to initially eliminate the optical axis deviation of each module.

[0098] The motor drive module has a large stroke and strong driving force, which can cover the large initial position deviation of each projection module caused by processing errors, assembly errors, etc. However, due to the precision limitations of the mechanical transmission mechanism itself, such as gear backlash and lead screw error, the adjustment accuracy of the motor drive module is usually limited to the micrometer level, which is difficult to meet the fine adjustment requirements of the sub-micrometer or nanometer level.

[0099] Therefore, after the motor drive module completes micron-level coarse adjustment, the electrostrictive module in the control actuator performs nanometer-level adjustment to the projection module. The electrostrictive module is a precision drive unit that operates using the inverse piezoelectric effect of electrostrictive materials. Its basic principle is that when an electric field is applied to the electrostrictive material, the internal lattice structure of the electrostrictive material undergoes a slight deformation, thereby producing an extremely small but controllable displacement output on a macroscopic scale. This displacement output can achieve nanometer-level precision, enabling position adjustment with higher precision than that of the motor drive module.

[0100] Based on the remaining pose adjustment amounts in each degree of freedom direction in the adjustment command, which represent the small deviations still existing after the coarse adjustment of the motor drive module, the electrostrictive module is controlled to generate compensating displacements in the corresponding degree of freedom direction, further correcting the pose of the projection module to the sub-micron level accuracy. The electrostrictive module boasts extremely high precision and fast response speed, but its stroke is typically small. Therefore, in a hybrid drive architecture, the electrostrictive module is mainly used for fine correction based on the coarse adjustment of the motor drive module.

[0101] Among them, the electrostrictive module can use low-temperature stable piezoelectric ceramic materials, such as a 125-layer PZT ceramic stack structure. The PZT ceramic stack structure is combined with a thermal expansion compensation structure. The thermal expansion compensation structure is made of materials with matching thermal expansion coefficients to offset the performance drift of piezoelectric ceramics caused by temperature changes, and control the performance loss caused by temperature to within 10%.

[0102] The electrostrictive module adopts an oil-free design, and the motion transmission interface of the drive component does not use grease lubrication, avoiding friction fluctuations caused by changes in grease viscosity and contamination. The oil-free design, combined with nanoscale surface treatment processes, such as honing technology, makes the surface roughness of the motion transmission interface reach the nanoscale, reducing friction errors caused by micro-undulations on the surface and ensuring the smoothness and repeatability of posture adjustment.

[0103] In addition, the motor drive module corresponds to the global layer drive component, used to handle changes in the overall projection distance. By adjusting the position of the main lens group through the motor drive module, coarse adjustment of the projection distance for all projection modules is achieved. The electrostriction module corresponds to the local layer drive component, used to perform sub-micron level compensation for minute deviations in individual projection modules. It independently adjusts the pose of the corresponding projection module through micro actuators. The global layer drive component and the local layer drive component work together. The global layer first completes the overall projection distance adjustment, and then the local layer performs fine compensation within the independent range of each projection module.

[0104] Furthermore, in the fine-tuning process, it can be specifically described as follows:

[0105] Based on the adjustment command of each projection module, the driving component in the electrostrictive module is controlled to generate electric field-driven deformation to output nanometer-level displacement; based on the inverse model of the hysteresis characteristic of the driving component, the input signal of the driving component is nonlinearly compensated to counteract the displacement deviation caused by the hysteresis characteristic of the driving component; through the mechanical amplification effect of the flexible hinge in the electrostrictive module, the nanometer-level displacement is converted into micrometer-level displacement to achieve the adjustment of the pose of the projection module.

[0106] The electrostrictive module is equipped with a drive component, which is made of an electrostrictive material with inverse piezoelectric effect, such as piezoelectric ceramic material. When an electric field is applied to the drive component, the internal lattice structure of the drive component undergoes a slight deformation, thereby producing a very small but controllable displacement output on a macroscopic scale.

[0107] After coarse adjustment, the remaining pose adjustment amount is determined, and a fine adjustment control signal for the electrostrictive module is generated based on the remaining pose adjustment amount. This fine adjustment control signal is applied to the electrodes of the driving component in the form of voltage. The driving component generates inverse piezoelectric deformation under the action of electric field. The deformation amount is linearly related to the applied electric field strength, which can achieve nanometer-level displacement output accuracy.

[0108] During the nanometer-level adjustment of the drive components in the electrostrictive module, when the drive speed is detected to be lower than the preset low-speed threshold or when the direction of motion is reversed, the controller detects the speed change of the drive components in real time through the encoder. In response to the stick-slip effect characterized by the speed change, the controller outputs a compensating piezoelectric drive signal in reverse to counteract the displacement jump caused by the stick-slip effect, ensuring the smoothness of the electrostrictive module under low-speed and reversing conditions.

[0109] Although the drive component can achieve displacement output with nanometer-level precision, its output stroke is usually small, which is difficult to meet the stroke requirements required for the pose adjustment of the projection module. At this time, a flexible hinge mechanism can be set between the drive component and the projection module. The flexible hinge mechanism is a frictionless transmission mechanism that uses the elastic deformation of materials to achieve precision motion. Unlike traditional mechanical hinges, flexible hinges have no mechanical backlash and friction, and can achieve smooth and backlash-free displacement transmission.

[0110] The output end of the drive component is mechanically connected to the input end of the flexible hinge mechanism, and the output end of the flexible hinge mechanism is mechanically connected to the projection module. The nanometer-level displacement generated by the drive component is input to the flexible hinge mechanism and, through mechanical amplification, is converted into a micrometer-level stroke output to the projection module. The flexible hinge mechanism can employ either a lever amplification mechanism or a bridge amplification mechanism. The lever amplification mechanism utilizes the lever principle and achieves displacement amplification by setting the lever arm ratio; the bridge amplification mechanism utilizes the geometric configuration of an elastic deformable body to achieve displacement amplification.

[0111] The micron-level displacement amplified by the flexible hinge mechanism is transmitted to the projection module, driving the projection module to generate fine-tuning displacement in the corresponding degree of freedom direction, thus completing the fine compensation for the remaining submicron-level deviation after the coarse adjustment of the motor drive module.

[0112] Furthermore, due to the inherent hysteresis nonlinearity of the drive component, that is, the same voltage value corresponds to different output displacement during the rise and fall of the drive voltage, forming a hysteresis curve, this nonlinearity will affect the positioning accuracy and repeatability of the drive component. Therefore, it is necessary to compensate for the nonlinearity of the drive component at the signal input stage.

[0113] Specifically, firstly, an inverse model of the hysteresis characteristics of the drive component is established. This inverse model is based on historical drive data of the piezoelectric ceramic drive component and describes the nonlinear mapping relationship between the drive voltage and the output displacement, including independent mapping functions for the voltage rise branch and the voltage fall branch. Before performing nanoscale adjustments, the controller inputs the target displacement to the inverse model. The inverse model calculates the compensated drive voltage value based on the current drive state and historical displacement output. The current drive state is either the voltage rise or fall phase. The compensated drive voltage value differs from the uncompensated linearly calculated voltage value, and the difference is used to offset the hysteresis nonlinearity of the piezoelectric ceramic material, making the actual output displacement approach the target displacement.

[0114] This embodiment achieves separate micron-level and nanon-level adjustments in pose adjustment through a collaborative drive design of a motor drive module and an electrostriction module. The motor drive module, with its large stroke and fast response, undertakes the micron-level coarse adjustment task, quickly driving the projection module to the vicinity of the target pose, significantly shortening the overall adjustment time. The electrostriction module, with its nanon-level resolution and lack of mechanical inertia, undertakes the nanon-level fine adjustment task, eliminating residual deviations on the basis of coarse adjustment and achieving sub-micron-level final accuracy. The collaboration between the two not only meets the stroke requirements of large-range pose adjustment but also achieves the sub-micron-level alignment accuracy required for pixel-level stitching, improving calibration efficiency.

[0115] This embodiment discloses a projection calibration method, the flowchart of which is shown below. Figure 5 As shown, it includes:

[0116] Step S51: Obtain image information of the projection area corresponding to each of the multiple projection modules through the sensor;

[0117] Step S52: Determine the optical axis deviation information of each projection module based on the image information;

[0118] Step S53: Obtain environmental perception information, which includes at least one of the following: temperature data and vibration acceleration data;

[0119] Step S54: Input the environmental perception information into the prediction model, and determine the predicted value of the optical axis shift caused by the environmental disturbance during the target time period through the prediction model. The target time period is the time period after the current time.

[0120] Step S55: Generate feedforward compensation information based on the optical axis offset prediction value;

[0121] Step S56: Based on the optical axis deviation information and feedforward compensation information of each projection module, generate adjustment instructions corresponding to each projection module, and output the adjustment instructions corresponding to each projection module to the actuator corresponding to the projection module so that the actuator can adjust the pose of the projection module.

[0122] Step S57: Verify the image information of the projection area corresponding to the projection module after the pose adjustment, and obtain the verification result;

[0123] Step S58: In response to the verification result indicating that the optical axis of the projection module after the pose adjustment meets the calibration requirements, determine that the current calibration is complete.

[0124] Image information of the projection area corresponding to each of the multiple projection modules is obtained by sensors to determine the optical axis deviation of each module. Based on this deviation, adjustment commands are generated for each module and output to the corresponding actuator. The actuator adjusts the pose of the projection module, and the image information of the projection area after adjustment is verified to obtain a result. When the verification result indicates that the optical axis of the adjusted module meets the calibration requirements, the calibration is considered complete. By configuring sensors and actuators separately for each projection module, independent detection and adjustment of the optical axis of each module are achieved. By generating adjustment commands for each module and outputting them to the corresponding actuator, the optical axis of each module can be adjusted independently. The adjustment process for each module is independent, allowing for targeted pose adjustments based on its own optical axis deviation, thus ensuring optimal image display.

[0125] The process of generating adjustment instructions can be specifically described as follows: obtaining environmental perception information, which includes at least one of the following: temperature data and vibration acceleration data; inputting the environmental perception information into the prediction model, and determining the predicted value of the optical axis offset caused by environmental disturbance during the target time period through the prediction model, wherein the target time period is the time period after the current time; generating feedforward compensation information based on the predicted value of optical axis offset; and generating adjustment instructions corresponding to each projection module based on the optical axis deviation information and feedforward compensation information of each projection module.

[0126] In a projection calibration system, each projection module generates heat during operation, leading to temperature changes. Simultaneously, road bumps and engine vibrations during vehicle operation are also transmitted to the projection modules. These temperature changes and vibration excitations cause thermal expansion of the optical components, deformation of the mechanical structure, and slight displacement of the mounting base in each projection module, resulting in dynamic drift of the optical axis of each projection module. Due to the time delay in temperature conduction and vibration response, the impact of the above-mentioned environmental disturbances on the optical axis is not instantaneous but has a certain lag effect. Therefore, relying solely on image information to determine the optical axis deviation has a response lag problem.

[0127] Therefore, in this embodiment, environmental sensors are installed in the system to obtain environmental perception information, which may include at least one of the following: temperature data and vibration acceleration data. Temperature sensors can be installed at the mounting base or housing of each projection module to monitor the ambient temperature of each projection module in real time. The temperature data output by the temperature sensors reflects the current temperature state of each projection module and is a basic input parameter for predicting future temperature-induced optical axis drift. Vibration acceleration sensors can also be installed at the mounting base of each projection module to monitor the vibration excitation experienced by each projection module in real time. The vibration acceleration data output by the vibration acceleration sensors reflects the current vibration state of each projection module, including vibration amplitude and vibration direction, and is a basic input parameter for predicting future vibration-induced optical axis drift.

[0128] After obtaining environmental perception information, this information is input into the prediction model. The prediction model determines the predicted optical axis offset for the target time period caused by environmental disturbances. The target time period is the period after the current moment, such as within the next 5 seconds. The prediction model can employ an Autoregressive Integral Moving Average (ARIMA) model, a time series prediction model suitable for predicting time series data such as temperature and vibration. Alternatively, a state estimation model can be used, including a Kalman filter model. The Kalman filter model fuses state prediction with observation updates to estimate the environmental disturbance state and optical axis offset in real time. Furthermore, the controller can utilize FPGA hardware to accelerate the calculation of the prediction model and the generation of adjustment instructions to meet millisecond-level real-time response requirements.

[0129] First, environmental sensing data sequences and corresponding optical axis offsets within a preset historical time period can be collected as training data. For example, temperature data and / or vibration acceleration data and corresponding optical axis offsets within the past minute can be collected to construct a time-series data sequence, which is organized in a timestamp-synchronized manner. Then, based on the collected historical environmental sensing data sequences, the ARIMA(p,d,q) model is trained to determine the autoregressive order p, differencing order d, and moving average order q of the ARIMA model. The differencing order d is used to eliminate the non-stationarity of the data, while the autoregressive order p and moving average order q are used to describe the temporal correlation of the data.

[0130] After the prediction model is trained, it is used to predict the optical axis offset for the target time period. The predicted optical axis offset for the target time period is due to the offset caused by environmental perception information. That is, the currently detected environmental perception information is input into the prediction model to obtain the predicted value of the optical axis offset for the target time period output by the prediction model. Then, based on the current optical axis position and the predicted value of the optical axis offset, the feedforward compensation information of the optical axis can be determined. This feedforward compensation information is obtained based on the environmental perception information and is used to drive the actuator to perform reverse compensation in advance before the optical axis actually offsets, so as to counteract the influence of the predicted environmental disturbance. It is also necessary to combine the optical axis deviation information determined based on the currently acquired image information to determine the adjustment command that actually needs to be calibrated. That is, the generation of the adjustment command must refer to both the feedforward compensation information obtained based on the environmental perception information and the optical axis deviation information determined based on the currently acquired image information to ensure the accuracy of the adjustment.

[0131] Specifically, it can be represented by the following formula:

[0132]

[0133] in, Here, K represents the predicted optical axis offset, and K is a pre-set compensation coefficient. For feedforward compensation information, For optical axis deviation information, The comprehensive adjustment amount is the pose adjustment amount corresponding to the adjustment command, which is determined based on the optical axis offset prediction value and feedforward compensation information.

[0134] Furthermore, the projection calibration method disclosed in this embodiment may also include:

[0135] In response to the rate of change of the environmental perception information exceeding the target threshold, the environmental perception information is input into the prediction model in order to determine the predicted value of the optical axis offset.

[0136] During system operation, the environmental sensing sensors continuously collect environmental sensing information from each projection module, including temperature data and vibration acceleration data. The environmental sensing sensors send the collected environmental sensing information to the controller, which processes the obtained environmental sensing information in real time to determine the rate of change of the environmental sensing information.

[0137] For temperature data, the rate of change of temperature over time is calculated, which is the amount of change in temperature value per unit time. The rate of change of temperature is calculated by the temperature difference between adjacent sampling times, for example, dT / dt = [T(t) - T(t-Δt)] / Δt, where dT / dt is the rate of change of temperature over time, T is the sampling temperature at time t, T(t-Δt) is the sampling temperature at time t-Δt, and Δt is the sampling interval. The rate of change of temperature reflects the speed and trend of temperature change in each projection module. The faster the temperature rise rate, the more intense the heat generation of the projection module or the more obvious the change in heat dissipation conditions, and the higher the risk of thermally induced optical axis drift.

[0138] For vibration acceleration data, calculate the rate of change of vibration acceleration amplitude over time, or directly detect whether the vibration acceleration amplitude exceeds the preset amplitude threshold. The vibration acceleration amplitude reflects the intensity of vibration excitation currently experienced by each projection module. The larger the amplitude, the more severe the vibration, and the higher the risk of vibration-induced optical axis drift.

[0139] After calculating the rate of change of the environmental perception information, the rate of change is compared with a preset target threshold to determine whether the rate of change exceeds the target threshold. The target threshold is the criterion for triggering feedforward compensation. Different target thresholds can be set for different types of environmental perception information.

[0140] For example, the target threshold for the temperature change rate can be set to 0.5℃ / min. When the temperature change rate is greater than 0.5℃ / min, it indicates that the temperature is changing rapidly, the thermal state of the projection module is changing significantly, and thermally induced optical axis drift will occur in a short time. Feedforward compensation needs to be activated for correction. That is, the optical axis offset prediction value for the target time period needs to be output by the prediction model, and feedforward compensation information is generated based on the optical axis offset prediction value. The feedforward compensation information and the optical axis deviation information are used together to determine the adjustment command. Conversely, when the temperature change rate is less than or equal to 0.5℃ / min, it indicates that the temperature changes slowly or tends to be stable. The risk of thermally induced optical axis drift is low, and feedforward compensation can be eliminated. The calibration accuracy can be maintained by relying solely on feedback control based on the optical axis deviation information.

[0141] For example, the target threshold for vibration acceleration amplitude can be set to 0.1g. When the vibration acceleration amplitude is greater than 0.1g, it indicates that the projection module is experiencing significant vibration excitation, and the risk of vibration-induced optical axis drift is high, requiring feedforward compensation to be activated for correction. Conversely, when the vibration acceleration amplitude is less than or equal to 0.1g, it indicates that the vibration excitation is small, and feedforward compensation does not need to be activated.

[0142] The specific values ​​of the target thresholds mentioned above are for illustrative purposes only and are not intended to be limiting. In practical applications, the specific values ​​of the target thresholds can be determined based on the thermal and mechanical characteristics of the projection module and the calibration accuracy requirements.

[0143] The projection calibration method disclosed in this embodiment introduces environmental perception information during the generation of adjustment instructions, and establishes a predictive compensation channel from environmental disturbance to optical axis offset. This breaks through the limitation of traditional calibration methods that rely solely on image feedback for post-correction. By using a predictive model to perceive the trend of environmental disturbance in advance, feedforward compensation information is generated before the optical axis actually shifts significantly, thus realizing the pre-positioning of disturbance compensation and significantly shortening the response time from environmental change to compensation execution.

[0144] Furthermore, the projection calibration method disclosed in this embodiment may also include:

[0145] In response to the verification results indicating that the optical axis of the projection module after pose adjustment does not meet the calibration requirements, the optical axis deviation information is re-determined based on the image information of the projection module after pose adjustment, and the adjustment command is re-generated based on the regenerated optical axis deviation information so that the actuator can readjust the pose of the projection module.

[0146] After the actuator adjusts the pose of the projection module according to the adjustment command, the system verifies the image information of the projection area corresponding to the adjusted projection module and obtains the verification result. Based on the verification result, the controller determines whether the optical axis of the adjusted projection module meets the calibration requirements, which may include the allowable range of optical axis deviation. Specifically, for each projection module, it is determined whether the optical axis deviation in multiple degrees of freedom is within the preset allowable range of optical axis deviation. The preset allowable range of optical axis deviation is determined based on pixel-level stitching accuracy requirements. For example, the optical axis deviation in each translational degree of freedom is less than 1 / 2 pixel size, and the optical axis deviation in each rotational degree of freedom is less than the corresponding angular pixel resolution.

[0147] The verification process involves acquiring images of the adjusted projection area using a global calibration camera, comparing the acquired images with a preset reference image or preset calibration pattern, and comparing the following: feature point position overlap, edge alignment, image clarity, and brightness consistency of the splicing area. The verification results are presented in the form of quantitative indicators, such as: residual deviation of each degree of freedom, image clarity score, and splicing gap width.

[0148] In response to the verification results indicating that the optical axes of the adjusted projection modules meet the calibration requirements, the current calibration is determined to be complete. Meeting the calibration requirements specifically means that the optical axis deviation of each projection module in each degree of freedom is within a preset allowable range, and the image clarity and stitching quality meet preset standards. If the current calibration is complete, the next calibration can be initiated after a certain interval, meaning a calibration is initiated at regular intervals during the projection process to ensure that the projected image maintains a consistently good presentation effect; alternatively, the system will only enter normal projection operation mode after the current calibration is completed, meaning calibration is only performed before projection.

[0149] In response to the verification results indicating that the optical axis of the adjusted projection module does not meet the calibration requirements, the optical axis deviation information is re-determined based on the image information of the adjusted projection module. The failure to meet the calibration requirements can specifically be defined as follows: the optical axis deviation of at least one projection module in at least one degree of freedom exceeds the preset allowable range for optical axis deviation, or the image clarity or stitching quality does not meet preset standards. When it is determined that the calibration requirements are not met, the current calibration is not completed, and iterative calibration is triggered.

[0150] In iterative calibration, the controller uses the image information of the projection area corresponding to the adjusted projection module as new input data and re-executes the above-mentioned optical axis deviation determination process. Specifically, the controller performs feature extraction and parameter analysis on the adjusted image information to determine the current actual optical axis deviation of the corresponding projection module. The corresponding projection module is one whose optical axis deviation in at least one degree of freedom exceeds the preset allowable range. The re-determined optical axis deviation information reflects the residual deviation after the previous adjustment, as well as any new errors that may have been introduced during the adjustment process. Based on the re-determined optical axis deviation information, the controller regenerates the adjustment instructions for each projection module according to the above steps. The regenerated adjustment instructions only... For projection modules and degrees of freedom that do not meet calibration requirements, or for all projection modules, a global recalibration is performed. The specific strategy is determined by the controller based on the deviation distribution. The newly generated adjustment command is output to the corresponding actuator. The actuator drives the projection module to adjust its pose according to the adjustment command. The recalibration process is the same as the first adjustment. After the recalibration is completed, it is verified again to determine whether the new verification result can guarantee that the optical axis meets the calibration requirements. If it still does not meet the requirements, it is recalibrated until a verification result that meets the calibration requirements is obtained after a certain recalibration. Then the iteration stops. Alternatively, if the requirements are still not met when the number of iterations reaches the maximum value, a prompt message can be output to prompt manual intervention to check for abnormalities.

[0151] The projection calibration method disclosed in this embodiment establishes a closed-loop verification and iterative correction mechanism for calibration effect by re-determining the optical axis deviation information and regenerating adjustment instructions based on the adjusted image information when the verification result indicates that the optical axis does not meet the calibration requirements. Compared with the open-loop calibration method that ends after a single adjustment, the closed-loop iterative mechanism can detect and correct the residual error in a single adjustment, avoiding calibration failure caused by actuator response error, sensor detection error or transient environmental disturbances, and significantly improving the reliability and final accuracy of the calibration results. The re-detection based on the actual image ensures the authenticity and accuracy of the deviation during the iteration process, providing reliable input data for the subsequent regeneration of adjustment instructions, and avoiding error accumulation and false convergence.

[0152] This embodiment discloses a projection calibration system, the schematic diagram of which is shown below. Figure 6 As shown, it includes:

[0153] Multiple projection modules 61, multiple sensors 62, multiple actuators 63, and a controller 64.

[0154] Among them, multiple projection modules 61, each projection module projects separately to form a projection area, and the projection areas formed by the projection of multiple projection modules are stitched together to form a projection image.

[0155] Multiple sensors 62 are used to obtain image information of the projection area corresponding to the projection module at its location.

[0156] Multiple actuators 63 are provided, with an actuator installed at the location of each projection module. The actuators are used to adjust the pose of the projection module.

[0157] The controller 64 is used to determine the optical axis deviation information of each projection module based on the image information, generate adjustment commands corresponding to each projection module based on the optical axis deviation information of each projection module, and output the adjustment commands corresponding to each projection module to the actuator corresponding to the projection module so that the actuator adjusts the pose of the projection module, verifies the image information of the projection area corresponding to the adjusted projection module, and obtains the verification result; in response to the verification result indicating that the optical axis of the adjusted projection module meets the calibration requirements, the controller determines that the current calibration is complete.

[0158] Furthermore, the actuator is a multi-degree-of-freedom adjustment structure.

[0159] The actuator includes: a motor drive module for micron-level adjustment of the projection module, and an electrostrictive module for nanon-level adjustment of the projection module;

[0160] The electrostrictive module includes: a drive component that generates an electric field to drive deformation to output nanoscale displacement, and a flexible hinge that uses mechanical amplification to convert nanoscale displacement into micrometer-scale displacement.

[0161] Furthermore, the projection calibration system disclosed in this embodiment may also include:

[0162] An environmental perception sensor is used to obtain environmental perception information so that the controller can input the environmental perception information into the prediction model. The prediction model determines the predicted value of the optical axis offset caused by environmental disturbances during the target time period, which is the time period after the current time. Feedforward compensation information is generated based on the optical axis offset prediction value. Based on the optical axis deviation information and feedforward compensation information of each projection module, adjustment instructions corresponding to each projection module are generated.

[0163] like Figure 7 The diagram shown illustrates the focusing logic flow of a projection calibration system, which includes three projection modules for adjusting the multi-degree-of-freedom poses of each module. Figure 7 In the diagram, the X direction is the projection direction, the Y direction is the horizontal direction, and the Z direction is the vertical direction. Figure 7 The diagram schematically illustrates the structural arrangement of three projection modules. Each projection module includes a projection chip, a projection lens, and a heat sink. The three projection modules are arranged side by side, and each projection module is positioned and clamped by a clamping structure, which is labeled "clamping" in the diagram. Figure 7 The image also shows a Fresnel lens and an image receiving screen, with the beam of light projected by the projection module being imaged onto the image receiving screen after passing through the Fresnel lens.

[0164] The projection calibration system disclosed in this embodiment is based on the projection calibration method disclosed in the above embodiments, and will not be described again here.

[0165] The projection calibration system disclosed in this embodiment achieves independent detection and adjustment of the optical axis of each projection module by configuring sensors and actuators for each module. This solves the problem of the inability to independently adjust the optical axes of multiple modules caused by mechanical fixing structures and integrated focusing mechanisms. By acquiring image information of the projection area corresponding to each projection module through sensors, and determining the optical axis deviation information of each module based on the image information, module-level optical axis deviation detection is achieved. The detection object directly corresponds to each independent projection module, avoiding the problem of mutual coupling and difficulty in distinguishing the optical axis deviations of each module in integrated detection. Each projection module generates adjustment commands and outputs them to the corresponding actuators, enabling separate adjustment of the optical axes of each module. The adjustment process of each module is independent of each other, and targeted pose adjustments can be made according to the deviation of their respective optical axes. This avoids problems such as misalignment of spliced ​​images, blurred edges, or inability to form a complete image that may occur in overall focusing. By verifying the images of the projection modules after pose adjustment, the verification results are obtained, and the calibration is determined to be complete based on the verification results. This achieves closed-loop confirmation of the calibration effect, ensuring that the calibration process ends only after the optical axes of each projection module meet the calibration requirements, thus guaranteeing the image quality of multi-module spliced ​​projection.

[0166] This application embodiment also provides a readable storage medium storing a computer program thereon. The computer program is loaded and executed by a processor to implement the steps of the above projection calibration method. The specific implementation process can be referred to the description of the corresponding part of the above embodiment, and will not be repeated in this embodiment.

[0167] This application also proposes a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the various optional implementations of the projection calibration method described above. Specific implementation processes can be referred to the descriptions of the corresponding embodiments above, and will not be repeated here.

[0168] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0169] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0170] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0171] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A projection calibration method, characterized in that, include: Image information of the projection area corresponding to each of the multiple projection modules is obtained through sensors; Based on the image information, the optical axis deviation information of each projection module is determined respectively; Based on the optical axis deviation information of each projection module, an adjustment command corresponding to each projection module is generated, and the adjustment command corresponding to each projection module is output to the actuator corresponding to the projection module so that the actuator can adjust the pose of the projection module. The image information of the projection area corresponding to the projection module after the pose adjustment is verified, and the verification results are obtained. In response to the verification result indicating that the optical axis of the projection module after pose adjustment meets the calibration requirements, the current calibration is determined to be complete.

2. The method according to claim 1, characterized in that, The step of generating adjustment commands corresponding to each projection module based on the optical axis deviation information of each projection module, and outputting the adjustment commands corresponding to each projection module to the actuator corresponding to the projection module, includes: Based on the optical axis deviation information of each projection module, the pose adjustment amount of each degree of freedom in the multi-degree of freedom corresponding to each projection module is determined, and an adjustment command containing the pose adjustment amount of the multi-degree of freedom corresponding to each projection module is generated. Based on the adjustment instructions of each projection module, the actuator corresponding to the projection module is controlled so that the actuator can perform multi-degree-of-freedom pose adjustment.

3. The method according to claim 2, characterized in that, The control of the actuator corresponding to each projection module based on the adjustment command of each projection module includes: Based on the adjustment instructions for each projection module, the motor drive module in the actuator is controlled to perform micron-level adjustments to the projection module; The electrostriction module in the actuator is controlled to make nanoscale adjustments to the projection module.

4. The method according to claim 3, characterized in that, The process of controlling the electrostriction module in the actuator to perform nanometer-level adjustments to the projection module includes: Based on the adjustment instructions of each projection module, the driving component in the electrostriction module is controlled to generate electric field-driven deformation to output nanoscale displacement. Based on the inverse model of the hysteresis characteristics of the drive component, nonlinear compensation is performed on the input signal of the drive component to counteract the displacement deviation caused by the hysteresis characteristics of the drive component. The nanoscale displacement is converted into a micrometer-level displacement by the mechanical amplification effect of the flexible hinge in the electrostrictive module, thereby achieving the adjustment of the pose of the projection module.

5. The method according to claim 1, characterized in that, The step of generating adjustment instructions for each projection module based on the optical axis deviation information of each projection module includes: Obtain environmental sensing information, which includes at least one of the following: temperature data and vibration acceleration data; The environmental perception information is input into the prediction model, and the prediction model is used to determine the predicted value of the optical axis offset for the target time period caused by environmental disturbance. The target time period is the time period after the current time. Feedforward compensation information is generated based on the optical axis offset prediction value; Based on the optical axis deviation information of each projection module and the feedforward compensation information, adjustment instructions corresponding to each projection module are generated.

6. The method according to claim 5, characterized in that, Also includes: In response to the rate of change of the environmental perception information exceeding a target threshold, the environmental perception information is input into the prediction model to determine the predicted value of the optical axis offset.

7. The method according to claim 1, characterized in that, Also includes: In response to the verification result indicating that the optical axis of the projection module after the pose adjustment does not meet the calibration requirements, the optical axis deviation information is re-determined based on the image information of the projection module after the pose adjustment, and the adjustment command is re-generated based on the regenerated optical axis deviation information so that the actuator readjusts the pose of the projection module.

8. A projection calibration system, characterized in that, include: Multiple projection modules, each projection module projects separately to form a projection area, and the projection areas formed by the projections of multiple projection modules are stitched together to form a projection image; Multiple sensors are used, with a sensor installed at the location of each projection module. The sensor is used to obtain image information of the projection area corresponding to the projection module at its location. Multiple actuators are provided, with an actuator installed at the location of each projection module. The actuators are used to adjust the pose of the projection module. The controller is configured to determine the optical axis deviation information of each projection module based on the image information, generate adjustment instructions corresponding to each projection module based on the optical axis deviation information of each projection module, output the adjustment instructions corresponding to each projection module to the actuator corresponding to the projection module, so that the actuator adjusts the pose of the projection module, and verify the image information of the projection area corresponding to the adjusted projection module to obtain the verification result. In response to the verification result indicating that the optical axis of the projection module after pose adjustment meets the calibration requirements, the current calibration is determined to be complete.

9. The system according to claim 8, characterized in that, The actuator is a multi-degree-of-freedom adjustable structure. The actuator includes: a motor drive module for micrometer-level adjustment of the projection module, and an electrostriction module for nanometer-level adjustment of the projection module; The electrostrictive module includes: a drive component that generates an electric field to drive deformation to output nanoscale displacement, and a flexible hinge that uses mechanical amplification to convert the nanoscale displacement into micrometer-level displacement.

10. The system according to claim 8, characterized in that, Also includes: An environmental perception sensor is used to obtain environmental perception information so that the controller can input the environmental perception information into a prediction model, and determine the predicted value of the optical axis offset caused by environmental disturbance during a target time period, wherein the target time period is the time period after the current time. Feedforward compensation information is generated based on the optical axis offset prediction value; Based on the optical axis deviation information of each projection module and the feedforward compensation information, adjustment instructions corresponding to each projection module are generated.

Images