Projection multi-dimensional adaptive focusing control method and system based on ToF linkage

Abstract

The application provides a kind of projector multi-dimensional adaptive focusing control method and system based on ToF linkage, depth information and image information of target projection surface are collected in parallel in response to focusing trigger signal, to fit space plane and calculate projection distance and side projection angle;When satisfying coarse adjustment condition, drive main focusing device and Schmidt adjustment device to execute feedforward coarse adjustment, then search center area sharpness peak based on multi-region sharpness index after judging coarse adjustment completion, and then trigger fine adjustment according to global sharpness standard deviation;According to the direction of the difference of each area sharpness, step search until the gradient sign reverses or the absolute value meets the standard, finally collect the global sharpness value of continuous multiple frames of pictures to judge whether it meets the standard after locking the focusing parameters;The application effectively improves the focusing efficiency and steady-state maintaining ability in complex scenes while ensuring full-screen optical sharpness.

Description

Technical Field

[0001] This invention relates to the field of automatic focusing technology for projection display devices, and more specifically, to a multi-dimensional adaptive focusing control method and system for projectors based on ToF linkage. Background Technology

[0002] With the increasing popularity of smart projectors, users have higher and higher requirements for the image quality of projectors in complex installation environments. Among them, full-screen clear focus in large-angle side projection scenarios has always been a technical challenge that restricts user experience. Existing projector autofocus solutions usually rely on a single camera to capture the focus pattern and calculate the sharpness of the center area. By controlling the main lens motor to move along the optical axis, it searches for the peak sharpness. However, moving the main lens along the optical axis can only change the front and back position of the entire focal plane, and cannot rotate the focal plane to match the tilted screen plane in large side projection. This results in the left side of the image being clear while the right side is blurry, and vice versa, with the edges dropping simultaneously when the center is clear. Secondly, existing solutions only use the center area of ​​the image as the sole object of sharpness evaluation. Center sharpness does not mean full-screen sharpness. When the sharpness of multiple areas (left, center, and right) is simply added together and then controlled by the same motor, the optimal number of steps required for different areas conflict. The optimization result is only a compromise and cannot achieve optical sharpness in all areas simultaneously. Secondly, some products use digital downsizing, cropping, or local sharpening to mask edge blur, which not only loses effective pixels and image brightness but also fails to restore true optical details, and cannot be called lossless full-screen focusing. Finally, when the side projection angle, projection distance, and screen posture change, the fixed compensation value is no longer applicable, and the camera is easily affected by the projected content, wall texture, and ambient light, leading to misjudgments. The ToF depth map may also contain holes and outliers. Without an effective 3D ranging and confidence assessment mechanism, the system cannot predict the focal plane tilt before performing the search, causing the two actuators to blindly search together over a large area, resulting in long focusing times and a tendency to get stuck in local optima. At the same time, if the two motors lack hysteresis compensation and linkage sequence control, problems such as inconsistent step counts, mechanical shocks, or alternating oscillations in left and right sharpness may occur during the focusing process. Summary of the Invention

[0003] In view of the above problems, the purpose of this invention is to provide a multi-dimensional adaptive focus control method and system for projectors based on ToF linkage. By responding to the focus trigger signal, the method acquires depth information and image information in parallel to fit the spatial plane, and performs a complete process of feedforward coarse adjustment, center peak search and regional difference gradient judgment, thereby realizing the hierarchical linkage control of ToF feedforward and camera feedback.

[0004] The first aspect of this invention provides a multi-dimensional adaptive focus control method for projectors based on ToF linkage, the method comprising: In response to the focus trigger signal, depth information and image information of the target projection surface are acquired; Based on the depth information, fit the spatial plane of the target projection surface, and calculate the projection distance and side projection angle; When the coarse adjustment conditions are met, the main focusing device and the SAM adjustment device are driven to perform feedforward coarse adjustment according to the projection distance and the side projection angle, respectively. Collect multi-area sharpness indicators of the current image; When the coarse adjustment is completed according to the multi-area sharpness index, the SAM adjustment device is fixed and the main focusing device is driven to search for the sharpness peak of the center area. Obtain the sharpness evaluation value for each region and calculate the global sharpness standard deviation; When fine-tuning is triggered based on the global sharpness standard deviation, a step search is performed according to the direction of the difference in sharpness of each region until the gradient sign of the difference in regional sharpness is reversed or the absolute value of the gradient is less than the preset gradient threshold. When the global sharpness value of multiple consecutive frames is determined to meet the sharpness standard, the current focus parameters are locked and the focusing process ends.

[0005] In this scheme, the coarse adjustment condition being met specifically includes: Based on the depth information and the image information, determine the confidence factor of the current scene; Determine whether the confidence factor is greater than a preset scenario confidence threshold; If so, then the coarse adjustment condition is satisfied. If not, it is determined that the coarse adjustment conditions are not met, and the system switches to the preset downgraded focus mode. The downgraded focus mode includes driving the main focus device to traverse and search at a preset step size, using the peak sharpness of the central area of ​​the image as the focus target position.

[0006] In this solution, the step of driving the main focusing device and the SAM adjustment device to perform feedforward coarse adjustment based on the projection distance and the side projection angle specifically includes: The system queries a preset main motor step mapping table based on the projection distance to obtain the initial value of the main motor, and then drives the main focusing device to move to the initial value of the main motor. The initial value of the SAM motor is obtained by querying the preset SAM step number mapping table according to the side projection angle, and the SAM adjustment device is driven to move to the initial value of the SAM motor. Obtain multi-region sharpness indices for the current image and calculate the regional sharpness difference value; If the difference in sharpness between the regions is greater than the preset alignment tolerance threshold, then the SAM adjustment device performs unidirectional step compensation according to the direction of the difference in sharpness between the regions until the difference in sharpness between the regions is less than the alignment tolerance threshold or the preset maximum number of compensations is reached.

[0007] In this solution, driving the main focusing device to search for the peak sharpness in the central region specifically includes: The main focusing device is held in the position after the feedforward coarse adjustment, and a focusing pattern containing multi-directional high-frequency texture is projected. Capture the current image and calculate the sharpness evaluation value of the central area based on the central area of ​​the image; With the current position of the main motor as the center, drive the main focusing device to perform step search within the first range, and record the central area sharpness evaluation value corresponding to each step; Once the search exceeds the sharpness peak, the search step size is reduced, and a second search is performed in the step neighborhood corresponding to the peak. Based on the number of steps obtained from the secondary search and the corresponding sharpness evaluation value, the peak position is determined as the target number of steps for the main focusing device and driven to the target number of steps.

[0008] In this solution, the step-by-step search based on the differences in sharpness between different regions specifically includes: The image is divided into a left region and a right region. The sharpness evaluation values ​​for the left and right regions are calculated separately to obtain the difference in evaluation values. If the sharpness evaluation value on the left side is greater than the sharpness evaluation value on the right side, then the Sham adjustment device is driven to step along the first direction; If the sharpness evaluation value on the left side is less than the sharpness evaluation value on the right side, then the Sham adjustment device is driven to step along the second direction; After each step, the evaluation value difference is recalculated until the sign of the evaluation value difference is reversed, or the absolute value of the evaluation value difference is less than the preset balance threshold, at which point the fine adjustment of the SAM adjustment device is stopped.

[0009] In this solution, the determination of whether the global sharpness value of multiple consecutive frames meets the sharpness standard specifically includes: After fine-tuning, the target actuator with the largest amount of motion is determined among the main focusing device and the SAM adjustment device; A stress relief pulse sequence is executed on the target actuator, the stress relief pulse sequence comprising: driving the first step along a first direction, then driving the first step along a second direction, and finally returning to the original target position; Within a preset time window after the stress relief pulse sequence is completed, multiple frames are continuously captured at a fixed frame rate. Calculate the global sharpness value for each frame and the standard deviation of the global sharpness value; Determine whether the standard deviation is less than a preset drift tolerance threshold; If so, the sharpness is deemed satisfactory, and the current focus parameters are locked. If not, the fine-tuning process of the SAM adjustment device will be retried.

[0010] A second aspect of the present invention provides a ToF-linked multi-dimensional adaptive focus control system for a projector, including a ToF-linked multi-dimensional adaptive focus control method program for a projector. When the ToF-linked multi-dimensional adaptive focus control method program is executed by the processor, it performs the following steps: In response to the focus trigger signal, depth information and image information of the target projection surface are acquired; Based on the depth information, fit the spatial plane of the target projection surface, and calculate the projection distance and side projection angle; When the coarse adjustment conditions are met, the main focusing device and the SAM adjustment device are driven to perform feedforward coarse adjustment according to the projection distance and the side projection angle, respectively. Collect multi-area sharpness indicators of the current image; When the coarse adjustment is completed according to the multi-area sharpness index, the SAM adjustment device is fixed and the main focusing device is driven to search for the sharpness peak of the center area. Obtain the sharpness evaluation value for each region and calculate the global sharpness standard deviation; When fine-tuning is triggered based on the global sharpness standard deviation, a step search is performed according to the direction of the difference in sharpness of each region until the gradient sign of the difference in regional sharpness is reversed or the absolute value of the gradient is less than the preset gradient threshold. When the global sharpness value of multiple consecutive frames is determined to meet the sharpness standard, the current focus parameters are locked and the focusing process ends.

[0011] A third aspect of the present invention provides a computer-readable storage medium comprising a ToF-linked projector multidimensional adaptive focus control method program, wherein when the ToF-linked projector multidimensional adaptive focus control method program is executed by a processor, the steps of the ToF-linked projector multidimensional adaptive focus control method as described in any of the preceding claims are implemented.

[0012] This invention provides a multi-dimensional adaptive focusing control method and system for projectors based on ToF linkage. Responding to a focus trigger signal, it acquires depth and image information of the target projection surface in parallel to fit the spatial plane and calculate the projection distance and side projection angle. When coarse adjustment conditions are met, it drives the main focusing device and the SAM adjustment device to perform feedforward coarse adjustment. After coarse adjustment is completed based on multi-region sharpness indicators, it fixes the SAM adjustment device and searches for the sharpness peak in the central region. Then, it triggers fine adjustment based on the global sharpness standard deviation. It searches stepwise according to the sharpness difference direction of each region until the gradient sign reverses or the absolute value reaches the standard. Finally, it acquires the global sharpness value of multiple consecutive frames and locks the focus parameters after determining that the standard has been met. This invention effectively improves focusing efficiency and steady-state maintenance capability in complex scenes while ensuring full-screen optical sharpness. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope.

[0014] Figure 1 A flowchart of a multi-dimensional adaptive focus control method for a projector based on ToF linkage according to the present invention is shown; Figure 2 A flowchart illustrating the determination of coarse adjustment conditions provided in an embodiment of the present invention is shown; Figure 3 The following is a flowchart illustrating the execution of a feedforward coarse adjustment according to an embodiment of the present invention; Figure 4 A block diagram of a multi-dimensional adaptive focusing control system for a projector based on ToF linkage is shown in this invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Unless otherwise defined, all terms (including technical and scientific terms) used in embodiments of this invention shall have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in a common dictionary shall be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as being interpreted in an idealized or highly formalized sense, unless expressly defined in this embodiment of the invention.

[0017] The terms "first," "second," and similar words used in the embodiments of this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "an," "a," or "the" do not indicate a quantity limitation, but rather indicate the presence of at least one. Similarly, terms such as "including" or "comprising" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The steps preceding or following the steps in the method of the embodiments of this invention are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0018] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0019] Figure 1 The flowchart of a multi-dimensional adaptive focus control method for projectors based on ToF linkage according to the present invention is shown.

[0020] like Figure 1 As shown, the first aspect of this invention discloses a multi-dimensional adaptive focus control method for projectors based on ToF linkage, the method comprising: S102, in response to the focus trigger signal, acquires depth information and image information of the target projection surface; S104, Based on the depth information, fit the spatial plane of the target projection surface, and calculate the projection distance and side projection angle; S106, when the coarse adjustment conditions are met, the main focusing device and the SAM adjustment device are driven to perform feedforward coarse adjustment according to the projection distance and the side projection angle, respectively. S108, collects multi-area sharpness indicators of the current image; S110, when the coarse adjustment is completed according to the multi-area sharpness index, fix the SAM adjustment device and drive the main focusing device to search for the sharpness peak of the center area. S112, obtain the sharpness evaluation value of each region and calculate the global sharpness standard deviation; S114, when fine-tuning is triggered based on the global sharpness standard deviation, a step search is performed according to the direction of the difference in sharpness of each region until the gradient sign of the difference in regional sharpness is reversed or the absolute value of the gradient is less than the preset gradient threshold. S116: When the global sharpness value of multiple consecutive frames is determined to meet the sharpness standard, the current focus parameters are locked and the focusing process ends.

[0021] It should be noted that in this embodiment, the projector responds to a focus trigger signal triggered by the user or a change in posture, simultaneously activating the ToF depth sensor and the camera to acquire depth information and image information of the target projection surface, respectively. The processor acquires the depth map, echo amplitude, and valid markers output by the ToF sensor, backprojects the valid depth pixels to the ToF coordinate system using the sensor's intrinsic parameters, and then converts them to the projector coordinate system using the ToF to projector extrinsic parameters. After removing obstructions and outliers outside the projection area, it fits the spatial plane equation of the target projection surface and calculates the projection center distance and the horizontal and vertical side projection angles. Subsequently, the system enters the coarse adjustment condition determination stage, comprehensively determining the confidence factor of the current scene based on the depth and image information, comparing it with the preset scene confidence threshold, and determining whether the coarse adjustment conditions are met. If met, the system enters the feedforward coarse adjustment stage, querying the main motor step count mapping table based on the projection distance to drive the main focus device to move to the initial value, and simultaneously querying the SAM step count mapping table based on the side projection angle to drive the SAM adjustment device to move to the initial value. After coarse adjustment, the camera captures the current projected image, extracts multi-region image blocks according to a preset segmentation strategy, calculates the sharpness index for each region, and determines whether coarse adjustment is complete based on the multi-region sharpness index. After coarse adjustment, the SAM adjustment device is fixed, and the main focusing device is driven to step around the initial value to search for the sharpness peak in the center region, determining the optimal number of steps for the main focusing device. Sharpness evaluation values ​​for each region are acquired, and the global sharpness standard deviation is calculated. The global sharpness standard deviation is used to determine whether fine adjustment should be triggered. If fine adjustment is triggered, the SAM adjustment device is driven to step around the direction of sharpness differences between regions. After each step, the sharpness evaluation value for each region is recalculated until the gradient sign of the regional sharpness difference is reversed or the absolute value of the gradient is less than a preset gradient threshold, at which point fine adjustment stops. After fine adjustment, the global sharpness value of multiple consecutive frames is acquired, its standard deviation is calculated, and compared with a preset drift tolerance threshold. Once the sharpness is deemed satisfactory, the step parameters of the current main focusing device and SAM adjustment device are locked, and the focusing process ends. This embodiment achieves complete closed-loop control of the projector's focusing process through the layered linkage of ToF feedforward and camera feedback, combined with a sharpness compliance determination mechanism.

[0022] Figure 2 A flowchart illustrating the determination of coarse adjustment conditions provided by an embodiment of the present invention is shown.

[0023] According to embodiments of the present invention, such as Figure 2 As shown, satisfying the coarse adjustment conditions specifically includes: S202, determine the confidence factor of the current scene based on the depth information and the image information; S204, determine whether the confidence factor is greater than the preset scene confidence threshold; S206, if so, then it is determined that the coarse adjustment condition is met; S208, if not, it is determined that the coarse adjustment conditions are not met, and the preset downgraded focus mode is switched; S210, the downgraded focus mode includes driving the main focus device to traverse and search at a preset step size, using the peak sharpness of the central area of ​​the image as the focus target position.

[0024] It should be noted that in this embodiment, after fitting the spatial plane of the target projection surface, the system simultaneously performs quality assessment on the ToF depth information and camera image information. It comprehensively considers three dimensions—the number of effective ToF points, the plane fitting residual, and the richness of image texture—and determines the confidence factor of the current scene through weighted calculation. The confidence factor is then compared with a preset scene confidence threshold. If the confidence factor is greater than or equal to the scene confidence threshold, the coarse adjustment conditions are met, and the system continues with the subsequent feedforward coarse adjustment process. If the confidence factor is less than the scene confidence threshold, the current scene is determined not to meet the coarse adjustment conditions. For example, if there is severe occlusion, a non-planar screen, or excessively strong ambient light causing unreliable depth information, the system automatically switches to a preset degraded focus mode. In degraded focus mode, the system pauses ToF feedforward control, fixes the Sham adjustment device at zero, and drives the main focusing device to gradually traverse and search from the near-focus end to the far-focus end in preset fixed steps. It collects the sharpness evaluation value of the central area of ​​each frame, selects the main motor step number corresponding to the sharpness peak as the focus target position, and drives the main focusing device to move to that position. This embodiment sets a scene confidence threshold as a coarse adjustment start condition to ensure that the focusing process only calls ToF feedforward in valid scenes, avoiding erroneous actions caused by unreliable depth information; at the same time, it provides a degraded mode to ensure basic focusing capabilities in complex environments.

[0025] Figure 3 The following is a flowchart illustrating the execution of a feedforward coarse adjustment according to an embodiment of the present invention.

[0026] According to embodiments of the present invention, such as Figure 3 As shown, the step of driving the main focusing device and the SAM adjustment device to perform feedforward coarse adjustment according to the projection distance and the side projection angle specifically includes: S302, according to the projection distance, query the preset main motor step number mapping table to obtain the initial value of the main motor, and drive the main focusing device to move to the initial value of the main motor; S304, according to the side projection angle, query the preset SAM step number mapping table to obtain the initial value of the SAM motor, and drive the SAM adjustment device to move to the initial value of the SAM motor; S306: Obtain multi-region sharpness indicators of the current image and calculate the regional sharpness difference value; S308, if the difference in regional sharpness is greater than a preset alignment tolerance threshold, then according to the direction of the difference in sharpness of each region, the SAM adjustment device performs unidirectional step compensation until the difference in regional sharpness is less than the alignment tolerance threshold or the preset maximum number of compensations is reached.

[0027] It should be noted that in this embodiment, the system queries a preset main motor step number mapping table based on the fitted projection center distance to obtain the initial value of the main motor, and then drives the main focusing device to move to that initial value. This mapping table is a record of the optimal sharpness position of the main focusing device calibrated at multiple known projection distances before leaving the factory. Simultaneously, the system queries a preset SAM step number mapping table based on the horizontal and vertical side projection angles to obtain the initial value of the SAM motor and drives the SAM adjustment device to move to that initial value, so that the focal plane initially approaches the screen plane. As one implementation method, it is assumed that the target tilt angle of the screen plane relative to the lens plane is... The target tilt angle required for the inner screen or equivalent image plane is... Since the actual projection lens assembly may contain multiple lens groups and relay reflection structures, this embodiment preferably uses optical design values ​​or factory calibration functions to establish the following: ; in, Indicates zoom, focus, or optical / mechanical status. Convert the target angle into the absolute number of steps for the second motor. The offset is zero. After the two devices complete their actions, the camera captures the current projected image, divides the image into multiple evaluation areas, calculates the sharpness value of each area, and then calculates the sharpness difference value of each area. If the difference value is greater than the preset alignment tolerance threshold, it indicates that there is still a significant unevenness in the sharpness of each area of ​​the image after coarse adjustment. The system performs unidirectional step compensation on the SAM adjustment device according to the direction of the sharpness difference of each area. For example, if the sharpness of the left side is higher than that of the right side, it indicates that the tilt direction of the focal plane is not completely matched with the screen. After each compensation, the sharpness difference value of the area is recalculated and it is determined whether to continue compensation, until the difference value is less than the alignment tolerance threshold or the number of compensations reaches the preset maximum number of times, at which point compensation stops. This embodiment quickly approaches the target position through feedforward coarse adjustment and then eliminates model errors with unidirectional compensation, effectively reducing the search burden in the subsequent fine adjustment stage.

[0028] According to an embodiment of the present invention, driving the main focusing device to search for the sharpness peak in the central region specifically includes: The main focusing device is held in the position after the feedforward coarse adjustment, and a focusing pattern containing multi-directional high-frequency texture is projected. Capture the current image and calculate the sharpness evaluation value of the central area based on the central area of ​​the image; With the current position of the main motor as the center, drive the main focusing device to perform step search within the first range, and record the central area sharpness evaluation value corresponding to each step; Once the search exceeds the sharpness peak, the search step size is reduced, and a second search is performed in the step neighborhood corresponding to the peak. Based on the number of steps obtained from the secondary search and the corresponding sharpness evaluation value, the peak position is determined as the target number of steps for the main focusing device and driven to the target number of steps.

[0029] It should be noted that in this embodiment, after the Sham adjustment device completes the feedforward coarse adjustment and locks, the system triggers the projector's optical engine to project a focus pattern containing bevels, thin lines, checkerboard patterns, or random high-frequency textures, ensuring that the image contains rich multi-directional high-frequency texture information for sharpness evaluation. After the camera captures the focus pattern, distortion correction is performed using the camera's intrinsic parameters. Then, using the calibration relationship between the camera and the projected image, an image patch of a preset size is extracted from the central region of the image, and the sharpness evaluation value of the image patch is calculated. As one implementation method, regional sharpness can be achieved using Laplacian variance, Tenengrad gradient, modulation transfer contrast, or multi-index fusion. The Tenengrad index is: ; in , For Sobel gradient, The effective pixel count for the region. To reduce the impact of brightness and exposure differences, the region is first normalized to the mean, and overexposed, underexposed, and insufficiently textured areas are excluded. Centered on the current main motor position, the main focusing device is driven within a first preset range to search in large step increments. Each time it moves to a position corresponding to a certain number of steps, the same focus pattern is projected, and the sharpness evaluation value of the central region corresponding to that step is recorded. When the sharpness is detected to first increase and then decrease with the step direction, exceeding a peak value, the system reduces the search step size and performs a secondary fine search within the neighborhood of the peak value's step count, further refining the peak position. Based on the multiple steps and corresponding sharpness evaluation values ​​obtained from the secondary search, a quadratic curve fitting method is used to determine the optimal sub-step position corresponding to the extreme point. This position is quantified into an integer number of steps that the motor can execute, and the main focusing device is driven from a specified direction to approach the target number of steps to eliminate gear backlash, ultimately completing the focusing and positioning of the main focusing device. This embodiment employs a coarse-fine search strategy, balancing focusing speed and positioning accuracy.

[0030] According to an embodiment of the present invention, the step-by-step search based on the difference in sharpness of each region specifically includes: The image is divided into a left region and a right region. The sharpness evaluation values ​​for the left and right regions are calculated separately to obtain the difference in evaluation values. If the sharpness evaluation value on the left side is greater than the sharpness evaluation value on the right side, then the Sham adjustment device is driven to step along the first direction; If the sharpness evaluation value on the left side is less than the sharpness evaluation value on the right side, then the Sham adjustment device is driven to step along the second direction; After each step, the evaluation value difference is recalculated until the sign of the evaluation value difference is reversed, or the absolute value of the evaluation value difference is less than the preset balance threshold, at which point the fine adjustment of the SAM adjustment device is stopped.

[0031] It should be noted that, in this embodiment, after the camera captures the projected focus pattern, the image is divided into a left region and a right region using the calibration relationship between the camera and the projected image. The sharpness evaluation values ​​for the left and right sides are calculated separately, and the difference between the two is recorded as the region sharpness evaluation value difference. This difference reflects the tilt direction deviation of the current focal plane relative to the screen plane, specifically corresponding to the Sham compensation error. The left-right difference... This primarily reflects the horizontal Sham compensation error. If the sharpness evaluation value on the left side is greater than that on the right side, it indicates that the sharpness of the left area of ​​the screen is better than that on the right, and the focal plane deflection direction is not yet completely aligned with the screen. In this case, the system drives the Sham adjustment device to step along the first direction, tilting the focal plane to the right to compensate for the current deviation. If the sharpness evaluation value on the left side is less than that on the right side, the system drives the Sham adjustment device to step along the opposite second direction. After each step, the image is re-acquired and the evaluation value difference is calculated. This process continues iterating until the sign of the evaluation value difference is reversed, meaning the sharpness advantage has switched from one side to the other, indicating that the optimal compensation point has been exceeded. Alternatively, the system stops fine-tuning when the absolute value of the evaluation value difference is less than a preset balance threshold. This embodiment achieves high-precision phase positioning of the Sham adjustment device through the difference direction driving and sign reversal stopping mechanism, avoiding adjustment oscillations caused by mechanical hysteresis or step overshoot.

[0032] According to an embodiment of the present invention, the determination of whether the global sharpness value of the acquired multi-frame images meets the sharpness standard specifically includes: After fine-tuning, the target actuator with the largest amount of motion is determined among the main focusing device and the SAM adjustment device; A stress relief pulse sequence is executed on the target actuator, the stress relief pulse sequence comprising: driving the first step along a first direction, then driving the first step along a second direction, and finally returning to the original target position; Within a preset time window after the stress relief pulse sequence is completed, multiple frames are continuously captured at a fixed frame rate. Calculate the global sharpness value for each frame and the standard deviation of the global sharpness value; Determine whether the standard deviation is less than a preset drift tolerance threshold; If so, the sharpness is deemed satisfactory, and the current focus parameters are locked. If not, the fine-tuning process of the SAM adjustment device will be retried.

[0033] It should be noted that, in this embodiment, after the SAM fine-tuning is completed, the system reads the total number of motion steps of the main focusing device and the SAM adjustment device during this focusing process, compares the two, and determines the actuator with the largest amount of motion as the target actuator. A stress release pulse sequence is executed on the target actuator. This sequence specifically includes: driving the device one step in a first direction, then driving the device the same number of steps in a second direction, and finally returning to the original target position. This forward and reverse reciprocating motion releases the gear meshing stress and mechanical deformation caused by prolonged unidirectional force. Within a preset time window after the stress release pulse sequence is completed, multiple frames are continuously acquired at a fixed frame rate. The global sharpness value of each frame is calculated, and then the standard deviation of these global sharpness values ​​is calculated to quantify the degree of focus drift. The standard deviation is compared with a preset drift tolerance threshold. If it is less than the drift tolerance threshold, the sharpness is deemed acceptable, and the step count parameters of the current main focusing device and the SAM adjustment device are locked. If it is greater than or equal to the drift tolerance threshold, the system is determined to have continuous drift, and the fine-tuning process of the SAM adjustment device is retried. This embodiment eliminates residual mechanical stress through stress release and determines steady state by combining the standard deviation of sharpness in consecutive frames, thus ensuring the long-term stability and anti-interference capability of the focusing parameters.

[0034] It is worth mentioning that, before driving the Sham adjustment device to perform feedforward coarse adjustment, the following steps are also included: Obtain the current physical position step of the Sham adjustment device; The target number of steps for the Sham motor is obtained by querying the preset Sham step number mapping table based on the side projection angle. Determine whether the target number of steps for the SAM motor is within a preset safe step range; If it is located, then drive the SAM adjustment device to move to the target number of steps of the SAM motor; If not located, the SAM adjustment device is driven to the boundary value of the safe step range, generating a side projection over-limit prompt message, and switching to the digital keystone correction compensation mode, which only adjusts the geometry of the projected image.

[0035] It should be noted that in this embodiment, after the system obtains the target number of steps for the SAM motor by querying the SAM step number mapping table based on the side projection angle, it does not directly drive the motor. Instead, it first obtains the current physical position step number of the SAM adjustment device and reads the preset safe step number range. The upper and lower limits of this range are determined by the physical travel limits of the projector's optical engine structure. Exceeding this range may cause the reflector or inner screen to collide with the mechanical boundary. The system determines whether the target number of steps for the SAM motor is within the safe step number range. If it is, the system normally drives the SAM adjustment device to move to the target number of steps. If the target number of steps exceeds the upper or lower limit of the range, the system drives the SAM adjustment device to the boundary value of the safe step number range, generates a side projection over-limit warning message to inform the user that the current side projection angle exceeds the optical compensation capability range, and automatically switches to the digital keystone correction compensation mode. In this mode, only the geometry of the projected image is adjusted by digital algorithms, and the optical focal plane tilt compensation is no longer relied on the SAM adjustment device. This embodiment effectively prevents collision damage to the actuator through safety judgment and over-limit protection mechanisms, and ensures user usability in extreme side projection scenarios through mode switching.

[0036] It is worth mentioning that the SAM adjustment device includes a horizontal axis adjustment mechanism and a vertical axis adjustment mechanism, and the step search based on the difference in sharpness of each region further includes: Divide the screen into left, right, top, and bottom areas; Calculate the sharpness difference between the left and right regions as the horizontal difference value; Calculate the sharpness difference between the upper and lower regions as the vertical difference value; The horizontal axis adjustment mechanism is driven to step according to the direction of the horizontal difference value; The vertical axis adjustment mechanism is driven to step according to the direction of the vertical difference value; After each step movement, the horizontal difference value and the vertical difference value are updated synchronously, and the horizontal axis adjustment mechanism and the vertical axis adjustment mechanism are driven alternately until the absolute value of the horizontal difference value and the absolute value of the vertical difference value are both less than the preset balance threshold.

[0037] It should be noted that, in this embodiment, the Sham adjustment device includes a horizontal axis adjustment mechanism and a vertical axis adjustment mechanism, which are used to compensate for the focal plane tilt caused by horizontal and vertical side projection, respectively. The system divides the image into left, right, upper, and lower regions, and calculates the sharpness evaluation value for each region. The sharpness difference between the left and right regions is calculated as the horizontal difference value, which corresponds to the Sham horizontal compensation error. The difference in sharpness between the upper and lower regions is calculated as the vertical difference value, which corresponds to the Sham vertical compensation error. The horizontal axis adjustment mechanism is driven to step along the corresponding direction based on the sign of the horizontal difference value; simultaneously, the vertical axis adjustment mechanism is driven to step along the corresponding direction based on the sign of the vertical difference value. After each step, the horizontal and vertical difference values ​​are updated synchronously. An alternating drive strategy is adopted, i.e., the horizontal axis is driven one step, then the vertical axis one step, and this cycle continues until the absolute values ​​of both the horizontal and vertical difference values ​​are less than a preset equalization threshold, at which point fine-tuning stops. This embodiment achieves simultaneous compensation for focal plane tilt in both horizontal and vertical directions through independent dual-axis detection and alternating drive, making it suitable for complex installation scenarios where the projector exhibits both horizontal side projection and vertical tilt.

[0038] It's worth mentioning that after locking the current focus parameters and ending the focusing process, the following also applies: Record the projection distance, side projection angle, final number of steps of the main focusing device, final number of steps of the SAM adjustment device, and global sharpness standard deviation of the current scene as historical focusing parameters; When the focus trigger signal is received again, the current projection distance and the current side projection angle are collected. Match the current projection distance and current side projection angle with the historical focus parameters; If the match is successful, the final step number of the main focusing device and the final step number of the SAM adjustment device in the historical focusing parameters are directly used as the initial drive position. A fine-tuning search is performed within a preset neighborhood of the initial driving position, skipping the feedforward coarse-tuning process.

[0039] It should be noted that, in this embodiment, after each successful completion of the focusing process and parameter locking, the system packages the current scene's projection distance, horizontal side projection angle, vertical side projection angle, final step count of the main focusing device, final step count of the SAM adjustment device, and final global sharpness standard deviation into a historical focusing parameter record and stores it in local non-volatile memory. When the system receives a focusing trigger signal again, it first acquires the current projection distance and current side projection angle through the ToF sensor, and then performs a matching search in the historical focusing parameter record library using the current projection distance and side projection angle as query keys. When the deviation between the current parameters and the parameters of a certain historical record is less than a preset matching tolerance threshold, it is determined that the match is successful. If the match is successful, the system directly calls the final step count of the main focusing device and the final step count of the SAM adjustment device in the historical record as the initial driving position for this focusing. After driving the two devices to this position, it only performs fine-tuning search within a preset neighborhood of the initial driving position to eliminate deviations caused by slight environmental changes, while directly skipping the feedforward coarse-tuning process. This embodiment reuses historical focusing results by matching scene parameters, which significantly shortens focusing time and reduces system power consumption in repetitive scenes.

[0040] Figure 4 A block diagram of a multi-dimensional adaptive focusing control system for a projector based on ToF linkage is shown in this invention.

[0041] like Figure 4 As shown, the second aspect of the present invention discloses a ToF-linked projector multi-dimensional adaptive focus control system 4, including a memory 41 and a processor 42. The memory includes a ToF-linked projector multi-dimensional adaptive focus control method program. When the ToF-linked projector multi-dimensional adaptive focus control method program is executed by the processor, it performs the following steps: In response to the focus trigger signal, depth information and image information of the target projection surface are acquired; Based on the depth information, fit the spatial plane of the target projection surface, and calculate the projection distance and side projection angle; When the coarse adjustment conditions are met, the main focusing device and the SAM adjustment device are driven to perform feedforward coarse adjustment according to the projection distance and the side projection angle, respectively. Collect multi-area sharpness indicators of the current image; When the coarse adjustment is completed according to the multi-area sharpness index, the SAM adjustment device is fixed and the main focusing device is driven to search for the sharpness peak of the center area. Obtain the sharpness evaluation value for each region and calculate the global sharpness standard deviation; When fine-tuning is triggered based on the global sharpness standard deviation, a step search is performed according to the direction of the difference in sharpness of each region until the gradient sign of the difference in regional sharpness is reversed or the absolute value of the gradient is less than the preset gradient threshold. When the global sharpness value of multiple consecutive frames is determined to meet the sharpness standard, the current focus parameters are locked and the focusing process ends.

[0042] It should be noted that in this embodiment, the projector responds to a focus trigger signal triggered by the user or a change in posture, simultaneously activating the ToF depth sensor and the camera to acquire depth information and image information of the target projection surface, respectively. The processor acquires the depth map, echo amplitude, and valid markers output by the ToF sensor, backprojects the valid depth pixels to the ToF coordinate system using the sensor's intrinsic parameters, and then converts them to the projector coordinate system using the ToF to projector extrinsic parameters. After removing obstructions and outliers outside the projection area, it fits the spatial plane equation of the target projection surface and calculates the projection center distance and the horizontal and vertical side projection angles. Subsequently, the system enters the coarse adjustment condition determination stage, comprehensively determining the confidence factor of the current scene based on the depth and image information, comparing it with the preset scene confidence threshold, and determining whether the coarse adjustment conditions are met. If met, the system enters the feedforward coarse adjustment stage, querying the main motor step count mapping table based on the projection distance to drive the main focus device to move to the initial value, and simultaneously querying the SAM step count mapping table based on the side projection angle to drive the SAM adjustment device to move to the initial value. After coarse adjustment, the camera captures the current projected image, extracts multi-region image blocks according to a preset segmentation strategy, calculates the sharpness index for each region, and determines whether coarse adjustment is complete based on the multi-region sharpness index. After coarse adjustment, the SAM adjustment device is fixed, and the main focusing device is driven to step around the initial value to search for the sharpness peak in the center region, determining the optimal number of steps for the main focusing device. Sharpness evaluation values ​​for each region are acquired, and the global sharpness standard deviation is calculated. The global sharpness standard deviation is used to determine whether fine adjustment should be triggered. If fine adjustment is triggered, the SAM adjustment device is driven to step around the direction of sharpness differences between regions. After each step, the sharpness evaluation value for each region is recalculated until the gradient sign of the regional sharpness difference is reversed or the absolute value of the gradient is less than a preset gradient threshold, at which point fine adjustment stops. After fine adjustment, the global sharpness value of multiple consecutive frames is acquired, its standard deviation is calculated, and compared with a preset drift tolerance threshold. Once the sharpness is deemed satisfactory, the step parameters of the current main focusing device and SAM adjustment device are locked, and the focusing process ends. This embodiment achieves complete closed-loop control of the projector's focusing process through the layered linkage of ToF feedforward and camera feedback, combined with a sharpness compliance determination mechanism.

[0043] According to an embodiment of the present invention, satisfying the coarse adjustment condition specifically includes: Based on the depth information and the image information, determine the confidence factor of the current scene; Determine whether the confidence factor is greater than a preset scenario confidence threshold; If so, then the coarse adjustment condition is satisfied. If not, it is determined that the coarse adjustment conditions are not met, and the system switches to the preset downgraded focus mode. The downgraded focus mode includes driving the main focus device to traverse and search at a preset step size, using the peak sharpness of the central area of ​​the image as the focus target position.

[0044] It should be noted that in this embodiment, after fitting the spatial plane of the target projection surface, the system simultaneously performs quality assessment on the ToF depth information and camera image information. It comprehensively considers three dimensions—the number of effective ToF points, the plane fitting residual, and the richness of image texture—and determines the confidence factor of the current scene through weighted calculation. The confidence factor is then compared with a preset scene confidence threshold. If the confidence factor is greater than or equal to the scene confidence threshold, the coarse adjustment conditions are met, and the system continues with the subsequent feedforward coarse adjustment process. If the confidence factor is less than the scene confidence threshold, the current scene is determined not to meet the coarse adjustment conditions. For example, if there is severe occlusion, a non-planar screen, or excessively strong ambient light causing unreliable depth information, the system automatically switches to a preset degraded focus mode. In degraded focus mode, the system pauses ToF feedforward control, fixes the Sham adjustment device at zero, and drives the main focusing device to gradually traverse and search from the near-focus end to the far-focus end in preset fixed steps. It collects the sharpness evaluation value of the central area of ​​each frame, selects the main motor step number corresponding to the sharpness peak as the focus target position, and drives the main focusing device to move to that position. This embodiment sets a scene confidence threshold as a coarse adjustment start condition to ensure that the focusing process only calls ToF feedforward in valid scenes, avoiding erroneous actions caused by unreliable depth information; at the same time, it provides a degraded mode to ensure basic focusing capabilities in complex environments.

[0045] According to an embodiment of the present invention, the step of driving the main focusing device and the SAM adjustment device to perform feedforward coarse adjustment based on the projection distance and the side projection angle specifically includes: The system queries a preset main motor step mapping table based on the projection distance to obtain the initial value of the main motor, and then drives the main focusing device to move to the initial value of the main motor. The initial value of the SAM motor is obtained by querying the preset SAM step number mapping table according to the side projection angle, and the SAM adjustment device is driven to move to the initial value of the SAM motor. Obtain multi-region sharpness indices for the current image and calculate the regional sharpness difference value; If the difference in sharpness between the regions is greater than the preset alignment tolerance threshold, then the SAM adjustment device performs unidirectional step compensation according to the direction of the difference in sharpness between the regions until the difference in sharpness between the regions is less than the alignment tolerance threshold or the preset maximum number of compensations is reached.

[0046] It should be noted that in this embodiment, the system queries a preset main motor step number mapping table based on the fitted projection center distance to obtain the initial value of the main motor, and then drives the main focusing device to move to that initial value. This mapping table is a record of the optimal sharpness position of the main focusing device calibrated at multiple known projection distances before leaving the factory. Simultaneously, the system queries a preset SAM step number mapping table based on the horizontal and vertical side projection angles to obtain the initial value of the SAM motor and drives the SAM adjustment device to move to that initial value, so that the focal plane initially approaches the screen plane. As one implementation method, it is assumed that the target tilt angle of the screen plane relative to the lens plane is... The target tilt angle required for the inner screen or equivalent image plane is... Since the actual projection lens assembly may contain multiple lens groups and relay reflection structures, this embodiment preferably uses optical design values ​​or factory calibration functions to establish the following: ; in, Indicates zoom, focus, or optical / mechanical status. Convert the target angle into the absolute number of steps for the second motor. The offset is zero. After the two devices complete their actions, the camera captures the current projected image, divides the image into multiple evaluation areas, calculates the sharpness value of each area, and then calculates the sharpness difference value of each area. If the difference value is greater than the preset alignment tolerance threshold, it indicates that there is still a significant unevenness in the sharpness of each area of ​​the image after coarse adjustment. The system performs unidirectional step compensation on the SAM adjustment device according to the direction of the sharpness difference of each area. For example, if the sharpness of the left side is higher than that of the right side, it indicates that the tilt direction of the focal plane is not completely matched with the screen. After each compensation, the sharpness difference value of the area is recalculated and it is determined whether to continue compensation, until the difference value is less than the alignment tolerance threshold or the number of compensations reaches the preset maximum number of times, at which point compensation stops. This embodiment quickly approaches the target position through feedforward coarse adjustment and then eliminates model errors with unidirectional compensation, effectively reducing the search burden in the subsequent fine adjustment stage.

[0047] According to an embodiment of the present invention, driving the main focusing device to search for the sharpness peak in the central region specifically includes: The main focusing device is held in the position after the feedforward coarse adjustment, and a focusing pattern containing multi-directional high-frequency texture is projected. Capture the current image and calculate the sharpness evaluation value of the central area based on the central area of ​​the image; With the current position of the main motor as the center, drive the main focusing device to perform step search within the first range, and record the central area sharpness evaluation value corresponding to each step; Once the search exceeds the sharpness peak, the search step size is reduced, and a second search is performed in the step neighborhood corresponding to the peak. Based on the number of steps obtained from the secondary search and the corresponding sharpness evaluation value, the peak position is determined as the target number of steps for the main focusing device and driven to the target number of steps.

[0048] It should be noted that in this embodiment, after the Sham adjustment device completes the feedforward coarse adjustment and locks, the system triggers the projector's optical engine to project a focus pattern containing bevels, thin lines, checkerboard patterns, or random high-frequency textures, ensuring that the image contains rich multi-directional high-frequency texture information for sharpness evaluation. After the camera captures the focus pattern, distortion correction is performed using the camera's intrinsic parameters. Then, using the calibration relationship between the camera and the projected image, an image patch of a preset size is extracted from the central region of the image, and the sharpness evaluation value of the image patch is calculated. As one implementation method, regional sharpness can be achieved using Laplacian variance, Tenengrad gradient, modulation transfer contrast, or multi-index fusion. The Tenengrad index is: ; in , For Sobel gradient, The effective pixel count for the region. To reduce the impact of brightness and exposure differences, the region is first normalized to the mean, and overexposed, underexposed, and insufficiently textured areas are excluded. Centered on the current main motor position, the main focusing device is driven within a first preset range to search in large step increments. Each time it moves to a position corresponding to a certain number of steps, the same focus pattern is projected, and the sharpness evaluation value of the central region corresponding to that step is recorded. When the sharpness is detected to first increase and then decrease with the step direction, exceeding a peak value, the system reduces the search step size and performs a secondary fine search within the neighborhood of the peak value's step count, further refining the peak position. Based on the multiple steps and corresponding sharpness evaluation values ​​obtained from the secondary search, a quadratic curve fitting method is used to determine the optimal sub-step position corresponding to the extreme point. This position is quantified into an integer number of steps that the motor can execute, and the main focusing device is driven from a specified direction to approach the target number of steps to eliminate gear backlash, ultimately completing the focusing and positioning of the main focusing device. This embodiment employs a coarse-fine search strategy, balancing focusing speed and positioning accuracy.

[0049] According to an embodiment of the present invention, the step-by-step search based on the difference in sharpness of each region specifically includes: The image is divided into a left region and a right region. The sharpness evaluation values ​​for the left and right regions are calculated separately to obtain the difference in evaluation values. If the sharpness evaluation value on the left side is greater than the sharpness evaluation value on the right side, then the Sham adjustment device is driven to step along the first direction; If the sharpness evaluation value on the left side is less than the sharpness evaluation value on the right side, then the Sham adjustment device is driven to step along the second direction; After each step, the evaluation value difference is recalculated until the sign of the evaluation value difference is reversed, or the absolute value of the evaluation value difference is less than the preset balance threshold, at which point the fine adjustment of the SAM adjustment device is stopped.

[0050] It should be noted that, in this embodiment, after the camera captures the projected focus pattern, the image is divided into a left region and a right region using the calibration relationship between the camera and the projected image. The sharpness evaluation values ​​for the left and right sides are calculated separately, and the difference between the two is recorded as the region sharpness evaluation value difference. This difference reflects the tilt direction deviation of the current focal plane relative to the screen plane, specifically corresponding to the Sham compensation error. The left-right difference... This primarily reflects the horizontal Sham compensation error. If the sharpness evaluation value on the left side is greater than that on the right side, it indicates that the sharpness of the left area of ​​the screen is better than that on the right, and the focal plane deflection direction is not yet completely aligned with the screen. In this case, the system drives the Sham adjustment device to step along the first direction, tilting the focal plane to the right to compensate for the current deviation. If the sharpness evaluation value on the left side is less than that on the right side, the system drives the Sham adjustment device to step along the opposite second direction. After each step, the image is re-acquired and the evaluation value difference is calculated. This process continues iterating until the sign of the evaluation value difference is reversed, meaning the sharpness advantage has switched from one side to the other, indicating that the optimal compensation point has been exceeded. Alternatively, the system stops fine-tuning when the absolute value of the evaluation value difference is less than a preset balance threshold. This embodiment achieves high-precision phase positioning of the Sham adjustment device through the difference direction driving and sign reversal stopping mechanism, avoiding adjustment oscillations caused by mechanical hysteresis or step overshoot.

[0051] According to an embodiment of the present invention, the determination of whether the global sharpness value of the acquired multi-frame images meets the sharpness standard specifically includes: After fine-tuning, the target actuator with the largest amount of motion is determined among the main focusing device and the SAM adjustment device; A stress relief pulse sequence is executed on the target actuator, the stress relief pulse sequence comprising: driving the first step along a first direction, then driving the first step along a second direction, and finally returning to the original target position; Within a preset time window after the stress relief pulse sequence is completed, multiple frames are continuously captured at a fixed frame rate. Calculate the global sharpness value for each frame and the standard deviation of the global sharpness value; Determine whether the standard deviation is less than a preset drift tolerance threshold; If so, the sharpness is deemed satisfactory, and the current focus parameters are locked. If not, the fine-tuning process of the SAM adjustment device will be retried.

[0052] It should be noted that, in this embodiment, after the SAM fine-tuning is completed, the system reads the total number of motion steps of the main focusing device and the SAM adjustment device during this focusing process, compares the two, and determines the actuator with the largest amount of motion as the target actuator. A stress release pulse sequence is executed on the target actuator. This sequence specifically includes: driving the device one step in a first direction, then driving the device the same number of steps in a second direction, and finally returning to the original target position. This forward and reverse reciprocating motion releases the gear meshing stress and mechanical deformation caused by prolonged unidirectional force. Within a preset time window after the stress release pulse sequence is completed, multiple frames are continuously acquired at a fixed frame rate. The global sharpness value of each frame is calculated, and then the standard deviation of these global sharpness values ​​is calculated to quantify the degree of focus drift. The standard deviation is compared with a preset drift tolerance threshold. If it is less than the drift tolerance threshold, the sharpness is deemed acceptable, and the step count parameters of the current main focusing device and the SAM adjustment device are locked. If it is greater than or equal to the drift tolerance threshold, the system is determined to have continuous drift, and the fine-tuning process of the SAM adjustment device is retried. This embodiment eliminates residual mechanical stress through stress release and determines steady state by combining the standard deviation of sharpness in consecutive frames, thus ensuring the long-term stability and anti-interference capability of the focusing parameters.

[0053] It is worth mentioning that, before driving the Sham adjustment device to perform feedforward coarse adjustment, the following steps are also included: Obtain the current physical position step of the Sham adjustment device; The target number of steps for the Sham motor is obtained by querying the preset Sham step number mapping table based on the side projection angle. Determine whether the target number of steps for the SAM motor is within a preset safe step range; If it is located, then drive the SAM adjustment device to move to the target number of steps of the SAM motor; If not located, the SAM adjustment device is driven to the boundary value of the safe step range, generating a side projection over-limit prompt message, and switching to the digital keystone correction compensation mode, which only adjusts the geometry of the projected image.

[0054] It should be noted that in this embodiment, after the system obtains the target number of steps for the SAM motor by querying the SAM step number mapping table based on the side projection angle, it does not directly drive the motor. Instead, it first obtains the current physical position step number of the SAM adjustment device and reads the preset safe step number range. The upper and lower limits of this range are determined by the physical travel limits of the projector's optical engine structure. Exceeding this range may cause the reflector or inner screen to collide with the mechanical boundary. The system determines whether the target number of steps for the SAM motor is within the safe step number range. If it is, the system normally drives the SAM adjustment device to move to the target number of steps. If the target number of steps exceeds the upper or lower limit of the range, the system drives the SAM adjustment device to the boundary value of the safe step number range, generates a side projection over-limit warning message to inform the user that the current side projection angle exceeds the optical compensation capability range, and automatically switches to the digital keystone correction compensation mode. In this mode, only the geometry of the projected image is adjusted by digital algorithms, and the optical focal plane tilt compensation is no longer relied on the SAM adjustment device. This embodiment effectively prevents collision damage to the actuator through safety judgment and over-limit protection mechanisms, and ensures user usability in extreme side projection scenarios through mode switching.

[0055] It is worth mentioning that the SAM adjustment device includes a horizontal axis adjustment mechanism and a vertical axis adjustment mechanism, and the step search based on the difference in sharpness of each region further includes: Divide the screen into left, right, top, and bottom areas; Calculate the sharpness difference between the left and right regions as the horizontal difference value; Calculate the sharpness difference between the upper and lower regions as the vertical difference value; The horizontal axis adjustment mechanism is driven to step according to the direction of the horizontal difference value; The vertical axis adjustment mechanism is driven to step according to the direction of the vertical difference value; After each step movement, the horizontal difference value and the vertical difference value are updated synchronously, and the horizontal axis adjustment mechanism and the vertical axis adjustment mechanism are driven alternately until the absolute value of the horizontal difference value and the absolute value of the vertical difference value are both less than the preset balance threshold.

[0056] It should be noted that, in this embodiment, the Sham adjustment device includes a horizontal axis adjustment mechanism and a vertical axis adjustment mechanism, which are used to compensate for the focal plane tilt caused by horizontal and vertical side projection, respectively. The system divides the image into left, right, upper, and lower regions, and calculates the sharpness evaluation value for each region. The sharpness difference between the left and right regions is calculated as the horizontal difference value, which corresponds to the Sham horizontal compensation error. The difference in sharpness between the upper and lower regions is calculated as the vertical difference value, which corresponds to the Sham vertical compensation error. The horizontal axis adjustment mechanism is driven to step along the corresponding direction based on the sign of the horizontal difference value; simultaneously, the vertical axis adjustment mechanism is driven to step along the corresponding direction based on the sign of the vertical difference value. After each step, the horizontal and vertical difference values ​​are updated synchronously. An alternating drive strategy is adopted, i.e., the horizontal axis is driven one step, then the vertical axis one step, and this cycle continues until the absolute values ​​of both the horizontal and vertical difference values ​​are less than a preset equalization threshold, at which point fine-tuning stops. This embodiment achieves simultaneous compensation for focal plane tilt in both horizontal and vertical directions through independent dual-axis detection and alternating drive, making it suitable for complex installation scenarios where the projector exhibits both horizontal side projection and vertical tilt.

[0057] It's worth mentioning that after locking the current focus parameters and ending the focusing process, the following also applies: Record the projection distance, side projection angle, final number of steps of the main focusing device, final number of steps of the SAM adjustment device, and global sharpness standard deviation of the current scene as historical focusing parameters; When the focus trigger signal is received again, the current projection distance and the current side projection angle are collected. Match the current projection distance and current side projection angle with the historical focus parameters; If the match is successful, the final step number of the main focusing device and the final step number of the SAM adjustment device in the historical focusing parameters are directly used as the initial drive position. A fine-tuning search is performed within a preset neighborhood of the initial driving position, skipping the feedforward coarse-tuning process.

[0058] It should be noted that, in this embodiment, after each successful completion of the focusing process and parameter locking, the system packages the current scene's projection distance, horizontal side projection angle, vertical side projection angle, final step count of the main focusing device, final step count of the SAM adjustment device, and final global sharpness standard deviation into a historical focusing parameter record and stores it in local non-volatile memory. When the system receives a focusing trigger signal again, it first acquires the current projection distance and current side projection angle through the ToF sensor, and then performs a matching search in the historical focusing parameter record library using the current projection distance and side projection angle as query keys. When the deviation between the current parameters and the parameters of a certain historical record is less than a preset matching tolerance threshold, it is determined that the match is successful. If the match is successful, the system directly calls the final step count of the main focusing device and the final step count of the SAM adjustment device in the historical record as the initial driving position for this focusing. After driving the two devices to this position, it only performs fine-tuning search within a preset neighborhood of the initial driving position to eliminate deviations caused by slight environmental changes, while directly skipping the feedforward coarse-tuning process. This embodiment reuses historical focusing results by matching scene parameters, which significantly shortens focusing time and reduces system power consumption in repetitive scenes.

[0059] A third aspect of the present invention provides a computer-readable storage medium comprising a ToF-linked projector multidimensional adaptive focus control method program, wherein when the ToF-linked projector multidimensional adaptive focus control method program is executed by a processor, the steps of the ToF-linked projector multidimensional adaptive focus control method as described in any of the preceding claims are implemented.

[0060] In summary, this invention provides a multi-dimensional adaptive focusing control method and system for projectors based on ToF linkage. It responds to a focusing trigger signal by simultaneously acquiring depth and image information of the target projection surface to fit a spatial plane and calculate the projection distance and side projection angle. When coarse adjustment conditions are met, it drives the main focusing device and the SAM adjustment device to perform feedforward coarse adjustment. After coarse adjustment is completed based on multi-region sharpness indicators, the SAM adjustment device is fixed, and the sharpness peak in the central region is searched. Fine adjustment is then triggered based on the global sharpness standard deviation. The search proceeds step-by-step according to the sharpness differences in each region until the gradient sign reverses or the absolute value reaches the target. Finally, the global sharpness value of multiple consecutive frames is collected, and after determining that the target is met, the focusing parameters are locked. This invention effectively improves focusing efficiency and steady-state maintenance capabilities in complex scenes while ensuring full-screen optical sharpness.

[0061] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-dimensional adaptive focus control method for projectors based on ToF linkage, characterized in that, The method includes: In response to the focus trigger signal, depth information and image information of the target projection surface are acquired; Based on the depth information, fit the spatial plane of the target projection surface, and calculate the projection distance and side projection angle; When the coarse adjustment conditions are met, the main focusing device and the SAM adjustment device are driven to perform feedforward coarse adjustment according to the projection distance and the side projection angle, respectively. Collect multi-area sharpness indicators of the current image; When the coarse adjustment is completed according to the multi-area sharpness index, the SAM adjustment device is fixed and the main focusing device is driven to search for the sharpness peak of the center area. Obtain the sharpness evaluation value for each region and calculate the global sharpness standard deviation; When fine-tuning is triggered based on the global sharpness standard deviation, a step search is performed according to the direction of the difference in sharpness in each region until the gradient sign of the difference in regional sharpness is reversed or the absolute value of the gradient is less than the preset gradient threshold. When the global sharpness value of multiple consecutive frames is determined to meet the sharpness standard, the current focus parameters are locked and the focusing process ends.

2. The multi-dimensional adaptive focus control method for projectors based on ToF linkage according to claim 1, characterized in that, When the coarse adjustment conditions are met, specifically including: Based on the depth information and the image information, determine the confidence factor of the current scene; Determine whether the confidence factor is greater than a preset scenario confidence threshold; If so, then the coarse adjustment condition is satisfied. If not, it is determined that the coarse adjustment conditions are not met, and the system switches to the preset downgraded focus mode. The downgraded focus mode includes driving the main focus device to traverse and search at a preset step size, using the peak sharpness of the central area of ​​the image as the focus target position.

3. The multi-dimensional adaptive focus control method for projectors based on ToF linkage according to claim 1, characterized in that, The step of driving the main focusing device and the SAM adjustment device to perform feedforward coarse adjustment according to the projection distance and the side projection angle respectively includes: The system queries a preset main motor step mapping table based on the projection distance to obtain the initial value of the main motor, and then drives the main focusing device to move to the initial value of the main motor. The initial value of the SAM motor is obtained by querying the preset SAM step number mapping table according to the side projection angle, and the SAM adjustment device is driven to move to the initial value of the SAM motor. Obtain multi-region sharpness indices for the current image and calculate the regional sharpness difference value; If the difference in sharpness between the regions is greater than the preset alignment tolerance threshold, then the SAM adjustment device performs unidirectional step compensation according to the direction of the difference in sharpness between the regions until the difference in sharpness between the regions is less than the alignment tolerance threshold or the preset maximum number of compensations is reached.

4. The multi-dimensional adaptive focus control method for projectors based on ToF linkage according to claim 1, characterized in that, The process of driving the main focusing device to search for the peak sharpness in the central region specifically includes: The main focusing device is held in the position after the feedforward coarse adjustment, and a focusing pattern containing multi-directional high-frequency texture is projected. Capture the current image and calculate the sharpness evaluation value of the central area based on the central area of ​​the image; With the current position of the main motor as the center, drive the main focusing device to perform step search within the first range, and record the central area sharpness evaluation value corresponding to each step; Once the search exceeds the sharpness peak, the search step size is reduced, and a second search is performed in the step neighborhood corresponding to the peak. Based on the number of steps obtained from the secondary search and the corresponding sharpness evaluation value, the peak position is determined as the target number of steps for the main focusing device and driven to the target number of steps.

5. The multi-dimensional adaptive focus control method for a projector based on ToF linkage according to claim 1, characterized in that, The step-by-step search based on the differences in sharpness between different regions specifically includes: The image is divided into a left region and a right region. The sharpness evaluation values ​​for the left and right regions are calculated separately to obtain the difference in evaluation values. If the sharpness evaluation value on the left side is greater than the sharpness evaluation value on the right side, then the Sham adjustment device is driven to step along the first direction; If the sharpness evaluation value on the left side is less than the sharpness evaluation value on the right side, then the Sham adjustment device is driven to step along the second direction; After each step, the evaluation value difference is recalculated until the sign of the evaluation value difference is reversed, or the absolute value of the evaluation value difference is less than the preset balance threshold, at which point the fine adjustment of the SAM adjustment device is stopped.

6. The multi-dimensional adaptive focus control method for a projector based on ToF linkage according to claim 1, characterized in that, The determination of whether the global sharpness value of the acquired multiple consecutive frames meets the sharpness standard specifically includes: After fine-tuning, the target actuator with the largest amount of motion is determined among the main focusing device and the SAM adjustment device; A stress relief pulse sequence is executed on the target actuator, the stress relief pulse sequence comprising: driving the first step along a first direction, then driving the first step along a second direction, and finally returning to the original target position; Within a preset time window after the stress relief pulse sequence is completed, multiple frames are continuously captured at a fixed frame rate. Calculate the global sharpness value for each frame and the standard deviation of the global sharpness value; Determine whether the standard deviation is less than a preset drift tolerance threshold; If so, the sharpness is deemed satisfactory, and the current focus parameters are locked. If not, the fine-tuning process of the SAM adjustment device will be retried.

7. A multi-dimensional adaptive focusing control system for a projector based on ToF linkage, characterized in that, The system includes a memory and a processor. The memory includes a program for a projector's multi-dimensional adaptive focus control method based on ToF linkage. When the processor executes the program, the projector's multi-dimensional adaptive focus control method implements the following steps: In response to the focus trigger signal, depth information and image information of the target projection surface are acquired; Based on the depth information, fit the spatial plane of the target projection surface, and calculate the projection distance and side projection angle; When the coarse adjustment conditions are met, the main focusing device and the SAM adjustment device are driven to perform feedforward coarse adjustment according to the projection distance and the side projection angle, respectively. Collect multi-area sharpness indicators of the current image; When the coarse adjustment is completed according to the multi-area sharpness index, the SAM adjustment device is fixed and the main focusing device is driven to search for the sharpness peak of the center area. Obtain the sharpness evaluation value for each region and calculate the global sharpness standard deviation; When fine-tuning is triggered based on the global sharpness standard deviation, a step search is performed according to the direction of the difference in sharpness in each region until the gradient sign of the difference in regional sharpness is reversed or the absolute value of the gradient is less than the preset gradient threshold. When the global sharpness value of multiple consecutive frames is determined to meet the sharpness standard, the current focus parameters are locked and the focusing process ends.

8. A multi-dimensional adaptive focusing control system for a projector based on ToF linkage according to claim 7, characterized in that, When the coarse adjustment conditions are met, specifically including: Based on the depth information and the image information, determine the confidence factor of the current scene; Determine whether the confidence factor is greater than a preset scenario confidence threshold; If so, then the coarse adjustment condition is satisfied. If not, it is determined that the coarse adjustment conditions are not met, and the system switches to the preset downgraded focus mode. The downgraded focus mode includes driving the main focus device to traverse and search at a preset step size, using the peak sharpness of the central area of ​​the image as the focus target position.

9. A multi-dimensional adaptive focusing control system for a projector based on ToF linkage according to claim 7, characterized in that, The step of driving the main focusing device and the SAM adjustment device to perform feedforward coarse adjustment according to the projection distance and the side projection angle respectively includes: The system queries a preset main motor step mapping table based on the projection distance to obtain the initial value of the main motor, and then drives the main focusing device to move to the initial value of the main motor. The initial value of the SAM motor is obtained by querying the preset SAM step number mapping table according to the side projection angle, and the SAM adjustment device is driven to move to the initial value of the SAM motor. Obtain multi-region sharpness indices for the current image and calculate the regional sharpness difference value; If the difference in sharpness between the regions is greater than the preset alignment tolerance threshold, then the SAM adjustment device performs unidirectional step compensation according to the direction of the difference in sharpness between the regions until the difference in sharpness between the regions is less than the alignment tolerance threshold or the preset maximum number of compensations is reached.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer-readable storage medium includes a ToF-linked projector multidimensional adaptive focus control method program. When the ToF-linked projector multidimensional adaptive focus control method program is executed by a processor, it implements the steps of the ToF-linked projector multidimensional adaptive focus control method as described in any one of claims 1 to 6.

Images