Targetless projector-camera joint self-calibration method, system, device and product

Abstract

The application provides a target-free projector-camera joint self-calibration method, system, device and product, and belongs to the technical field of optical measurement. The method comprises the following steps: controlling a projector to project a pre-designed optical pattern on a measured object surface; controlling a camera to collect images of the measured object surface covered with the optical pattern from an initial posture and at least two other different postures; based on known information of the optical pattern, establishing a corresponding relationship between feature points in the images collected by the camera and pattern coordinates of the projector; based on the corresponding relationship, establishing a joint optimization objective function comprising a camera imaging model and a projector projection model, the objective function aiming to minimize a reprojection error; using a nonlinear optimization algorithm to solve the joint optimization objective function to simultaneously obtain intrinsic parameters of the camera, intrinsic parameters of the projector and a relative pose between the camera and the projector; and calculating an average reprojection error based on the obtained parameters to evaluate the calibration quality.

Description

Technical Field

[0001] This invention relates to the field of optical measurement technology, and in particular to a targetless projector-camera combined self-calibration method, system, device and product. Background Technology

[0002] Digital Image Correlation (DIC) is a non-contact, full-field optical measurement technique. In DIC systems employing projected speckle, accurately calibrating the geometric relationship between the projector and the camera is a prerequisite for ensuring measurement accuracy.

[0003] Existing projector-camera system calibration methods mainly include the traditional calibration board method and the phase-shift structured calibration method. The traditional calibration board method uses a high-precision checkerboard or dot calibration board, and calculates system parameters by photographing the calibration board in different postures. The phase-shift structured calibration method establishes a dense correspondence between camera and projector pixels by projecting and analyzing a series of phase-shift fringe patterns.

[0004] However, these existing technologies have significant drawbacks in application. First, they all heavily rely on physical calibration targets. In measurement environments of large structures (such as bridges and wind turbine blades), fabricating and deploying massive, high-precision calibration boards is impractical, while using small calibration boards for area-by-area calibration is inefficient and prone to introducing cumulative errors. Second, the calibration and measurement processes are separated. In complex field environments, factors such as temperature changes and equipment vibrations can cause calibration parameters to drift, affecting the accuracy of subsequent measurement data. Finally, existing methods lack online monitoring and adaptive capabilities, making it impossible to verify and correct calibration parameters in real time during the measurement process. Therefore, existing projector-camera calibration technologies suffer from inconvenient operation, low efficiency, difficulty adapting to environmental changes, and the inability to monitor and adaptively adjust calibration quality in real time. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of existing projector-camera calibration technology, which relies too much on physical calibration targets, is complicated to operate and inefficient, and provides a targetless projector-camera joint self-calibration method, system, device and product that does not require physical targets and can quickly calibrate and evaluate quality on site.

[0006] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a targetless projector-camera joint self-calibration method, including: Control the projector to project a pre-designed optical pattern onto the surface of the object being measured; The camera is controlled to acquire images of the surface of the object covered with the optical pattern from one initial posture and at least two other different postures, thereby obtaining the camera-acquired images; Based on the known information of the optical pattern, establish the correspondence between the feature points of the image acquired by the camera and the optical pattern; Based on the correspondence of the feature points, a joint optimization objective function is established, which includes the camera imaging model and the projector projection model. The joint optimization objective function aims to minimize the reprojection error. A nonlinear optimization algorithm is used to solve the joint optimization objective function to simultaneously obtain the intrinsic parameters of the camera, the intrinsic parameters of the projector, and the relative pose between the camera and the projector. Based on the two obtained intrinsic parameters and the relative pose, the average reprojection error, as a quality metric, is calculated to assess the quality of this calibration.

[0007] In some embodiments, the optical pattern is a virtual speckle pattern or a phase-shifted fringe pattern.

[0008] In some embodiments, the nonlinear optimization algorithm is the Levenberg-Marquardt algorithm.

[0009] In some embodiments, the joint optimization objective function is: ; in, This represents the coordinates of the first feature point in the image captured by the camera on the camera's imaging plane. This indicates the coordinates of the second feature point in the optical pattern corresponding to the first feature point on the projector image plane. For camera projection function, For the projection function of the projector, Let be the coordinates of the first feature point and the second feature point in three-dimensional space.

[0010] In some embodiments, it also includes: The average reprojection error is compared with a preset threshold. When the average reprojection error exceeds the preset threshold, a new calibration process is automatically triggered or a prompt is issued to the user.

[0011] In some embodiments, for each of the other postures, the component of its displacement relative to the initial posture in the Z-axis direction of the coordinate system in which the initial posture is located is 0.3 m to 0.5 m.

[0012] In some embodiments, for each of the other postures, it includes translational and rotational motions relative to the initial posture.

[0013] Secondly, this application provides a targetless projector-camera joint self-calibration system, including a projector, a camera, and a processor, wherein the processor is configured to execute the targetless projector-camera joint self-calibration method described in the first aspect to calibrate the projector and the camera.

[0014] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the targetless projector-camera joint self-calibration method described in any one of the above.

[0015] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the targetless projector-camera joint self-calibration method described above.

[0016] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the targetless projector-camera joint self-calibration method described above.

[0017] According to the specific embodiments provided in this application, the following technical effects are disclosed: 1. This invention eliminates the reliance on physical calibration targets, improving applicability and ease of operation. It projects a pre-designed optical pattern onto the surface of the object being measured using a projector. This pattern serves as a virtual calibration target, and images are captured from multiple different orientations using a camera. Therefore, calibration can be completed without the use of any physical calibration target. This makes high-precision calibration possible in complex scenarios such as in the field, at high altitudes, or on large structures where it is difficult or impossible to place a calibration plate.

[0018] 2. Significantly improved calibration efficiency. This invention automates the calibration process, eliminating the tedious and time-consuming steps of traditional methods, such as target placement, position adjustment, and manual feature identification. Users only need to control the camera to perform a few pose changes and take images; the processor can automatically complete the calculation of all calibration parameters, significantly reducing the time required for calibration.

[0019] 3. Online quality assessment and adaptive adjustment are achieved. After obtaining the calibration parameters, this invention calculates the average reprojection error as a quality metric and compares it with a preset threshold. This enables the system to monitor the reliability of the calibration results in real time. When the calibration error exceeds the limit due to environmental changes or other factors, it can promptly warn the user or automatically trigger the recalibration process, ensuring the continuous validity of the measurement data.

[0020] This invention ensures calibration accuracy. By establishing a joint optimization objective function that includes the camera imaging model and the projector projection model, and using a nonlinear optimization algorithm for global optimization, it can simultaneously solve for the camera intrinsic parameters, projector intrinsic parameters, and relative pose. This avoids the accuracy loss caused by error accumulation in step-by-step calibration methods, thereby obtaining high-precision calibration results. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating a targetless projector-camera joint self-calibration method in one embodiment of this application; Figure 2 This is a schematic diagram of the architecture of a targetless projector-camera joint self-calibration system in one embodiment of this application; Figure 3 This is a schematic diagram illustrating the working operation of the projector and camera in one embodiment of this application; Figure 4 This is an example diagram of the operation of a targetless projector-camera joint self-calibration system in one embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] The targetless projector-camera joint self-calibration method provided in this application is executed by a computer device, which can be a terminal computing device or a server. The terminal computing device can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, etc. The server can be a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0026] This application provides a targetless projector-camera joint self-calibration method, aiming to solve the technical problem that the calibration of projector-camera systems in the prior art heavily relies on physical targets, resulting in inconvenient on-site operation, low efficiency, and difficulty in adapting to environmental changes. This method achieves high-precision and high-efficiency on-site calibration through an innovative self-calibration process.

[0027] Reference Figure 1 In one embodiment, the targetless projector-camera joint self-calibration method, executed by a processor, specifically includes: Step S110: Control the projector to project a pre-designed optical pattern onto the surface of the object being measured.

[0028] This step temporarily transforms the surface of the object under test into a "virtual calibration board" with rich and known features, thereby eliminating the dependence on a physical calibration board. In the prior art, the fabrication, transportation, and placement of calibration boards in large-scale structural measurement sites are extremely difficult. This invention cleverly circumvents this problem through active projection.

[0029] Step S120: Control the camera to acquire images of the surface of the object covered with optical patterns from an initial posture and at least two other different postures, and obtain the camera-acquired images.

[0030] This step involves observing the same "virtual calibration board" from multiple different perspectives, providing necessary data redundancy for subsequent solution of system parameters through multi-view geometric constraints. Compared to existing technologies that require repeatedly moving the calibration board or camera to acquire dozens of images, this invention only requires at least three shots, greatly improving data acquisition efficiency.

[0031] Step S130: Based on the known information of the optical pattern, establish the correspondence between the feature points of the camera-acquired image and the optical pattern.

[0032] For example, if the projection is coded speckle, the shape and relative position of each speckle are known. The processor can find these specks in the camera image using an image matching algorithm and directly determine their coordinates in the original projector pattern based on their coded information. This step is a bridge connecting the camera observation space and the projector pattern space, and the correspondence established is the basis for subsequent joint optimization.

[0033] Step S140: Based on the correspondence of feature points, establish a joint optimization objective function that includes the camera imaging model and the projector projection model. The joint optimization objective function aims to minimize the reprojection error.

[0034] Specifically, for each matched feature point pair, it has (unknown) coordinates in 3D space, which are projected onto both the camera image plane and the projector pattern plane. The reprojection error is the difference between its theoretical projected position and its actual observed / known position. A total error function is constructed by summing the reprojection errors of all feature points on both devices. This step incorporates all unknown parameters (camera intrinsics, projector intrinsics, relative pose, and 3D point coordinates) into a unified mathematical model, embodying the concept of "joint" modeling.

[0035] In step S150, a nonlinear optimization algorithm is used to solve the joint optimization objective function to simultaneously obtain the intrinsic parameters of the camera, the intrinsic parameters of the projector, and the relative pose between the camera and the projector.

[0036] Through iterative calculations, the algorithm continuously adjusts these parameters to minimize the total reprojection error. This step, through a one-time global optimization, avoids the accuracy degradation caused by error accumulation in traditional step-by-step calibration methods, thus achieving high-precision parameter estimation.

[0037] Step S160: Based on the two obtained intrinsic parameters and relative pose, calculate the average reprojection error as a quality metric to evaluate the quality of this calibration.

[0038] This error value directly reflects the degree of fit between the calibration model and the actual observation data, and is the core evaluation criterion for assessing the quality of the calibration results. Through this step, the system achieves self-quality monitoring, providing a reliability guarantee for field applications.

[0039] In one specific embodiment, the optical pattern is a virtual speckle pattern or a phase-shifted fringe pattern.

[0040] Virtual speckle patterns mimic natural textures, featuring rich and randomly distributed feature points, making them suitable for feature extraction and matching algorithms based on digital image correlation (DIC). Phase-shifted fringe patterns, on the other hand, are structured coded patterns that establish pixel-level dense correspondences by projecting a series of sinusoidal fringes with different phases onto the surface of the object being measured. Both patterns have their advantages: virtual speckle patterns offer faster acquisition speeds, while phase-shifted fringe patterns provide higher matching accuracy. Users can choose based on their specific application scenarios and accuracy requirements.

[0041] In one specific implementation, the nonlinear optimization algorithm is the Levenberg-Marquardt (LM) algorithm. The LM algorithm is a standard method for solving nonlinear least squares problems. It combines the fast convergence of the Gauss-Newton method with the stability of gradient descent, making it particularly suitable for minimizing reprojection errors. In practice, the LM algorithm can converge to the optimal solution stably and efficiently in most cases. Explicitly stating the use of the LM algorithm provides an efficient implementation scheme that has been widely validated in industry and academia for the relatively general step of "solving the problem using a nonlinear optimization algorithm" in this invention, enhancing the feasibility of the technical solution.

[0042] In one specific implementation, the projector is modeled as an inverse pinhole camera model, thereby constructing the joint optimization objective function as follows: ; in, This represents the coordinates of the first feature point in the image captured by the camera on the camera's imaging plane. This represents the coordinates of the second feature point in the optical pattern, corresponding to the first feature point, on the projector image plane. Transfer function to camera, For the projection function of the projector, Let be the coordinates of the first and second feature points in three-dimensional space.

[0043] The aforementioned joint optimization objective function unifies the camera's intrinsic parameters (in the camera transfer function), the projector's intrinsic parameters (in the projector projection function), and the relative pose within a single optimization framework, providing a mathematical model for achieving the global optimal estimate.

[0044] Furthermore, to improve calibration accuracy, the camera projection function can include distortion parameters describing camera lens distortion. Similarly, the projector projection function can also include distortion parameters. In an ideal pinhole model, a straight line remains a straight line after imaging. However, actual lenses or projection devices exhibit varying degrees of radial and tangential distortion, causing straight lines to be imaged as curves. Ignoring these distortions in the projection function will introduce significant systematic errors. Therefore, by introducing distortion parameters into the projection functions of both the camera and the projector, and solving for these parameters together during joint optimization, the imaging and projection processes can be modeled more accurately, thereby significantly reducing reprojection errors and obtaining more accurate intrinsic and extrinsic parameters and relative pose. This is a key step in achieving high-precision calibration.

[0045] In one specific implementation, the method further includes a step of comparing the calculated average reprojection error with a preset threshold. This step provides an automated, quantitative criterion for judging the validity of the calibration results. For example, a threshold, such as 0.2 pixels, can be set based on experience or experimentation. If the calculated average reprojection error is lower than this threshold, the calibration is considered successful; otherwise, the calibration quality is considered poor. This design transforms the manual judgment that originally required experienced operators into an objective, machine-executable logical judgment, which is the foundation for automating the calibration process.

[0046] Based on the above implementation, when the average reprojection error exceeds a preset threshold, the system can automatically trigger a new calibration process or issue a prompt to the user. This constitutes a complete quality control closed loop. For example, if the error exceeds the standard, the system can display a dialog box prompting "Insufficient calibration accuracy, please try increasing the camera movement range or changing the shooting angle and then recalibrating." Alternatively, in some automated application scenarios, the system can directly control the camera or robotic arm to move to a new position and automatically start a new round of calibration. Through this feedback mechanism, the system possesses the ability to self-correct and adapt to environmental changes, greatly improving the system's intelligence level and robustness in complex field environments.

[0047] In one specific implementation, for each other posture, the component of its displacement relative to the initial posture in the Z-axis direction (depth direction) of the coordinate system in which the initial posture is located is 0.3 meters to 0.5 meters.

[0048] In multi-view geometry, a sufficiently large baseline (the distance between camera centers) is crucial for accurate 3D reconstruction and parameter solving. If the camera movement range is too small, the geometry becomes "weak," the optimization problem becomes ill-conditioned, and the solution becomes difficult or unstable. By constraining sufficient displacement in the depth direction, sufficient parallax between the acquired images from different viewpoints can be ensured, thus providing strong geometric constraints for the joint optimization algorithm and significantly improving the accuracy and robustness of the calibration results.

[0049] In one specific implementation, for each other pose, its relative to the initial pose includes translational and rotational motions.

[0050] Compared to pure translation or pure rotation, composite motion provides richer information on viewpoint changes and geometric constraints. For example, pure translation may struggle to effectively constrain rotation parameters, while pure rotation fails to provide effective depth information. By guiding or requiring users to perform composite motions involving both translation and rotation, unknowns such as camera intrinsic and extrinsic parameters and 3D structure can be more effectively decoupled. This makes joint optimization algorithms more likely to converge to the correct global optimum, further improving calibration accuracy and stability.

[0051] This application also provides a targetless projector-camera joint self-calibration system, which is the physical carrier for implementing the above method.

[0052] Reference Figure 2 In one embodiment, the targetless projector-camera joint self-calibration system includes a processor, a camera, and a projector. The processor, which can be an industrial computer, embedded system, or personal computer, serves as the core control unit of the system, responsible for performing all control and computational tasks. The camera, such as a high-resolution industrial CMOS camera, is used to acquire images of the surface of the object being measured. The projector, such as a DLP (Digital Light Processing) projector, is used to project optical patterns. These three components are interconnected and work together via data cables (such as USB or Ethernet) and control cables.

[0053] In this system, the processor is configured to execute all the steps of the aforementioned method. Specifically, the software program running inside the processor first generates or loads a pre-designed optical pattern and sends instructions to the projector via a control interface, causing it to project the pattern onto the surface of the object being measured. Next, the processor sends trigger signals to the camera via the control interface, controlling it to acquire images at different times (corresponding to different poses), and then sends the acquired digital image data back to the processor's memory. Subsequently, the processor's internal algorithm module processes these images, establishing the correspondence between feature points and pattern coordinates, constructing and solving a joint optimization objective function, and finally calculating all calibration parameters and the average reprojection error. Finally, the processor also executes quality assessment and feedback logic, outputting the calibration results to a display or storage device, or triggering a recalibration process when necessary. In this way, the processor organically organizes the functions of each hardware module, realizing a fully automated targetless self-calibration process.

[0054] Corresponding to the above-described method embodiments, various preferred embodiments of the system can also be implemented. For example, the processor can be configured to generate and control the projector to project virtual speckle patterns or phase-shifting fringe patterns. When using a phase-shifting fringe pattern, the processor is further configured to execute phase-shifting and phase-unfolding algorithms to establish correspondences. The nonlinear optimization algorithm implemented in the processor can be the Levenberg-Marquardt algorithm. The camera imaging model used by the processor can include distortion parameters. The processor is also configured to compare the calculated average reprojection error with a preset threshold and trigger a new calibration process or issue a prompt to the user when the error exceeds the threshold. When controlling the camera to perform multi-pose acquisition, the processor can guide the user or control the robotic arm to ensure that the camera's movement meets specific displacement (e.g., 0.3-0.5 meters in the Z-axis direction) and motion form (including translation and rotation) requirements. These specific configurations enable the system to complete the calibration task efficiently, accurately, and intelligently.

[0055] The following combination Figures 2 to 4 The above-described targetless projector-camera joint self-calibration system and its implementation method will be described through a more specific embodiment.

[0056] First, the system starts up, and the processor controls the projector to project a pre-generated digital speckle pattern onto the surface of any diffuse-reflective object being measured. This speckle pattern has unique, known texture information.

[0057] Next, the operator manually or via a robotic arm controls the camera to acquire images in multiple poses. (Refer to...) Figure 3 The camera first captures an image of the object's surface in its initial pose, obtaining the first frame. Subsequently, the camera moves to poses 2 and 3, capturing a second and third image, respectively. These pose changes are crucial for ensuring calibration accuracy. For example, the movement from pose 1 (the initial pose) to pose 2 can be designed as a displacement of approximately 0.4 meters along the Z-axis (depth direction) of the camera coordinate system, while the movement from pose 1 to pose 3 can be a composite motion involving translation and rotation. The entire acquisition process requires images from at least three different poses.

[0058] After data acquisition, the processor begins data processing. First, the processor extracts features from the three acquired speckle images, using algorithms such as SIFT or ORB to find stable corner points or speckle centers as feature points. Then, using pre-known projection speckle pattern information, through methods such as digital image correlation (DIC) or template matching, it finds the corresponding pixel coordinates in the original projection pattern for each feature point in each camera image. This establishes a large number of "(camera image coordinates) <-> (projector pattern coordinates)" correspondences.

[0059] Based on these correspondences, the processor establishes a joint optimization objective function. This joint optimization objective function is a complex mathematical expression, the core idea of ​​which is to minimize the sum of reprojection errors for all corresponding point pairs.

[0060] The joint optimization objective function is: ; in, This represents the coordinates of the first feature point in the image captured by the camera on the camera's imaging plane. This represents the coordinates of the second feature point in the optical pattern, corresponding to the first feature point, on the projector image plane. For camera projection function, For the projection function of the projector, Let be the coordinates of the first and second feature points in three-dimensional space.

[0061] Subsequently, the processor employs a nonlinear optimization algorithm, such as the Levenberg-Marquardt algorithm, to iteratively solve the joint optimization objective function. After multiple iterations, the algorithm finds a set of optimal parameters (including the intrinsic parameters of the camera and projector, and the relative pose between the camera and projector) that minimize the value of the aforementioned joint optimization objective function. This set of parameters represents the final result of this calibration.

[0062] Finally, to verify the quality of the calibration results, the processor uses the obtained parameters to calculate the average reprojection error of all feature points. The processor's calculated average reprojection error is immediately compared to a preset threshold (e.g., 0.15 pixels for high precision requirements). If the error exceeds this threshold, it indicates poor data quality or geometry in the calibration, leading to unreliable results. In this case, the processor does not directly output a failure result but instead provides the operator with specific information, such as "Insufficient calibration accuracy; average reprojection error is 0.28 pixels. Recommendation: Move the camera back approximately 0.5 meters and tilt it slightly upwards before trying again." In more automated systems, the processor can directly control the robotic arm carrying the camera, automatically planning and moving it to a new, more favorable calibration position based on the error analysis results, and then automatically triggering a new calibration process. Through this closed-loop design of "evaluation-feedback-re-optimization", this invention not only ensures that the final output calibration results are of high quality, but also intelligently guides users or equipment to complete complex calibration tasks, greatly reducing the skill requirements for operators and improving the calibration success rate and robustness in various complex field environments.

[0063] The signaling interaction throughout the process can be found in the following reference. Figure 4After the operator starts the process, the processor coordinates the projector and camera to complete data acquisition, then performs calculations, and finally feeds back the results to the operator.

[0064] Through the above steps, this embodiment does not require any physical calibration plate. It can automatically complete the high-precision calibration of the projector-camera system within minutes by simply taking a few pictures of the surface of the object being tested, which greatly improves the convenience and efficiency of on-site operation.

[0065] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0066] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0067] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0068] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0069] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0070] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0071] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0072] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A targetless projector-camera joint self-calibration method, characterized in that, include: Control the projector to project a pre-designed optical pattern onto the surface of the object being measured; The camera is controlled to acquire images of the surface of the object covered with the optical pattern from one initial posture and at least two other different postures, thereby obtaining the camera-acquired images; Based on the known information of the optical pattern, establish the correspondence between the feature points of the image acquired by the camera and the optical pattern; Based on the aforementioned feature point correspondence, a joint optimization objective function is established, comprising a camera imaging model and a projector projection model. This joint optimization objective function aims to minimize the reprojection error. The joint optimization objective function is as follows: ； in, This represents the coordinates of the first feature point in the image captured by the camera on the camera's imaging plane. This indicates the coordinates of the second feature point in the optical pattern corresponding to the first feature point on the projector image plane. Transfer function to camera, For the projection function of the projector, Let be the coordinates of the first feature point and the second feature point in three-dimensional space; A nonlinear optimization algorithm is used to solve the joint optimization objective function to simultaneously obtain the intrinsic parameters of the camera, the intrinsic parameters of the projector, and the relative pose between the camera and the projector. Based on the two intrinsic parameters and the relative pose, the average reprojection error, as a quality metric, is calculated to assess the quality of this calibration.

2. The targetless projector-camera joint self-calibration method according to claim 1, characterized in that, The optical pattern is a virtual speckle pattern or a phase-shifted fringe pattern.

3. The targetless projector-camera joint self-calibration method according to claim 1, characterized in that, The nonlinear optimization algorithm is the Levenberg-Marquardt algorithm.

4. The targetless projector-camera joint self-calibration method according to claim 1, characterized in that, Also includes: The average reprojection error is compared with a preset threshold. When the average reprojection error exceeds the preset threshold, a new calibration process is automatically triggered or a prompt is issued to the user.

5. The targetless projector-camera joint self-calibration method according to claim 1, characterized in that, For each of the other postures, the component of its displacement relative to the initial posture in the Z-axis direction of the coordinate system in which the initial posture is located is 0.3 m to 0.5 m.

6. The targetless projector-camera joint self-calibration method according to claim 1, characterized in that, For each of the other postures, it includes translational and rotational motions relative to the initial posture.

7. A targetless projector-camera joint self-calibration system, comprising a projector, a camera, and a processor, characterized in that, The processor is configured to calibrate the projector and the camera using the targetless projector-camera joint self-calibration method as described in any one of claims 1-6.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the targetless projector-camera joint self-calibration method according to any one of claims 1-6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the targetless projector-camera joint self-calibration method as described in any one of claims 1-6.

Images