Calibration method of scanning device, calibration system of scanning device and electronic device

Abstract

Embodiments of the present application relate to the technical field of three-dimensional scanning, and particularly relate to a calibration method of a scanning device, a calibration system of the scanning device, and an electronic device. The scanning device comprises a first imaging unit and a second imaging unit. The method comprises: receiving multi-state observation data obtained by the first imaging unit and the second imaging unit for a calibration auxiliary object, the calibration auxiliary object comprising a plurality of observable features, and the plurality of observable features comprising at least two scale reference features with a known scale relationship. Based on the multi-state observation data, observation position information of the plurality of observable features in the first imaging unit and the second imaging unit is determined. Based on the observation position information and the known scale relationship, calibration parameters of the scanning device are determined according to multi-state observation constraints and scale constraints. In this way, the degree of influence of the calibration result by a single observation state, a collection posture, or the overall precision of the calibration auxiliary object can be reduced, and the accuracy of the calibration result of the scanning device can be improved.

Description

Technical Field

[0001] This application relates to the field of 3D scanning technology, and in particular to calibration methods, calibration systems and electronic devices for scanning equipment. Background Technology

[0002] With the development of technologies such as machine vision, 3D measurement, industrial inspection, medical scanning, robot localization, and augmented reality, scanning equipment is widely used to acquire image information, spatial topography information, or pose information of target objects. Some scanning devices include multiple imaging units, which can acquire observation data from different perspectives. The geometric relationship between these imaging units affects the matching, fusion, and spatial calculation results of the data acquired by different imaging units. Inaccurate geometric relationships between the imaging units can lead to distorted scanning results, increased spatial positioning errors, or unstable subsequent measurement results.

[0003] In related technologies, calibration plates, calibration rods, calibration frames, or other calibration aids are typically used to calibrate scanning equipment. During the calibration process, the operator needs to establish different observation states between the scanning equipment and the calibration aids, collect the corresponding observation data, and then calculate the calibration parameters of the scanning equipment based on the collected data.

[0004] However, existing calibration methods typically place high demands on the accuracy, placement, acquisition posture, and operational procedures of calibration aids. When calibration aids are worn, deformed, or contaminated, or when there are obstructions, vibrations, or insufficient posture coverage during the acquisition process, the accuracy of the calibration results can be easily affected. Summary of the Invention

[0005] The purpose of this application is to provide a calibration method, a calibration system, and an electronic device for scanning equipment, so as to effectively improve the accuracy of the calibration results of the scanning equipment.

[0006] The objective of this application is achieved through the following technical solution: In a first aspect, embodiments of this application provide a calibration method for a scanning device, applied to a scanning device including at least two imaging units, the at least two imaging units including a first imaging unit and a second imaging unit, the method including: receiving multi-state observation data obtained by the first imaging unit and the second imaging unit for a calibration auxiliary object, wherein the multi-state observation data is obtained when the scanning device and the calibration auxiliary object form multiple relative observation states, the calibration auxiliary object including multiple observable features, the multiple observable features including at least two scale reference features, the at least two scale reference features having a known scale relationship; based on the multi-state observation... The data determines the observation position information of multiple observable features in the first imaging unit and the second imaging unit, respectively. Based on the observation position information and the known scale relationship, the calibration parameters of the scanning device are determined according to multi-state observation constraints and scale constraints. The calibration parameters of the scanning device include the relative pose parameters between the first imaging unit and the second imaging unit. The multi-state observation constraints are used to constrain the projection relationship of multiple observable features corresponding to the first imaging unit and the second imaging unit in multiple relative observation states. The scale constraints are used to constrain the correspondence between the spatial scale of the at least two scale reference features and the known scale relationship.

[0007] In some embodiments, determining the observation position information of multiple observable features in the first imaging unit and the second imaging unit based on the multi-state observation data includes: performing feature recognition on the multi-state observation data to determine that the multiple observable features correspond to first observation position information of the first imaging unit and second observation position information of the second imaging unit; determining the correspondence between the first observation position information and the second observation position information according to the feature identification information and / or image feature information of the multiple observable features; and associating the image position information of the same observable feature in the first imaging unit and the second imaging unit according to the correspondence to obtain the observation position information.

[0008] In some embodiments, determining the calibration parameters of the scanning device based on the observation location information and the known scale relationship, according to multi-state observation constraints and scale constraints, includes: determining multiple target observation states from the multiple relative observation states based on the observation location information; determining target observation location information associated with the multiple target observation states from the observation location information; and determining the calibration parameters of the scanning device based on the target observation location information and the known scale relationship, according to the multi-state observation constraints and the scale constraints.

[0009] In some embodiments, determining multiple target observation states from the multiple relative observation states based on the observation location information includes: determining an evaluation result for each relative observation state based on the feature observation quality corresponding to each relative observation state and / or the geometric distribution corresponding to the multiple relative observation states; and determining the multiple target observation states from the multiple relative observation states based on the evaluation results. The feature observation quality includes at least one of the number of observable features, the distribution range of observable features, and image sharpness; the geometric distribution includes at least one of the change in observation pose and the change in observation distance.

[0010] In some embodiments, determining the calibration parameters of the scanning device based on the target observation location information and the known scale relationship, according to the multi-state observation constraints and the scale constraints, includes: optimizing the state parameters based on the target observation location information and the known scale relationship to obtain the calibration parameters of the scanning device; wherein, the state parameters include device pose parameters corresponding to multiple target observation states, spatial position parameters of multiple observable features, and calibration parameters to be optimized; the device pose parameters are used to characterize the pose of the scanning device relative to the calibration auxiliary object in the corresponding target observation state, and the calibration parameters to be optimized include the relative pose parameters between the first imaging unit and the second imaging unit.

[0011] In some embodiments, optimizing the state parameters based on the target observation location information and the known scale relationship to obtain the calibration parameters of the scanning device includes: determining the theoretical observation positions of multiple observable features corresponding to the first imaging unit and the second imaging unit, respectively, based on the current values ​​of the state parameters and the imaging model; determining the projection position deviation based on the theoretical observation positions and the target observation location information; determining the computational scale relationship corresponding to the known scale relationship based on at least a portion of the spatial position parameters in the current values ​​of the state parameters; determining the scale relationship deviation based on the computational scale relationship and the known scale relationship; and updating the current values ​​of the state parameters based on the projection position deviation and the scale relationship deviation.

[0012] In some embodiments, updating the current value of the state parameter based on the projection position deviation and the scale relationship deviation includes: determining an objective function based on the projection position deviation and the scale relationship deviation; wherein the objective function is used to characterize the comprehensive error corresponding to the projection position deviation and the scale relationship deviation; iteratively determining the state parameter increment based on the objective function, and updating the current value of the state parameter according to the state parameter increment; and, if a convergence condition is met, determining the calibration parameter to be optimized when the convergence condition is met as the calibration parameter of the scanning device.

[0013] In some embodiments, the step of iteratively determining the state parameter increment based on the objective function and updating the current value of the state parameter according to the state parameter increment includes: in the current iteration, determining the residual vector and Jacobian matrix corresponding to the objective function based on the current value of the state parameter; wherein the residual vector includes the residual corresponding to the projection position deviation and the residual corresponding to the scale relationship deviation; determining the state parameter increment according to the residual vector, the Jacobian matrix, and the damping factor; updating the current value of the state parameter according to the state parameter increment to obtain the next value of the state parameter.

[0014] Secondly, embodiments of this application provide a calibration system for a scanning device, including a control module, the control module being used to execute the calibration method for the scanning device described in any one of the first aspects above.

[0015] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the calibration method of the scanning device as described in any one of the first aspects above.

[0016] This application provides a calibration method, a calibration system, and an electronic device for scanning equipment. Addressing the issue that calibration results of scanning equipment are easily affected by calibration conditions and operational processes in related technologies, this application improves the accuracy of calibration results by acquiring multi-state observation data of the scanning equipment and the calibration auxiliary object under multiple relative observation states, and determining the calibration parameters of the scanning equipment according to multi-state observation constraints and scale constraints based on the observation position information of observable features and the known scale relationship between scale reference features. Attached Figure Description

[0017] This application will be further described below with reference to the accompanying drawings and specific embodiments.

[0018] Figure 1 This is a schematic diagram of a calibration plate provided in an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of a calibration rod provided in an embodiment of this application.

[0020] Figure 3 This is a flowchart illustrating a calibration method for a scanning device provided in an embodiment of this application.

[0021] Figure 4 This is a schematic diagram of a calibration auxiliary object provided in an embodiment of this application.

[0022] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0025] In related technologies, scanning devices with multiple imaging units typically require calibration before use to determine the relative pose relationships between the different imaging units. For example, for scanning devices including binocular imaging units, it is usually necessary to determine the rotational and translational relationships between the first and second imaging units. This relative pose relationship affects the matching of data acquired by different imaging units and the spatial calculation results, thereby affecting the scanning accuracy and positioning accuracy of the scanning device.

[0026] Please see Figure 1 and Figure 2 In related technologies, calibration auxiliary tools such as calibration plates or calibration rods are usually used to calibrate scanning equipment. Figure 1 The diagram illustrates a calibration plate with multiple marker points. Figure 2The diagram illustrates a calibration rod that can provide a certain length reference. In some specific implementations, the positional or length relationships of the marked points on the calibration plate or calibration rod are usually obtained in advance. Then, based on this known position and length information, as well as the observation positions acquired by the imaging units, the rotational and translational relationships between different imaging units are deduced.

[0027] However, this type of calibration typically relies on high-precision, specialized calibration tools. For example, multiple markers on the calibration plate usually need to be precisely machined to their preset positions, and the endpoints, markers, or length references on the calibration rod also need to have high dimensional accuracy. If the calibration plate or calibration rod is bumped, deformed, worn, or soiled during manufacturing, assembly, transportation, or use, its actual positional or length relationships may deviate from the preset values. In this case, if the calibration plate or calibration rod is still used as an accurate reference for calculation, the error of the calibration tool itself will be introduced into the calibration parameters of the scanning equipment.

[0028] Furthermore, the calibration process in related technologies typically places high demands on the operator's methods. To obtain good calibration data, operators often need to establish multiple specific observation states between the scanning equipment and the calibration plate or rod, such as different observation angles, different observation distances, and different field-of-view positions. During data acquisition, the calibration aids or scanning equipment must also remain stable to avoid the influence of jitter, occlusion, or posture deviations on the observation data. If the acquisition posture is too uniform, the field-of-view coverage is insufficient, or the marker points are mainly concentrated in local areas of the image, the acquired data will be insufficient to adequately constrain the relative pose relationships between different imaging units, easily leading to unstable calculation results.

[0029] Furthermore, in on-site calibration scenarios, the calibration process is easily affected by factors such as space constraints, environmental changes, equipment vibration, temperature variations, and the operator's skill level. For handheld scanning devices, tracking scanning devices, or scanning devices requiring on-site maintenance, operators often find it difficult to consistently maintain the ideal position and posture for multiple data acquisitions. Once the scanning device is subjected to collisions, displacement, or changes in installation status, recalibration usually requires a more complex process, resulting in low calibration efficiency and increased uncertainty in the calibration results.

[0030] Therefore, the calibration of scanning equipment typically relies on high-precision calibration plates or rods, and has high requirements for observation conditions, acquisition posture, and operation procedures. When the actual state of the calibration aids, the acquisition process, or the quality of the observation data deviates from ideal conditions, it can easily affect the calculation results of the relative pose parameters between different imaging units, resulting in low accuracy of the calibration results of the scanning equipment.

[0031] Based on this, embodiments of this application provide a calibration method for a scanning device. This method utilizes observation data from multiple relative observation states to impose multi-state constraints on the calibration parameters of the scanning device, and uses known scale relationships within the calibration auxiliary object to constrain the spatial scale. This reduces the dependence of the calibration results on the overall accuracy of a single observation state or the calibration auxiliary object, thereby improving the accuracy of the scanning device's calibration results. Figure 3 The flowchart shown in this application embodiment illustrates a calibration method for a scanning device. This method is applied to a scanning device, which includes at least two imaging units, including a first imaging unit and a second imaging unit. The method may include steps S101-S103: S101: Receive multi-state observation data obtained by the first imaging unit and the second imaging unit for the calibration auxiliary object. The multi-state observation data is obtained when the scanning device and the calibration auxiliary object form multiple relative observation states. The calibration auxiliary object includes multiple observable features, each including at least two scale reference features, and the at least two scale reference features have a known scale relationship.

[0032] S102: Based on multi-state observation data, determine the observation position information of multiple observable features in the first imaging unit and the second imaging unit respectively.

[0033] S103: Based on the observation location information and known scale relationships, determine the calibration parameters of the scanning equipment according to multi-state observation constraints and scale constraints.

[0034] The calibration parameters of the scanning device include the relative pose parameters between the first imaging unit and the second imaging unit. The multi-state observation constraint is used to constrain the projection relationship between multiple observable features and the first and second imaging units in multiple relative observation states. The scale constraint is used to constrain the correspondence between the spatial scale and the known scale relationship between at least two scale reference features.

[0035] In some embodiments, the scanning device may be a device for scanning, measuring, locating, or reconstructing a target object in three dimensions. For example, the scanning device may be a handheld scanner, a tracking scanner, a 3D scanner, or other device with spatial measurement capabilities.

[0036] The imaging units included in the scanning device can be functional units used to acquire observation data of a calibration auxiliary object or a target object. At least two imaging units can acquire observation data of the calibration auxiliary object from different perspectives. In some embodiments, the scanning device can be a tracking scanner, which can include a tracking head that may contain a first camera and a second camera. The first camera can serve as a first imaging unit, and the second camera can serve as a second imaging unit. The first and second cameras can observe the calibration auxiliary object from different perspectives to obtain multi-state observation data for determining the calibration parameters of the scanning device. In this embodiment, the calibration parameters of the scanning device may include relative pose parameters between the first and second imaging units. Relative pose parameters can be used to characterize the rotational and translational relationships between the first and second imaging units. Therefore, after determining the relative pose parameters, the scanning device can perform spatial positioning, 3D reconstruction, or tracking calculations based on the observation data acquired by the first and second imaging units.

[0037] A calibration aid is an object used to assist the scanning equipment in calibration. The calibration aid can be observed by both the first and second imaging units, providing an observational and dimensional basis for calculating the calibration parameters of the scanning equipment. For example, a calibration aid can be a calibration plate, calibration rod, calibration frame, calibration plane, or a carrier with marked patterns, or other structures that can be observed by the scanning equipment. Figure 4 As shown, the calibration aid can be a foldable calibration plate, as illustrated in the figure. This foldable calibration plate can include multiple calibration areas, which can be connected by a connecting structure for easy storage, carrying, or unfolding. When unfolded, the multiple calibration areas can collectively form a larger observable range, allowing the scanning device to observe multiple marking patterns on the calibration plate from different observation angles, distances, or fields of view. Therefore, the calibration process can proceed without relying on a dedicated calibration plate manufactured with high precision, nor does it require pre-obtaining the precise three-dimensional coordinates of all observable features.

[0038] Figure 4 The calibration rod shown can be used to provide a scale reference. Specifically, the calibration rod can have at least two scale reference positions, and the distance between the at least two scale reference positions can be used as a known scale relationship. Thus, during the calibration process, the known scale relationship provided by the calibration rod can be used to constrain the spatial scale.

[0039] In other embodiments, if the calibration plate itself can provide known scale relationships, the calibration aid may not include calibration rods. For example, the actual distance between at least two marker patterns on the calibration plate, the actual spacing between adjacent marker patterns, and the actual size of the area formed by multiple marker patterns can all be used as known scale relationships. In this case, at least two marker patterns on the calibration plate can be used as scale reference features to constrain the spatial scale.

[0040] Multi-state observation data is obtained when the scanning device and the calibration aid form multiple relative observation states. Relative observation states characterize the relative observation relationship between the scanning device and the calibration aid. For example, relative observation states can be related to the scanning device's observation angle, observation distance, observation position, observation attitude, or field of view coverage relative to the calibration aid. Multiple relative observation states can be formed by moving the scanning device, moving the calibration aid, changing the orientation of the scanning device, changing the distance between the scanning device and the calibration aid, or changing the observation area of ​​the scanning device relative to the calibration aid.

[0041] In some implementations, multiple relative observation states can be formed during the continuous movement of the scanning device relative to the calibration auxiliary object. For example, the operator can hold the scanning device and continuously scan relative to the calibration auxiliary object, allowing the scanning device to form different observation angles, observation distances, or observation postures at different sampling times. Therefore, embodiments of this application do not require the operator to place the scanning device or calibration auxiliary object in multiple preset fixed poses for static data acquisition.

[0042] In some embodiments, multi-state observation data may include image data acquired by the first imaging unit and the second imaging unit in multiple relative observation states. In other embodiments, multi-state observation data may also include feature observation data extracted from image data, such as feature point coordinates, feature center positions, feature contour information, or feature matching information. Thus, multi-state observation data is not limited to the original image data, but may also be intermediate observation data obtained after preprocessing the original image data.

[0043] The calibration aids include multiple observable features. Observable features are those that can be observed, identified, or located by the first imaging unit and the second imaging unit. For example, observable features can be marker points, corner points, dots, line segments, patterns, coded markers, reflective markers, contour features, or other features that can determine their location in the observation data.

[0044] Combination Figure 4Multiple ring marks on the calibration plate can serve as observable features. When the scanning device and the calibration aid form multiple relative observation states, the first and second imaging units can observe these ring marks respectively and obtain the observation position information of each ring mark under different relative observation states. Multi-state observation constraints can be used to constrain the projection relationship of these ring marks to the first and second imaging units under multiple relative observation states. If the calibration parameters of the scanning device are inaccurate, the theoretical observation position of the ring marks determined based on these calibration parameters is usually difficult to match simultaneously with the actual observation positions of multiple ring marks under multiple relative observation states. Therefore, the accuracy of the calibration parameters of the scanning device can be improved through multi-state observation constraints.

[0045] Multiple observable features include at least two scale reference features. Scale reference features can be observable features used to provide a scale reference, and there is a known scale relationship between the at least two scale reference features. The known scale relationship is a pre-determined relationship used to characterize the spatial scale relationship between the at least two scale reference features. For example, a known scale relationship may include the actual distance between two scale reference features, the actual spacing between multiple scale reference features, the actual size formed by at least three scale reference features, or other geometric scale relationships between the scale reference features. Continue reading Figure 4 You can select Figure 4 At least two ring markers are used as scale reference features, and the actual distance between these two ring markers is predetermined. During calibration, the calculated distance between these two ring markers can be determined based on state parameters, and scale constraints can be used to match the calculated distance with the actual distance. Thus, even without... Figure 4 The three-dimensional coordinates of all the ring marks are used as high-precision known quantities, and spatial scale can also be constrained by the locally known scale relationships.

[0046] Observation location information is used to characterize the position of an observable feature in the observation results of an imaging unit. For example, observation location information may include the image coordinates of the observable feature in the image corresponding to the first imaging unit, and the image coordinates of the observable feature in the image corresponding to the second imaging unit. Observation location information may also include the distortion-corrected image position, normalized image coordinates, feature center position, or other information that can characterize the observation location of the observable feature.

[0047] Specifically, under multiple relative observation states, the first imaging unit and the second imaging unit can respectively observe the calibration auxiliary object. Since the calibration auxiliary object has multiple observable features, the observation positions of each observable feature in the first imaging unit and the observation positions of each observable feature in the second imaging unit can be determined from the multi-state observation data. Thus, the observation position information formed by multiple observable features under multiple relative observation states can provide a multi-state observation basis for calculating the calibration parameters of the scanning equipment.

[0048] The calibration parameters of a scanning device are used to characterize the geometric relationships between at least some structures within the scanning device, or as data processing parameters for subsequent scanning, measurement, positioning, and reconstruction. The calibration parameters of a scanning device may include relative pose parameters between a first imaging unit and a second imaging unit. Relative pose parameters are parameters used to characterize the relative positional and orientational relationships between the first and second imaging units. For example, relative pose parameters may include rotation parameters and translation parameters. The rotation parameters can be used to characterize the rotational relationship between the first and second imaging units, and the translation parameters can be used to characterize the translational relationship between the first and second imaging units.

[0049] Multi-state observation constraints can be used to constrain the projection relationships of multiple observable features corresponding to the first and second imaging units under multiple relative observation states. In other words, when determining the calibration parameters of the scanning device, the spatial positions of multiple observable features under different relative observation states, the relative observation states between the scanning device and the calibration auxiliary object, and the relative pose parameters between the first and second imaging units can be matched with the actual observation position information observed by the first and second imaging units. Therefore, observation data from multiple relative observation states can jointly constrain the calibration parameters of the scanning device, thereby reducing the problem of insufficient constraint on calibration parameters caused by relying on only a single observation state.

[0050] Scale constraints are used to constrain the correspondence between the spatial scale of at least two scale reference features and a known scale relationship. In other words, when determining the calibration parameters of a scanning device, the spatial scale determined based on the spatial positions of at least two scale reference features can be matched with the known scale relationship between those at least two scale reference features. Therefore, the spatial scale can be constrained using locally verifiable scale relationships within the calibration aid object, reducing the dependence of the calibration results on the overall high-precision structure of the calibration aid object.

[0051] In some implementations, error terms corresponding to multi-state observation constraints can be established based on observation location information, and error terms corresponding to scale constraints can be established based on known scale relationships. The calibration parameters of the scanning device are then determined based on these error terms. For example, the theoretical observation positions of multiple observable features in the first and second imaging units can be matched with their corresponding observation location information, and the calculated scale relationship between at least two scale reference features can be matched with known scale relationships, thereby determining the calibration parameters of the scanning device.

[0052] For example, targeting Figure 4 The circular markers in the image can be used to establish error terms corresponding to multi-state observation constraints based on the differences between the theoretical and actual observation positions of multiple circular markers in the first and second imaging units; and error terms corresponding to scale constraints can be established based on the differences between the calculated distances and known actual distances between the circular markers selected as scale reference features. By jointly considering the above error terms, the calibration parameters of the scanning equipment can be determined.

[0053] In this embodiment, by utilizing multi-state observation data obtained under multiple relative observation states, the calibration parameters of the scanning device can be constrained by the projection relationships under multiple observation states. Simultaneously, by utilizing the known scale relationships between at least two scale reference features in the calibration auxiliary object, spatial scale constraints can be provided. Therefore, the calibration process of the scanning device does not necessarily rely solely on a single observation state, nor does it depend entirely on the high-precision geometric model of the entire calibration auxiliary object. This reduces the degree to which the calibration results are affected by a single observation state, acquisition posture, or the overall accuracy of the calibration auxiliary object, thereby improving the accuracy of the scanning device's calibration results.

[0054] In some embodiments, determining the observation position information of multiple observable features in a first imaging unit and a second imaging unit based on multi-state observation data may include: firstly, performing feature recognition on the multi-state observation data to determine the first observation position information of the multiple observable features corresponding to the first imaging unit and the second observation position information corresponding to the second imaging unit. Feature recognition can refer to the process of detecting, locating, or extracting observable features from multi-state observation data. For example, when the observable features are marker points, ring markers, coded markers, or reflective markers, the position of the observable features in the observation data of the corresponding imaging unit can be determined by methods such as image detection, contour recognition, center fitting, code parsing, or feature matching.

[0055] The first observation location information can be used to characterize the position of the observable feature in the observation data corresponding to the first imaging unit. The second observation location information can be used to characterize the position of the observable feature in the observation data corresponding to the second imaging unit. For example, the first observation location information may include the image coordinates of the observable feature in the image acquired by the first imaging unit, and the second observation location information may include the image coordinates of the observable feature in the image acquired by the second imaging unit. In some embodiments, the first and second observation location information may also include the image position after distortion correction, normalized image coordinates, feature center position, or contour fitting position.

[0056] In some embodiments, the method may further include determining the correspondence between the first observation location information and the second observation location information based on feature identification information and / or image feature information of multiple observable features. Feature identification information may be information used to distinguish different observable features. For example, when the observable feature is a coded marker, the feature identification information may include the number, code value, or identifier corresponding to the coded marker. When the observable feature is a marker point with a preset arrangement pattern, the feature identification information may also include the sequence number or arrangement position of the marker point on the calibration auxiliary object. Image feature information may be information used to describe the image representation of the observable feature in the observation data. For example, image feature information may include the shape, size, grayscale distribution, contour features, corner features, center position, relative positional relationship between adjacent features, or other information that can be used to identify and match observable features.

[0057] The correspondence is used to characterize which observation positions in the first and second observation position information belong to the same observable feature. In other words, through this correspondence, the matching relationship between an observable feature observed by the first imaging unit and the same observable feature observed by the second imaging unit can be determined. For example, if a ring-shaped marker is identified in both the first and second imaging units, its first observation position in the first imaging unit and its second observation position in the second imaging unit can be determined based on the marker's feature identification information, image feature information, or its relative positional relationship with other observable features.

[0058] After establishing the correspondence, the image position information of the same observable feature in the first and second imaging units can be associated to obtain observation position information. Image position information can be the position data of the observable feature in the corresponding image of the imaging unit. Associating the image position information of the same observable feature in the first and second imaging units forms paired observation information for subsequent calibration parameter calculations. Therefore, the observation results of multiple observable features in the first and second imaging units can be associated according to the same observable feature, reducing the impact of mismatches between different observable features on calibration parameter calculations. This allows subsequent multi-state observation constraints to be established based on accurate observation position information, thereby improving the accuracy of the scanning equipment's calibration results.

[0059] In some implementations, the calibration parameters of the scanning device are determined based on the observation location information and known scale relationships, in accordance with multi-state observation constraints and scale constraints. This may include determining multiple target observation states from multiple relative observation states based on the observation location information.

[0060] The target observation state can be one of multiple relative observation states used to participate in the calculation of calibration parameters. In other words, when the scanning equipment and the calibration auxiliary object form multiple relative observation states, the more suitable relative observation state for determining the calibration parameters can be selected as the target observation state based on the observation position information.

[0061] In some implementations, if multiple relative observation states originate from a continuous scanning process, the target observation state can also be understood as a virtual marker location selected from the continuous scanning data. This virtual marker location can be determined based on observation location information, characteristic observation quality, and / or geometric distribution, without needing to be pre-placed by the operator.

[0062] Specifically, observation location information can reflect the observation status of multiple observable features under different relative observation states. For example, observation location information can reflect the number of observable features successfully observed under a certain relative observation state, the distribution of observable features in the first and second imaging units, and the correlation of the same observable feature in the first and second imaging units. Based on the above observation location information, it can be determined whether each relative observation state can provide effective constraints for the calibration parameters of the scanning device, and multiple target observation states can be determined from multiple relative observation states.

[0063] In some implementations, a relative observation state that meets preset screening criteria can be determined as the target observation state. The preset screening criteria can be used to characterize whether the observation data corresponding to a relative observation state is suitable for participating in calibration parameter calculation. For example, if a relatively observable feature can be observed under a certain relative observation state, or if the observation location information corresponding to that relative observation state can provide sufficient geometric constraints together with other relative observation states, then that relative observation state can be determined as the target observation state. This application does not specifically limit this aspect.

[0064] In some implementations, after determining the observation states of multiple targets, the target observation location information associated with the observation states of multiple targets can be determined from the observation location information.

[0065] The target observation location information can be a portion of the observation location information corresponding to the target observation state. In other words, the observation location information can include the observation locations of multiple observable features corresponding to the first imaging unit and the second imaging unit respectively under multiple relative observation states; after determining multiple target observation states, the observation locations formed under these target observation states can be extracted from the observation location information as the target observation location information.

[0066] For example, in the case of multiple relative observation states, including a first relative observation state, a second relative observation state, and a third relative observation state, if the first and third relative observation states are determined as the target observation states based on the observation location information, the observation locations corresponding to the first and third relative observation states can be extracted from the observation location information to obtain the target observation location information. Therefore, the target observation location information can serve as the data basis for subsequently determining calibration parameters according to multi-state observation constraints.

[0067] In some implementations, the calibration parameters of the scanning device can be determined based on target observation location information and known scale relationships, according to multi-state observation constraints and scale constraints. Specifically, the target observation location information can be used to establish the projection relationship between multiple observable features in the target observation state and the first and second imaging units. The known scale relationships can be used to establish spatial scale constraints between at least two scale reference features. By using the multi-state observation constraints corresponding to the target observation location information and the scale constraints corresponding to the known scale relationships together to determine the calibration parameters, the calibration parameters of the scanning device can simultaneously satisfy the observation matching relationship in multiple target observation states and the scale relationship between scale reference features.

[0068] Therefore, by determining the target observation state from multiple relative observation states and determining the calibration parameters of the scanning device based on the target observation position information associated with the target observation state, the influence of poor quality or weakly constrained relative observation states on the calculation of calibration parameters can be reduced. This makes the observation data involved in the calculation more conducive to constraining the relative pose parameters between the first imaging unit and the second imaging unit, thereby improving the accuracy and stability of the calibration results of the scanning device.

[0069] In some embodiments, determining multiple target observation states from multiple relative observation states based on observation location information may include: determining an evaluation result for each relative observation state based on the characteristic observation quality corresponding to each relative observation state and / or the geometric distribution of multiple relative observation states; and determining multiple target observation states from multiple relative observation states based on the evaluation results.

[0070] Feature observation quality characterizes the quality at which observable features are observed, identified, or located by the first and second imaging units under corresponding relative observation conditions. Higher feature observation quality indicates that the observation location information formed under that relative observation condition is more suitable for participating in the calculation of calibration parameters for the scanning equipment. Feature observation quality includes at least one of the following: the number of observable features, the distribution range of observable features, and image sharpness; geometric distribution includes at least one of the changes in observation attitude and the changes in observation distance.

[0071] The number of observable features can be used to characterize the number of observable features that are effectively identified under a corresponding relative observation state. The distribution range of observable features can be used to characterize the distribution of observable features in the observation results of the imaging unit. Image sharpness can be used to characterize the clarity of the observation data under a corresponding relative observation state. For example, under a certain relative observation state, if the first imaging unit and the second imaging unit can jointly observe a large number of observable features, the multiple observable features are relatively dispersed in the image area, and the observation data is relatively clear, then this relative observation state can provide better observation constraints.

[0072] Geometric distribution can be used to characterize the distribution of multiple relative observation states in terms of observation attitude, observation distance, or observation spatial range. Geometric distribution can reflect whether multiple relative observation states can jointly form sufficient geometric constraints.

[0073] In some implementations, the geometric distribution may include at least one of the changes in observation attitude and the changes in observation distance.

[0074] The change in observation attitude can be used to characterize the degree of change in the observation angle or attitude of the scanning device relative to the calibration auxiliary object between different relative observation states. For example, when multiple relative observation states correspond to different pitch angles, yaw angles, or roll angles, multiple observable features can form different projection relationships from different perspectives, thereby enhancing the constraints of multi-state observation.

[0075] The change in observation distance can be used to characterize the degree of variation in the observation distance between the scanning device and the calibration auxiliary object under different relative observation states. For example, when multiple relative observation states cover different observation distances, it can provide richer constraints on the observation position information in the scale and depth directions, which helps to reduce the instability of calibration parameters caused by a single observation state.

[0076] In some implementations, an evaluation result can be determined for each relative observation state based on the quality and / or geometric distribution of the feature observations. The evaluation result can be used to characterize the suitability of the corresponding relative observation state for participating in the calibration parameter calculation. For example, the evaluation score, evaluation level, or evaluation result indicating whether preset conditions are met can be determined based on at least one of the following: the number of observable features, the distribution range of observable features, image sharpness, the amount of change in observation posture, and the amount of change in observation distance.

[0077] In some implementations, multiple target observation states can be determined from multiple relative observation states based on the evaluation results. For example, the relative observation state with a higher evaluation score can be determined as the target observation state; the relative observation state that meets the preset evaluation conditions can also be determined as the target observation state; or, while ensuring that there are certain differences in observation posture and observation distance among the multiple target observation states, the relative observation state with higher feature observation quality can be selected as the target observation state.

[0078] In some implementations, if a relative observation state is subject to jitter, occlusion, blurring, or insufficient observable features, the feature observation quality corresponding to that relative observation state can be low. When determining the target observation state, this relative observation state may not be selected as the target observation state, thereby reducing the impact of low-quality observation data on the calculation of calibration parameters.

[0079] Therefore, by combining the quality of feature observations and / or the geometric distribution to determine the target observation state, the target observation position information corresponding to the target observation state can have better feature recognition quality and geometric constraint capability, reducing the impact of factors such as ambiguity, occlusion, insufficient number of features, single observation posture or single observation distance on the calculation of calibration parameters, thereby improving the accuracy and stability of the calibration results of the scanning equipment.

[0080] In some implementations, the calibration parameters of the scanning device are determined based on the target observation location information and known scale relationships, according to multi-state observation constraints and scale constraints. This may include optimizing the state parameters based on the target observation location information and known scale relationships to obtain the calibration parameters of the scanning device.

[0081] State parameters can be a set of parameters to be calculated for calibration parameters. In other words, state parameters not only include the calibration parameters that the scanning device ultimately needs to output, but can also include intermediate parameters describing the observation states of multiple targets and the spatial relationships of multiple observable features. By optimizing these parameters together as state parameters, the observation results under multiple target observation states, the spatial positions of observable features, and the relative pose relationships between the first and second imaging units can be coordinated in the same calculation process.

[0082] In this embodiment, the state parameters include the device pose parameters corresponding to multiple target observation states, the spatial position parameters of multiple observable features, and the calibration parameters to be optimized.

[0083] Device pose parameters are used to characterize the pose of the scanning device relative to the calibration auxiliary object in a corresponding target observation state. Specifically, different target observation states correspond to different relative observation relationships between the scanning device and the calibration auxiliary object; therefore, each target observation state can correspond to a set of device pose parameters. Device pose parameters may include rotation and translation parameters of the scanning device relative to the calibration auxiliary object, or equivalent pose expressions may be used; this application does not specifically limit this. By setting device pose parameters corresponding to multiple target observation states, the relative position and relative attitude of the scanning device when observing the calibration auxiliary object in multiple target observation states can be described.

[0084] Spatial location parameters are used to characterize the positions of multiple observable features in space. To ensure that these observable features form consistent spatial constraints under different target observation states, the spatial positions of multiple observable features can be considered as one of the optimization parameters. Spatial location parameters can include the three-dimensional coordinates of the observable features in the coordinate system corresponding to the calibration auxiliary object, or other parameter forms that can characterize the spatial positions of the observable features.

[0085] The calibration parameters to be optimized can be at least some of the calibration parameters of the scanning device that need to be optimized. In this embodiment, the calibration parameters to be optimized include the relative pose parameters between the first imaging unit and the second imaging unit. The relative pose parameters can be used to characterize the rotational and translational relationships between the first and second imaging units. Thus, by optimizing the calibration parameters to be optimized, calibration parameters of the scanning device for subsequent spatial positioning, 3D reconstruction, or tracking calculations can be obtained.

[0086] In some implementations, optimizing state parameters based on target observation location information and known scale relationships can be understood as adjusting the device pose parameters, spatial position parameters, and calibration parameters to be optimized, so that the observation results determined based on these state parameters match the target observation location information, and the scale relationships determined based on the spatial position parameters match the known scale relationships. In other words, the target observation location information can be used to constrain the projection relationships of multiple observable features corresponding to the first and second imaging units under multiple target observation states; the known scale relationships can be used to constrain the spatial scale between at least two scale reference features. Through the combined effect of these two types of constraints, calibration parameters for the scanning device that satisfy multi-state observation constraints and scale constraints can be obtained.

[0087] For example, under multiple target observation states, the scanning device can observe and calibrate auxiliary objects from different observation angles or distances. If a certain set of state parameters cannot accurately characterize the device pose, the spatial position of observable features, and the relative pose relationship between the first and second imaging units, then the observation results of the observable features determined based on that set of state parameters are usually difficult to match with the target observation position information. Accordingly, the calibration parameters of the scanning device can be obtained by optimizing the state parameters so that the observation matching relationship and scale relationship under multiple target observation states simultaneously tend to be consistent.

[0088] In this embodiment, by incorporating the device pose parameters corresponding to multiple target observation states, the spatial position parameters of multiple observable features, and the calibration parameters to be optimized into the state parameters for optimization, the problem of insufficient constraints caused by relying solely on a single observation state or solving for a single type of parameter can be reduced. Therefore, the target observation position information and known scale relationships under multiple target observation states can jointly participate in the calculation of the scanning device's calibration parameters, thereby improving the accuracy and stability of the scanning device's calibration results.

[0089] In some implementations, the state parameters are optimized based on the target observation location information and known scale relationships to obtain the calibration parameters of the scanning device. This may include: determining the theoretical observation positions of multiple observable features corresponding to the first imaging unit and the second imaging unit, respectively, based on the current values ​​of the state parameters and the imaging model.

[0090] The current values ​​of the state parameters refer to the values ​​of the device pose parameters, spatial position parameters, and calibration parameters to be optimized used in the current calculation stage during the optimization process. Since the state parameters usually need to be updated multiple times to obtain a better result, the corresponding theoretical observation results can be calculated based on the current values ​​of the state parameters before each update.

[0091] An imaging model can be used to characterize the mapping relationship between a spatial point and the observation position of an imaging unit. For example, the imaging model may include intrinsic parameters, distortion parameters, projection models, or other models used to map spatial positions to observation positions for a first and a second imaging unit. This application does not specifically limit this aspect.

[0092] The theoretical observation position can be the predicted position of the observable feature in the corresponding imaging unit, calculated based on the current values ​​of the state parameters and the imaging model. Specifically, the spatial position of the observable feature can be determined based on the spatial position parameters, and the spatial position of the observable feature can be transformed into the observation coordinate system corresponding to the first or second imaging unit based on the device pose parameters and the calibration parameters to be optimized. Then, the theoretical observation position of the observable feature corresponding to the first or second imaging unit can be determined based on the imaging model.

[0093] In some implementations, the projection position deviation can be determined based on the theoretical observation position and the target observation position information.

[0094] Projection position deviation can be used to characterize the difference between the theoretical observation position and the target observation position information. In other words, projection position deviation reflects the degree of matching between the theoretical and actual observation results of an observable feature under the current values ​​of the state parameters. For example, if there is a deviation between the theoretical observation position of an observable feature in the first imaging unit and the actual observation position in the target observation position information, the corresponding projection position deviation can be determined based on this deviation. Similarly, the projection position deviation of the observable feature in the second imaging unit can also be determined.

[0095] In some implementations, the computational scale relationship corresponding to a known scale relationship can be determined based on at least some of the spatial location parameters in the current values ​​of the state parameters.

[0096] The computational scale relationship can be a scale relationship calculated based on the current values ​​of state parameters and used for comparison with known scale relationships. Since spatial location parameters characterize the spatial locations of multiple observable features, the spatial scale between at least two scale reference features can be calculated based on their corresponding spatial location parameters. For example, if the known scale relationship is the actual distance between two scale reference features, the computational distance between them can be calculated based on their corresponding spatial location parameters, and this computational distance can be used as the computational scale relationship. If the known scale relationship is the spacing, region size, or other geometric scale relationship between multiple scale reference features, the corresponding computational scale relationship can also be determined based on the spatial location parameters of the respective scale reference features.

[0097] In some implementations, the scale relationship deviation can be determined based on the calculated scale relationship and the known scale relationship.

[0098] Scale relationship deviation can be used to characterize the difference between the calculated scale relationship and the known scale relationship. In other words, scale relationship deviation reflects whether the scale relationship determined by the spatial location parameters, given the current values ​​of the state parameters, is consistent with the pre-determined scale relationship in the calibration auxiliary object. For example, if the known scale relationship between at least two scale reference features is the actual distance, but the calculated distance obtained based on the current values ​​of the state parameters is inconsistent with this actual distance, then the difference between the two can be considered as scale relationship deviation.

[0099] In some implementations, the current values ​​of the state parameters can be updated based on projection position deviation and scale relationship deviation. Specifically, projection position deviation can reflect the observation matching error under multi-state observation constraints, while scale relationship deviation can reflect the spatial scale error under scale constraints. By simultaneously considering projection position deviation and scale relationship deviation, the device pose parameters, spatial position parameters, and calibration parameters to be optimized can be updated, enabling the updated state parameters to better match the target observation position information and making the calculated scale relationship closer to the known scale relationship.

[0100] For example, a large deviation in projection position indicates that the current value of the state parameters is insufficient to accurately describe the projection relationship of multiple observable features in the first and second imaging units; similarly, a large deviation in scale relationship indicates that the current value of the state parameters is insufficient to accurately maintain the spatial scale between scale reference features. By updating the current value of the state parameters based on these deviations, the device pose parameters, spatial position parameters, and calibration parameters to be optimized can be gradually corrected, thereby obtaining more accurate calibration parameters for the scanning device.

[0101] In this embodiment, by determining the theoretical observation position based on the current values ​​of the state parameters and the imaging model, and combining this with the target observation position information to determine the projection position deviation, the state parameters can be constrained using observation data from multiple target observation states. Simultaneously, by determining the computational scale relationship based on the spatial position parameters, and combining this with the known scale relationship to determine the scale relationship deviation, the spatial scale can be constrained using the scale relationship between scale reference features. Therefore, the state parameter update process can simultaneously consider both observation matching relationships and scale relationships, thereby improving the accuracy and stability of the scanning equipment's calibration results.

[0102] In some implementations, updating the current values ​​of the state parameters based on the projection position deviation and scale relationship deviation may include: determining an objective function based on the projection position deviation and scale relationship deviation. The objective function characterizes the combined error corresponding to the projection position deviation and scale relationship deviation.

[0103] The objective function can be a function used to evaluate whether the current values ​​of the state parameters meet the calibration requirements. Specifically, the objective function can use both projection position deviation and scale relationship deviation as error sources, so that the update process of the state parameters is simultaneously affected by multi-state observation constraints and scale constraints. Among them, projection position deviation can reflect the degree of matching between the theoretical observation positions of multiple observable features and the target observation position information; scale relationship deviation can reflect the degree of consistency between the calculated scale relationship and the known scale relationship between at least two scale reference features.

[0104] In some implementations, the objective function may include a first error term and a second error term. The first error term may be related to the projection position deviation and is used to characterize the error corresponding to the multi-state observation constraint. The second error term may be related to the scale relationship deviation and is used to characterize the error corresponding to the scale constraint. The objective function can be determined based on the weighted result of the first and second error terms. Thus, by adjusting the weights of different error terms, a reasonable relationship can be maintained between the projection matching accuracy and the scale constraint strength during the optimization process.

[0105] In some implementations, based on the objective function, the increment of the state parameter can be determined iteratively, and the current value of the state parameter can be updated according to the increment.

[0106] The state parameter increment can be the amount of parameter change used to correct the current value of the state parameter. Specifically, in each iteration, the changes in the device pose parameters, spatial position parameters, and calibration parameters to be optimized can be determined based on the comprehensive error corresponding to the objective function, and the current values ​​of the state parameters are updated based on these changes. The updated state parameters can then be used to determine new theoretical observation positions, calculate scale relationships, projection position deviations, and scale relationship deviations. Through multiple iterations, the comprehensive error corresponding to the objective function can be gradually made to meet the calibration requirements.

[0107] For example, in a given iteration, if the objective function indicates a significant discrepancy between the theoretical and target observation positions, the device pose parameters, spatial position parameters, or calibration parameters to be optimized can be adjusted incrementally using state parameters to make the subsequently calculated theoretical observation position closer to the target observation position. Similarly, if the objective function indicates a discrepancy between the calculated scale relationship and the known scale relationship, the relevant spatial position parameters can be adjusted incrementally using state parameters to make the calculated scale relationship closer to the known scale relationship. Thus, the state parameter update process can simultaneously consider both observation matching and scale relationships.

[0108] In some implementations, the calibration parameters to be optimized when the convergence conditions are met can be determined as the calibration parameters of the scanning device.

[0109] Convergence criteria can be used to determine whether the iterative update process of the state parameters meets preset calibration requirements. For example, convergence criteria may include at least one of the following: the change in the objective function is less than a preset change threshold, the increment of the state parameters is less than a preset increment threshold, the projection position deviation is less than a preset projection deviation threshold, the scale relationship deviation is less than a preset scale deviation threshold, or the number of iterations reaches a preset number. This application does not specifically limit this aspect.

[0110] When the convergence condition is met, it can be assumed that the calibration parameters to be optimized in the state parameters are sufficient to match the theoretical observation positions with the target observation positions under multiple target observation states, and to match the computational scale relationship with the known scale relationship. Therefore, the calibration parameters to be optimized at this moment can be determined as the calibration parameters of the scanning device. Since the calibration parameters to be optimized include the relative pose parameters between the first imaging unit and the second imaging unit, the final calibration parameters of the scanning device can be used for subsequent spatial positioning, 3D reconstruction, or tracking calculations.

[0111] In this embodiment, the combined error corresponding to the projection position deviation and scale relationship deviation is uniformly represented by the objective function, and the state parameter increment is determined iteratively based on the objective function. This allows the state parameters to be gradually updated under the combined effect of multi-state observation constraints and scale constraints. Therefore, the calibration parameters of the scanning equipment no longer rely solely on a single observation state or a single scale information, but can comprehensively utilize target observation position information under multiple target observation states and known scale relationships, thereby improving the accuracy and stability of the scanning equipment's calibration results.

[0112] In some implementations, the increment of the state parameters is determined iteratively based on the objective function, and the current value of the state parameters is updated according to the increment of the state parameters. This may include: in the current iteration, determining the residual vector and Jacobian matrix corresponding to the objective function based on the current value of the state parameters.

[0113] The current iteration can be any update process within the iterative update process of the state parameters. The current value of the state parameters can be used as input to the current iteration to calculate the error information and trend of the objective function under the current value.

[0114] The residual vector can be used to characterize the error values ​​of each error term in the objective function during the current iteration. Specifically, the residual vector includes the residuals corresponding to the projection position deviation and the scale relationship deviation. The residuals corresponding to the projection position deviation characterize the deviation between the theoretical observation position and the target observation position information; the residuals corresponding to the scale relationship deviation characterize the deviation between the calculated scale relationship and the known scale relationship. Therefore, the residual vector can simultaneously reflect the multi-state observation error and the scale error under the current state parameters.

[0115] The Jacobian matrix can be used to characterize the relationship between the residual vector and the state parameters. In other words, the Jacobian matrix reflects the sensitivity of each residual term in the residual vector to changes when the device pose parameters, spatial position parameters, and calibration parameters to be optimized change. By determining the Jacobian matrix, the direction and magnitude of the subsequent determination of the state parameter increments can be provided.

[0116] In some implementations, the state parameter increments can be determined based on the residual vector, the Jacobian matrix, and the damping factor.

[0117] The damping factor can be used to adjust the magnitude of the state parameter increment or the stability of the iterative update. Specifically, during the iterative update process, if the state parameter increment is too large, it may lead to instability in the overall error corresponding to the objective function; if the state parameter increment is too small, it may lead to a slow convergence speed. By introducing a damping factor, the balance between update stability and convergence speed can be adjusted, making the iterative update process of the state parameters more stable.

[0118] The state parameter increment can be a change in the current value of the state parameter determined by the current iteration process. The state parameter increment can include increments corresponding to the device pose parameters, spatial position parameters, and calibration parameters to be optimized. Through this state parameter increment, the current state parameters can be corrected so that the updated state parameters better meet the comprehensive error requirements of the objective function.

[0119] In some implementations, the current value of the state parameter can be updated based on the state parameter increment to obtain the next value of the state parameter. Specifically, the state parameter increment can be applied to the current value of the state parameter to obtain the state parameter used in the next iteration. Subsequently, the theoretical observation position, projection position deviation, calculation scale relationship, and scale relationship deviation can be re-determined based on the next value of the state parameter, and iterative updates can continue to be performed until the convergence condition is met.

[0120] In this embodiment, by determining the residual vector and Jacobian matrix during the current iteration and combining them with the damping factor to determine the state parameter increment, the state parameter update process can simultaneously consider the magnitude of the comprehensive error and its relationship with the state parameters. This improves the stability and convergence of the state parameter iterative update process, resulting in more accurate calibration parameters for the scanning device.

[0121] In some specific implementations, the determination process of the calibration parameters of the scanning device will be further explained below with reference to a specific calculation process. This embodiment can be considered a specific implementation of the aforementioned optimization of state parameters based on target observation position information and known scale relationships to obtain the calibration parameters of the scanning device. In this embodiment, the scanning device may include a first imaging unit and a second imaging unit. The first imaging unit and the second imaging unit may be two cameras in a binocular observation structure, respectively. The observable features on the calibration auxiliary object may correspond to spatial marker points, and the multiple target observation states may correspond to multiple camera poses formed by the scanning device relative to the calibration auxiliary object.

[0122] Specifically, we can first define the state parameters to be optimized. For ease of representation, all the parameters to be optimized can be integrated into a state vector. x State vector x It can include M camera poses, N spatial point coordinates, and the extrinsic parameters of the stereo camera to be calculated. State vector. x It can be represented as: Where M can represent the number of camera poses involved in the optimization, or it can correspond to the number of the aforementioned multiple target observation states; N can represent the number of spatial markers involved in the optimization, or it can correspond to the number of the aforementioned multiple observable features. It can represent the first i The pose parameters of each pose, including the rotation matrix. Translation vector ,Right now .

[0123] in, Indicates the first i The rotation matrix corresponding to each pose Indicates the first i The translation vector corresponding to each pose. i Each pose can correspond to the first position formed by the scanning device and the calibration auxiliary object. i The observation status of the target. In the embodiments of this application, the first target observation status. i The pose can be understood as the scanning device at the [number]th position. i The device pose parameters relative to the calibration auxiliary object under the target observation state.

[0124] Indicates the first j The global 3D coordinates of a spatial marker point. Spatial marker points can correspond to observable features on auxiliary objects. It can be represented as: ,in, , , They represent the first j The three-dimensional coordinate components of a spatial marker point in the global coordinate system. In this embodiment, the spatial marker point can correspond to observable features on an auxiliary object. It can correspond to the first j Spatial location parameters of an observable feature.

[0125] , , , , and This represents the extrinsic parameters of the stereo camera to be calculated. Specifically, , , It can be used to characterize the rotational relationship between the first imaging unit and the second imaging unit. , , It can be used to characterize the translational relationship between the first imaging unit and the second imaging unit. That is, , , , , and It can correspond to the relative pose parameters in the calibration parameters to be optimized, and can be used as part of the calibration parameters of the scanning device.

[0126] In some implementations, the objective function can be constructed based on reprojection error and scale constraints. This objective function can then be used to evaluate the state vector. x The corresponding overall error. Specifically, it can be minimized by the following total energy function. : .in, (·) denotes a robust kernel function, such as the Huber kernel; λ denotes the scale constraint weight coefficient; Indicates the first j The spatial marker point at the th i Reprojection error in individual camera images; This represents the scale constraint error. The reprojection error term corresponds to the aforementioned multi-state observation constraint, and the scale constraint term corresponds to the aforementioned scale constraint.

[0127] In some implementations, reprojection error is used to measure the first... jThe spatial point at the th i Observations on camera images Compared with theoretical projection value The difference between them. Reprojection error. It can be represented as: ,in, Indicates the first j The spatial point at the th i The observations on the camera images; π(·) represents the camera projection model, and π(·) can include camera intrinsic parameters and distortion correction; Indicates according to the first i Rotation matrix for each pose Translation vector , will the j spatial points Transform to the position in the corresponding camera coordinate system. Therefore, It can be used as the first j The spatial point at the th i Theoretical projection value on a camera image .

[0128] In the embodiments of this application, It can correspond to the actual observation location in the target observation location information. The theoretical observation position can be determined based on the state parameters and the imaging model. For both the first and second imaging units, the theoretical observation position can be determined according to their respective imaging models and compared with the corresponding target observation position information to obtain the projection position deviation.

[0129] It should be noted that when considering the theoretical observation location of the second imaging unit, it is possible to combine... , , , , and The relative pose relationship between the first and second imaging units is represented by the transformation of the spatial point into the observation coordinate system corresponding to the second imaging unit, and then projected through the imaging model of the second imaging unit. Thus, the reprojection error can be used to constrain the calibration parameters to be optimized.

[0130] In some implementations, scale constraints are used to enforce the two endpoints of the scale. , The Euclidean distance between them is equal to their actual physical length. This scale constraint can be expressed as: Correspondingly, scale constraint error It can be represented as: .in, and They can represent the spatial location parameters corresponding to at least two scale reference features, express and The actual physical length between them. In the embodiments of this application, and It can correspond to at least two scale reference features in the auxiliary object. It can be used to compare the known scale relationship between at least two scale reference features.

[0131] Therefore, based on the state vector x The spatial location parameters are calculated to establish the computational scale relationship between at least two scale reference features, and the error is constrained by scale. This ensures that the calculated scale relationship matches the known scale relationship.

[0132] In some implementations, the Levenberg-Marquardt algorithm (LM) can be used to evaluate the objective function. The iterative optimization process involves the following steps.

[0133] First, initialize the state vector. Initial state vector x 0 can include the initial values ​​of each camera pose, the initial values ​​of each spatial marker coordinate, and the initial values ​​of the binocular camera extrinsic parameters. The initial values ​​of the binocular camera extrinsic parameters can be determined based on the structural design parameters of the scanning equipment, the initial installation relationship between the first and second imaging units, or based on a rough estimate of some observation data.

[0134] Second, iterative optimization. In the... k In the next iteration, we can first base it on the current state vector. Calculate the residual vector Residual vector This can include the residuals corresponding to reprojection errors and the residuals corresponding to scale constraint errors. For example, the residual vector. It can include multiple as well as In this embodiment, the residual vector may include the residual corresponding to the projection position deviation and the residual corresponding to the scale relationship deviation.

[0135] Then, the Jacobian matrix can be calculated. Jacobian matrix It can represent the residual vector relative to the state vector. x Partial derivative relation: .in, Indicates the first k The Jacobian matrix corresponding to the next iteration Represents the residual vector.x This represents the state vector.

[0136] Next, the incremental equations can be solved: .in, This represents the damping factor of the LM algorithm. Represents the identity matrix. Indicates the increment of state parameters. Damping factor. It can be used to adjust the increment of state parameters. The size of the value is adjusted to improve the stability of the iterative solution process.

[0137] Then, the state parameter increment can be used. Update the state vector: .in, Indicates the first k The state vector corresponding to +1 iterations.

[0138] Third, perform a convergence check. The algorithm can converge if any of the following conditions are met: ,or Or, until the maximum number of iterations is reached.

[0139] in, , This represents the convergence threshold. This can indicate that the increment of the state parameters is small enough, meaning that the change in the state vector is already small; This can indicate that the gradient of the objective function is small enough, meaning that the current state vector is close to a local optimum or a global optimum; reaching the maximum number of iterations can be used to avoid the iteration process from continuing indefinitely.

[0140] After the convergence condition is met, a high-precision global optimal solution can be output. x The globally optimal solution x This can include camera extrinsic parameters and the coordinates of spatial marker points. Specifically, x In , , , , and It can be used as a relative pose parameter between the first imaging unit and the second imaging unit; x In , ..., It can serve as a spatial location parameter corresponding to multiple observable features. Therefore, calibration parameters for the scanning device can be obtained and used for subsequent spatial positioning, 3D reconstruction, or tracking calculations. Since these calibration parameters are obtained under the combined influence of multi-state observation constraints and scale constraints, the dependence of the calibration results on a single observation state or the overall high-precision structure of the calibration auxiliary object can be reduced, thus improving the accuracy of the scanning device's calibration results.

[0141] This application also provides a calibration system for a scanning device, including a control module, which is used to execute the calibration method for the scanning device provided in the above embodiments.

[0142] See Figure 5 , Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application.

[0143] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the calibration method of the scanning device provided in the above embodiments.

[0144] The electronic device may include: a memory 110, a processor 120, and a communication interface 130. The memory 110, the processor 120, and the communication interface 130 are connected through internal connection paths.

[0145] The memory 110 is used to store computer programs, which in some implementations may include code for implementing the methods of the embodiments of this application.

[0146] The processor 120 executes the computer program stored in the memory 110 to control the communication interface 130 to receive input data and information, and output operation results and other data. In some implementations, when the solutions of the embodiments of this application are implemented by software or firmware, the computer program used to implement the solutions of the embodiments of this application can be stored in the processor 120 and executed by the processor 120.

[0147] The memory 110 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM). It should be noted that the memory 110 described herein is intended to include, but is not limited to, any memory of these and other suitable types. As an example, the memory 110 includes random access memory (RAM), cache memory, and read-only memory (ROM). The memory 110 stores a computer program that can be executed by processor 120, causing processor 120 to implement the steps of any of the methods described above.

[0148] The processor 120 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor 120 can be any conventional processor.

[0149] In implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware in the processor 120 or by instructions in software form. The method disclosed in the embodiments of this application can be directly implemented by the hardware processor, or by a combination of hardware and software modules in the processor 120. The software modules can be located in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in the memory 110, and the processor 120 reads the information in the memory 110 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are not provided here.

[0150] In some implementations, in addition to the hardware units described above, electronic devices may also include software modules, such as operating systems, basic input / output systems (BIOS), and application software.

[0151] An operating system is used to manage one or more hardware and software resources of an electronic device; it is the kernel and foundation of the electronic device. The operating system handles fundamental tasks such as managing and configuring memory, determining the priority of system resource allocation and demand, controlling input and output devices, operating networks, and managing file systems. To facilitate user operation, most operating systems provide a user interface for interaction with the system.

[0152] The BIOS is used to perform hardware initialization during the power-on boot phase and to provide runtime services for the operating system and applications. In some implementations, the BIOS can also monitor and display processor temperature and execute temperature protection strategies.

[0153] Application software, also known as an application program, can be understood as software written for a specific user application purpose, and is one of the main categories of computer software. For example, application software can be a program used to achieve purposes such as power control and temperature management.

[0154] It is understood that the specific examples in this application are only intended to help those skilled in the art better understand the implementation of this application, and are not intended to limit the scope of protection of this application.

[0155] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0156] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and this application does not limit them.

[0157] Unless otherwise stated, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "one or more" as used in this application includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0158] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0159] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes and beneficial effects of the embodiments described above can be referred to the corresponding processes and beneficial effects in other embodiments, and will not be repeated here.

[0160] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0161] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the technical solution in this application, depending on actual needs.

[0162] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0163] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0164] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A calibration method for a scanning device, characterized in that, Applied to a scanning device, the scanning device comprising at least two imaging units, the at least two imaging units comprising a first imaging unit and a second imaging unit, the method comprising: The system receives multi-state observation data obtained by the first imaging unit and the second imaging unit for a calibration auxiliary object. The multi-state observation data is obtained when the scanning device and the calibration auxiliary object form multiple relative observation states. The calibration auxiliary object includes multiple observable features, and the multiple observable features include at least two scale reference features. The at least two scale reference features have a known scale relationship. Based on the multi-state observation data, the observation location information of multiple observable features in the first imaging unit and the second imaging unit is determined respectively. Based on the observation location information and the known scale relationship, the calibration parameters of the scanning device are determined according to the multi-state observation constraints and scale constraints. The calibration parameters of the scanning device include the relative pose parameters between the first imaging unit and the second imaging unit. The multi-state observation constraint is used to constrain the projection relationship of multiple observable features corresponding to the first imaging unit and the second imaging unit in the multiple relative observation states. The scale constraint is used to constrain the correspondence between the spatial scale of the at least two scale reference features and the known scale relationship.

2. The calibration method for the scanning device according to claim 1, characterized in that, The step of determining the observation location information of multiple observable features in the first imaging unit and the second imaging unit based on the multi-state observation data includes: Feature recognition is performed on the multi-state observation data to determine that multiple observable features correspond to the first observation position information of the first imaging unit and the second observation position information of the second imaging unit, respectively. Based on the feature identification information and / or image feature information of multiple observable features, determine the correspondence between the first observation location information and the second observation location information; Based on the correspondence, the image position information of the same observable feature in the first imaging unit and the second imaging unit is associated to obtain the observation position information.

3. The calibration method for the scanning device according to claim 1, characterized in that, The process of determining the calibration parameters of the scanning device based on the observed location information and the known scale relationship, according to multi-state observation constraints and scale constraints, includes: Based on the observation location information, multiple target observation states are determined from the multiple relative observation states; From the observation location information, determine the target observation location information associated with the observation status of the plurality of targets; Based on the target observation location information and the known scale relationship, the calibration parameters of the scanning device are determined according to the multi-state observation constraints and the scale constraints.

4. The calibration method for the scanning device according to claim 3, characterized in that, The step of determining multiple target observation states from the multiple relative observation states based on the observation location information includes: The evaluation result of each relative observation state is determined based on the characteristic observation quality corresponding to each relative observation state and / or the geometric distribution of the multiple relative observation states. Based on the evaluation results, the multiple target observation states are determined from the multiple relative observation states; The quality of feature observation includes at least one of the number of observable features, the distribution range of observable features, and image sharpness; the geometric distribution includes at least one of the change in observation pose and the change in observation distance.

5. The calibration method for the scanning device according to claim 3, characterized in that, The step of determining the calibration parameters of the scanning device based on the target observation location information and the known scale relationship, according to the multi-state observation constraints and the scale constraints, includes: Based on the target observation location information and the known scale relationship, the state parameters are optimized to obtain the calibration parameters of the scanning device; The state parameters include device pose parameters corresponding to multiple target observation states, spatial position parameters of multiple observable features, and calibration parameters to be optimized. The device pose parameters are used to characterize the pose of the scanning device relative to the calibration auxiliary object in the corresponding target observation state, and the calibration parameters to be optimized include the relative pose parameters between the first imaging unit and the second imaging unit.

6. The calibration method for the scanning device according to claim 5, characterized in that, The optimization of the state parameters based on the target observation location information and the known scale relationship to obtain the calibration parameters of the scanning device includes: Based on the current values ​​of the state parameters and the imaging model, the theoretical observation positions of the multiple observable features corresponding to the first imaging unit and the second imaging unit are determined respectively. Based on the theoretical observation position and the target observation position information, determine the projection position deviation; Based on at least a portion of the spatial location parameters from the current values ​​of the state parameters, determine the computational scale relationship corresponding to the known scale relationship; Based on the calculated scale relationship and the known scale relationship, determine the scale relationship deviation; The current value of the state parameter is updated based on the projection position deviation and the scale relationship deviation.

7. The calibration method for the scanning device according to claim 6, characterized in that, The step of updating the current value of the state parameter based on the projection position deviation and the scale relationship deviation includes: Based on the projection position deviation and the scale relationship deviation, an objective function is determined; wherein, the objective function is used to characterize the comprehensive error corresponding to the projection position deviation and the scale relationship deviation; Based on the objective function, the increment of the state parameter is determined iteratively, and the current value of the state parameter is updated according to the increment of the state parameter. If the convergence condition is met, the calibration parameters to be optimized when the convergence condition is met are determined as the calibration parameters of the scanning device.

8. The calibration method for the scanning device according to claim 7, characterized in that, The step of iteratively determining the state parameter increment based on the objective function, and updating the current value of the state parameter according to the state parameter increment, includes: In the current iteration, the residual vector and Jacobian matrix corresponding to the objective function are determined based on the current value of the state parameter; wherein, the residual vector includes the residual corresponding to the projection position deviation and the residual corresponding to the scale relationship deviation; The state parameter increment is determined based on the residual vector, the Jacobian matrix, and the damping factor. The current value of the state parameter is updated based on the increment of the state parameter to obtain the next value of the state parameter.

9. A calibration system for a scanning device, characterized in that, It includes a control module, which is used to perform the calibration method of the scanning device according to any one of claims 1 to 8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the calibration method of the scanning device as described in any one of claims 1 to 8.

Images