Scanner calibration method and apparatus, electronic device, and computer-readable storage medium

Abstract

The present application discloses a scanner calibration method and apparatus, an electronic device, and a computer-readable storage medium. The scanner calibration method comprises: displaying, in real time, a calibration image currently acquired by a scanner for a calibrator; displaying a virtual reference pattern in the calibration image in a superimposed manner, the virtual reference pattern being used for indicating in the calibration image an initial calibration attitude to which the scanner needs to adjust, relative to the calibrator; and in response to detecting that the scanner is adjusted to the initial calibration attitude relative to the calibrator indicated by the virtual reference pattern, displaying a virtual calibration indication in the calibration image, the virtual calibration indication being used for instructing a user to calibrate on the basis of the initial calibration attitude. The efficiency of a calibration process can be improved.

Description

Scanner calibration method and device, electronic equipment and computer readable storage medium Cross-reference to related applications This application claims priority to the Chinese patent application entitled "Scanner calibration method and device, electronic equipment and computer readable storage medium" filed on September 25, 2024 with the China Patent Office, application number 202411346558.5, the entire contents of which are incorporated herein by reference. The Chinese patent application entitled "Scanner calibration method and device, electronic equipment and computer readable storage medium" is hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0001] The present application relates to the field of scanner calibration, and in particular to a scanner calibration method, device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] The calibrator can be applied in the fields of machine vision, image measurement, photogrammetry, three-dimensional reconstruction, etc., for example, to correct lens distortion, determine the conversion relationship between physical size and pixels, and determine the mutual relationship between the three-dimensional geometric position of a point on the surface of a space object and the corresponding point in the image, thereby providing a reference for establishing a geometric model of camera imaging. In some examples, the geometric model of the camera can be obtained through the camera shooting of a flat plate or other shaped structure with a fixed interval pattern array, and the calculation of the calibration algorithm, thereby obtaining high-precision measurement and reconstruction results; and the flat plate or other shaped structure with a fixed interval pattern array belongs to the calibrator.

[0003] The calibrator can be used in various scenarios. Taking the scanning of a scene as an example, in order to ensure the scanning accuracy of the scanner, the user needs to use the calibrator to perform a calibration operation on the scanner. In the calibration operation, the traditional calibration method is not user-friendly. For example, in the user interface, the main information representing the calibration direction is often outside the camera shooting screen of the scanner. In the calibration process for the current calibration direction in the screen of the display interface, the user needs to pay attention to the calibration direction information outside the camera shooting screen, so that the scanner can be calibrated while maintaining the current calibration direction. This may lead to: on the one hand, the user has difficulty in operating the scanner to align with the current calibration direction, and on the other hand, when calibrating based on the current calibration direction, the user needs to pay attention to the calibration direction information outside the screen, which may cause the projection center of the scanner to deviate from the center of the calibrator, resulting in low efficiency of the calibration process. SUMMARY

[0004] The present application provides a scanner calibration method, device, electronic equipment and computer readable storage medium, and the technical solutions are as follows:

[0005] According to a first aspect of an embodiment of the present application, a scanner calibration method is provided, which comprises: displaying a calibration image currently captured by a scanner on a calibration device in real time; superimposing a virtual reference figure on the calibration image, the virtual reference figure being used to indicate an initial calibration pose to which the scanner needs to be adjusted relative to the calibration device in the calibration image; and in response to detecting that the scanner is adjusted to the initial calibration pose indicated by the virtual reference figure relative to the calibration device, displaying a virtual calibration indication in the calibration image, the virtual calibration indication being used to indicate that a user calibrates based on the initial calibration pose.

[0006] The technical solution provided by the first aspect of the embodiment of the present application directly displays the calibration direction of the scanner and the corresponding position in the form of a virtual reference figure, for example, an AR figure, to guide the user to efficiently complete the entire calibration process. Specifically, by superimposing the virtual reference figure indicating the initial calibration pose to which the scanner needs to be adjusted relative to the calibration device in the calibration image and the virtual calibration indication indicating that the user calibrates based on the initial calibration pose in the calibration image, the information representing the calibration direction can be concentrated in the calibration image, the indication of the required calibration pose is intuitive and easy to understand, the user can focus on the information in the calibration image as much as possible to calibrate, and does not need to be distracted by the information in the area outside the calibration image, thereby improving the efficiency of the calibration process.

[0007] According to a second aspect of an embodiment of the present application, a scanner calibration device is provided, which comprises: a display unit configured to display a calibration image currently captured by a scanner on a calibration device in real time; the display unit is further configured to superimpose a virtual reference figure on the calibration image, the virtual reference figure being used to indicate an initial calibration pose to which the scanner needs to be adjusted relative to the calibration device in the calibration image; and a calibration unit configured to, in response to detecting that the scanner is adjusted to the initial calibration pose indicated by the virtual reference figure relative to the calibration device, display a virtual calibration indication in the calibration image, the virtual calibration indication being used to indicate that a user calibrates based on the initial calibration pose.

[0008] According to a third aspect of an embodiment of the present application, an electronic device is provided, which comprises: a processor; a memory for storing processor-executable instructions; and wherein the processor is configured to implement the method according to the first aspect.

[0009] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps in the method according to the first aspect.

[0010] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory and are not restrictive of the application. BRIEF DESCRIPTION OF DRAWINGS

[0011] FIG. 1 is a schematic diagram of a calibrator application scenario in the related art;

[0012] FIG. 2 is a schematic diagram of a scanner calibration scenario in the related art;

[0013] FIG. 3 is a schematic diagram of a calibration direction in the related art;

[0014] FIG. 4 is a schematic diagram of a flow of a scanner calibration method according to an embodiment of the present application;

[0015] FIG. 5 is a schematic diagram of a virtual reference pattern included in a calibration image according to an embodiment of the present application;

[0016] FIG. 6 is a schematic diagram of a virtual reference pattern included in a calibration image according to another embodiment of the present application;

[0017] FIG. 7 is a schematic diagram of a virtual reference pattern included in a calibration image according to another embodiment of the present application;

[0018] FIG. 8 is a schematic diagram of a calibration indication included in a calibration image according to an embodiment of the present application;

[0019] FIG. 9 is a schematic diagram of a calibration indication included in a calibration image according to another embodiment of the present application;

[0020] FIG. 10 is a schematic diagram of a calibration indication included in a calibration image according to another embodiment of the present application;

[0021] FIG. 11 is a schematic diagram of a calibration indication included in a calibration image according to another embodiment of the present application;

[0022] FIG. 12 is a schematic diagram of a calibrator deviating from a virtual reference pattern according to an embodiment of the present application;

[0023] FIG. 13 is a schematic diagram of a structure of a scanner calibration apparatus according to an embodiment of the present application;

[0024] FIG. 14 is a schematic diagram of a structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order for those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art should belong to the scope of protection of the present application.

[0026] The calibrator can be applied in the fields of machine vision, image measurement, photogrammetry, three-dimensional reconstruction, etc., for example, to correct lens distortion, determine the conversion relationship between physical size and pixels, and determine the mutual relationship between the three-dimensional geometric position of a point on the surface of a space object and the corresponding point in the image, thereby providing a reference for establishing a geometric model of camera imaging. In some examples, the geometric model of the camera can be obtained through the calculation of the camera shooting of a flat plate with a fixed interval pattern array and the calibration algorithm, thereby obtaining high-precision measurement and reconstruction results; taking the calibrator shown in FIG. 1 as an example, the calibrator can be a flat plate with a fixed interval pattern array.

[0027] The calibrator can be used in various scenarios. For example, in order to ensure the scanning accuracy of a scanner, a user needs to use the calibrator to perform a calibration operation on the scanner. In the calibration operation, the traditional calibration method is not user-friendly. For example, in the user interface, the main information representing the calibration direction is often outside the camera shooting screen of the scanner. In the calibration process for the current calibration direction in the camera screen, the user needs to pay attention to the calibration direction information outside the camera shooting screen, so that the scanner can be calibrated under the premise of maintaining the current calibration direction. This may lead to: on the one hand, the user is more difficult to operate the scanner to align with the current calibration direction, and on the other hand, when the calibration operation is performed based on the current calibration direction, the projection center of the scanner is easily deviated from the center of the calibrator due to the need to pay attention to the calibration direction information outside the screen, resulting in low efficiency of the calibration process.

[0028] Please refer to FIGS. 2-3, and the following exemplary introduction to the scanner calibration scenario in the related art is given:

[0029] As shown in FIG. 2, a calibration operation method in the related art includes:

[0030] (1) The user adjusts the inclination angle of the scanner in the horizontal direction relative to the calibrator based on the given inclination angle (for example, the position shown by the dark gray pattern) by observing the "left tilt" / "right tilt" area located above the display area, and overlaps the dark gray pattern in the "left tilt" / "right tilt" area with the black frame pattern, that is, it is considered that the scanner has reached a suitable initial calibration position in the horizontal direction angle relative to the calibrator.

[0031] (2) The user adjusts the inclination angle of the scanner in the vertical direction relative to the calibrator based on the given inclination angle (for example, the position shown by the dark gray pattern) by observing the "forward tilt" / "backward tilt" area located on the left side of the display area, and overlaps the dark gray pattern in the "forward tilt" / "backward tilt" area with the black frame pattern, that is, it is considered that the scanner has reached a suitable initial calibration position in the vertical direction angle relative to the calibrator.

[0032] (3) The user adjusts the position to which the projection center of the scanner points (such as the central circular area in FIG. 2) by observing the position of the central area (i.e. the gray circular area in FIG. 2) of the marker in the image collected by the camera of the scanner, so that the projection center of the scanner points to the center of the marker, i.e. the central circular area in FIG. 2 coincides with the gray circular area.

[0033] As an example, the actual direction of "tilt left" / "tilt right", "tilt forward" / "tilt backward" of the scanner relative to the marker can be referred to the schematic shown in FIG. 3, in which the scanner and the marker are kept horizontal, and the marker is in the correct direction.

[0034] After the positions of the scanner indicated by steps (1)-(3) are adjusted, it can be considered that the scanner is in the initial pose corresponding to the current calibration direction relative to the marker, at this time, it is required to keep the angle between the line connecting the projection center of the scanner and the center of the marker and the straight line perpendicular to the center of the marker unchanged, and it is also required to keep the projection center of the scanner aligned with the center of the marker, then it is required to move the scanner on the line connecting the projection center of the scanner and the center of the marker, and finally all the height indication bars on the right side outside the camera are lighted, and the height indicated by the height indication bars refers to the height of the scanner relative to the vertical direction of the marker.

[0035] The calibration operation mode of steps (1)-(3) in the related art is not user-friendly, during calibration, it is required to hold and keep a certain posture and move in one direction, which is time-consuming and laborious, and is easy to feel tired.

[0036] Moreover, the operation is complicated, it is required to pay attention to the information of multiple different areas at the same time, which increases the calibration time, requires the user to be highly concentrated, and it is difficult for the user to accurately find the correct position of the scanner according to the information on the interface, and it is easy to deviate from the position during the calibration process, and it is difficult to smoothly complete the entire calibration process.

[0037] In view of the above problems, an embodiment of the present application provides a scanner calibration method, which can improve the efficiency of the calibration process. As shown in FIG. 4, the method comprises the following steps:

[0038] S401, display a calibration image currently collected by a scanner on a marker in real time.

[0039] S402, superimpose a virtual reference figure on the calibration image.

[0040] The virtual reference figure is used to indicate an initial calibration pose to which the scanner needs to be adjusted relative to the marker in the calibration image.

[0041] S403, in response to detecting that the scanner is adjusted to the initial calibration pose indicated by the virtual reference pattern relative to the calibration device, displaying a virtual calibration instruction in the calibration image.

[0042] The virtual calibration instruction is used to instruct the user to calibrate based on the initial calibration pose.

[0043] The technical scheme provided by the embodiments of the present application directly displays the calibration direction and the corresponding position of the scanner in the form of a virtual reference pattern, such as an AR pattern, to guide the user to efficiently complete the entire calibration process. Specifically, by superimposing the virtual reference pattern indicating the initial calibration pose to which the scanner needs to be adjusted relative to the calibration device in the calibration image, and the virtual calibration instruction used to instruct the user to calibrate based on the initial calibration pose, the information representing the calibration direction is concentrated in the calibration image, the indication of the required calibration pose is intuitive and easy to understand, and the user can focus on the information in the calibration image as much as possible to calibrate, without needing to be distracted by the information outside the calibration image, thereby improving the efficiency of the calibration process.

[0044] The calibration image in S401 can be acquired in various ways. As an example, the calibration image can be a calibration image acquired in real time by a camera arranged on the scanner. As another example, the position of the projection center of the scanner can be located at the central position of the calibration image. As another example, the center position of the virtual reference pattern displayed in S402 can be located at the central position of the calibration image, so that the center position of the virtual reference pattern represents the position of the projection center of the projector.

[0045] It can be understood that since the real-time acquired calibration image contains the image of the actual calibration device, the pose and size of the calibration device in the calibration image will change with the change of the pose and distance of the scanner relative to the actual calibration device. As an example, if the center position of the virtual reference pattern displayed in S402 needs to represent the position of the projection center of the projector, the center position of the virtual reference pattern can remain at the central position of the calibration image, and at the same time, the size and pose of the virtual reference pattern can not change when the initial calibration pose of the calibration device in the calibration image does not change.

[0046] There are various ways to detect whether the scanner is adjusted to the initial calibration pose indicated by the virtual reference figure relative to the calibrator, as an example, the virtual reference figure can be determined by determining a current calibration direction in a preset calibration direction, and based on the current calibration direction and the initial calibration distance, the preset calibration direction can include at least one calibration direction required by the scanner to calibrate based on the calibrator; the virtual calibration indication displayed in S403 can be used to instruct the user to calibrate based on the current calibration direction; when it is detected that the calibrator and the virtual reference figure are overlapped in size and pose in the calibration image, it can be determined that the calibrator is adjusted to the initial calibration pose indicated by the virtual reference figure; as another example, the initial calibration pose indicated by the virtual reference figure can include the current calibration direction and the initial calibration distance between the scanner and the actual calibrator (the distance size can be displayed by the size of the virtual reference figure).

[0047] The preset calibration direction can include at least one calibration direction required by the scanner to calibrate based on the calibrator, and the at least one calibration direction required by the scanner to calibrate based on the calibrator is determined according to which poses the scanner needs to calibrate relative to the calibrator in the actual calibration process, such as vertical upward, forward tilt, backward tilt, left tilt, right tilt, etc. Different calibration directions can uniquely identify a calibration pose. It can be understood that if the number of preset calibration directions is multiple, the scanner needs to calibrate in each of the calibration directions to complete the complete calibration process.

[0048] As an example, the initial calibration distance between the scanner and the actual calibrator described above can be the farthest calibration distance required between the scanner and the actual calibrator; as another example, the initial calibration distance can also be the nearest calibration distance, or any calibration distance between the farthest calibration distance and the nearest calibration distance; therefore, the specific implementation of the initial calibration distance is not limited.

[0049] The virtual reference figure displayed in S402 can be generated in various ways. As an example, the virtual reference figure can be a virtual reticle figure, which can be determined based on the current calibration direction determined in the preset calibration direction and the initial calibration distance, to determine a virtual pose of the scanner relative to the virtual reticle, and then generate the virtual reticle figure based on the determined virtual pose and the camera parameters of the scanner. As another example, the intrinsic matrix of the camera on the scanner that captures the calibration image can be obtained first, and after the virtual reticle is determined relative to the virtual pose of the scanner, the coordinate transformation matrix of the virtual reticle to the camera is obtained, and the three-dimensional coordinates of the key points on the virtual reticle in the camera coordinate system are calculated according to the coordinate transformation matrix and the virtual pose, and then the two-dimensional coordinates of the key points on the reticle in the camera pixel plane are calculated according to the three-dimensional coordinates and the intrinsic matrix of the camera, and the virtual reticle figure is generated and displayed based on the calculated two-dimensional coordinates. It should be noted that the above description of the generation method of the virtual reticle figure is only an example, and other generation methods can exist in actual applications, which are not limited in this regard.

[0050] As an example, the shape of the virtual reticle figure has a mapping relationship with the current calibration direction and the initial calibration distance, wherein the current calibration direction includes an angle component of the scanner relative to the actual reticle, and the initial calibration distance can be associated with the size of the virtual reticle figure. As another example, the angle component of the scanner relative to the actual reticle can be the tilt angle component of the scanner relative to the actual reticle, such as the left / right tilt angle and / or the front / back tilt angle. It can be understood that the current calibration direction can determine the inclination degree of the virtual reticle figure in the calibration image, and the initial calibration distance can determine the size of the virtual reticle figure in the calibration image. The shape of the virtual reticle figure can intuitively reflect the initial calibration distance and the angle component of the scanner relative to the actual reticle, and convert the angle component that the user needs to align during calibration into a figure pose and size in the calibration image, which is convenient for the user to intuitively calibrate and align.

[0051] Please refer to FIG. 5 and FIG. 6, the following exemplary describes the virtual reticle figure and the reticle in the calibration image of the embodiment of the present application:

[0052] As shown in FIG. 5, the black and white camera of the scanner captures the calibration image in real time, which includes the actual calibration target (the gray part with white dots) and the rectangular pattern superimposed in the center of the calibration image. The rectangular pattern is the virtual calibration target pattern. For example, the initial calibration distance is the farthest calibration distance between the scanner and the calibration target. The rectangular pattern is the size and pose of the calibration target that should be seen in the calibration image at the farthest calibration distance. The user needs to adjust the relative position between the scanner and the actual calibration target to align the actual calibration target (the gray part with white dots) in the calibration image with the rectangular pattern (the virtual calibration target pattern), including the size and pose.

[0053] As shown in FIG. 6, if the current calibration direction requires the scanner to have an inclination angle (e.g., left / right inclination, front / back inclination) relative to the actual calibration target, the rectangular pattern (the virtual calibration target pattern) can be displayed with perspective effect and form an inclined pose (e.g., the pose inclined to one side in FIG. 6) to guide the user to move and / or rotate the scanner so that the scanner also forms a corresponding inclined pose and the required distance position relative to the actual calibration target, thereby adjusting the calibration target part in the calibration image to the pose corresponding to the rectangular pattern.

[0054] As shown in FIG. 7, the virtual reference pattern can also be other patterns. As an example, the virtual reference pattern can also be a pattern composed of a circular ring and a cross at the center of the circular ring, as shown in FIG. 7. The first pattern formed by the circular ring at the center of the calibration image and the cross at the center is the virtual reference pattern (e.g., the pattern on the left in FIG. 7). The calibration image can also include a second pattern (i.e., a pattern composed of a circular ring at the center of the calibration target and a cross at the center) drawn in real time based on the pose of the actual calibration target relative to the scanner. As another example, another way to detect that the calibration target is adjusted to the initial calibration pose indicated by the virtual reference pattern can include detecting whether the second pattern corresponding to the calibration target in the calibration image is aligned with the first pattern at the center of the calibration image (e.g., whether the center crosses of the two are aligned). If they are aligned, it is determined that the calibration target is adjusted to the initial calibration pose.

[0055] The pose of the actual calibrator can be determined in various ways. As an example, three-dimensional coordinates of the marker points on the calibrator can be set first, and then two-dimensional coordinates of the marker points of the calibrator in the calibration image are identified, and then the pose of the scanner relative to the calibrator is determined based on the set three-dimensional coordinates and the identified two-dimensional coordinates. As another example, after the two-dimensional coordinates of the marker points of the marker are identified, the identified two-dimensional coordinates can be aligned with the set three-dimensional coordinates, and the pose of the actual calibrator relative to the scanner is calculated according to the Perspective-n-Point (PnP) algorithm. The basic idea of the PnP algorithm is to use the coordinates of some known points (usually the corner points or feature points of the object, such as the marker points of the calibrator) on the object (calibrator) in three-dimensional space and the projection coordinates of these points on the two-dimensional image to calculate the rotation matrix (R) and displacement vector (t) of the camera, i.e., the extrinsic parameters of the camera, so that the object model in three-dimensional space can be mapped to the two-dimensional image, thereby determining the pose of the camera relative to the three-dimensional object (marker).

[0056] There are various ways to set three-dimensional coordinates for the marker points on the calibrator. As an example, the calibrator plane can be taken as the Z=0 plane, and three-dimensional coordinates can be generated according to the two-dimensional coordinates of the marker points on the calibrator on the X-axis or Y-axis plane (the two-dimensional coordinates are obtained by measurement), which are set as the set coordinates. As another example, the three-dimensional coordinates can be coordinates relative to the center of the actual calibrator, and the center of the calibrator is generally the geometric center of the calibrator. The Z-axis can be taken as the positive direction upward perpendicular to the calibrator, and an orthogonal right-handed coordinate system is established.

[0057] The virtual calibration indication displayed in S403 can have various specific implementations. As an example, the virtual calibration indication can include a current distance indication and a calibration distance range indication. The current distance indication is used to represent the real-time distance of the calibrator relative to the scanner, and the calibration distance range indication is used to represent the calibration distance range determined based on the nearest calibration distance and the farthest calibration distance between the scanner and the calibrator.

[0058] The current distance indication and the calibrated distance range indication can be displayed in the calibration image in various manners. As an example, the current distance indication can be represented by a first virtual annular pattern, and the calibrated distance range indication can be represented by a second virtual annular pattern. The second virtual annular pattern can be generated by: generating the second virtual annular pattern with an inner ring and an outer ring representing a nearest calibration distance and a farthest calibration distance between the scanner and the calibrator, respectively; and representing the nearest calibration distance by the inner ring of the generated second virtual annular pattern and representing the farthest calibration distance by the outer ring of the generated second virtual annular pattern. The first virtual annular pattern can be generated by: obtaining a real-time distance of the calibrator relative to the scanner; and generating the first virtual annular pattern based on the real-time distance of the calibrator relative to the scanner according to a proportional relationship between a size of the second virtual annular pattern and an actual calibration distance represented by the size.

[0059] The first virtual annular pattern and the second virtual annular pattern can have various specific implementations. As an example, the first virtual annular pattern and the second virtual annular pattern can both be circular annular patterns, or both be other types of patterns, such as rectangular annular patterns, triangular annular patterns, and the like. Therefore, the specific implementations of the first virtual annular pattern and the second virtual annular pattern are not limited.

[0060] The first virtual annular pattern can be generated in various manners. As an example, when the first virtual annular pattern and the second virtual annular pattern are both circular annular patterns, the outer ring radius R and the inner ring radius r of the second virtual annular pattern displayed in the calibration image can be determined first, and then the farthest calibration distance H corresponding to the outer ring and the nearest calibration distance h corresponding to the inner ring can be determined. The proportional relationship between the size of the second virtual annular pattern and the actual calibration distance represented by the size can be calculated by the following formula (1):

[0061] In the formula (1), K is a proportional coefficient between the size of the second virtual annular pattern and the actual calibration distance represented by the size.

[0062] The real-time distance of the calibrator relative to the scanner is denoted as height. The radius of the first virtual annular pattern can be calculated by the following formula (2):

[0063] Based on the calculated radius radius, the first virtual annular pattern can be generated.

[0064] The virtual calibration indication can be used to indicate the user to calibrate at different calibration distances based on the current calibration direction. As an example, the first virtual annular pattern can be coincided with the center of the second virtual annular pattern. The second virtual annular pattern can be divided into a plurality of calibration distance gears corresponding to the number of calibration distances required for the scanner to calibrate in the current calibration direction. In response to detecting that the first virtual annular pattern is in a current calibration distance gear of the plurality of calibration distance gears, the calibration image currently collected by the scanner on the calibrator is saved, and after the saving is successful, the part of the second virtual annular pattern corresponding to the current calibration distance gear is eliminated. It can be understood that the size of the first virtual annular pattern can change with the real-time distance of the calibrator relative to the scanner. When the real-time distance becomes larger, the first virtual annular pattern becomes larger. When the real-time distance becomes smaller, the first virtual annular pattern becomes smaller. During the change of the size of the first virtual annular pattern, the first virtual annular pattern will stay in the pattern part corresponding to each calibration distance gear of the plurality of calibration distance gears of the second virtual annular pattern. Whenever the first virtual annular pattern stays in the pattern part corresponding to the current calibration distance gear, the pattern part corresponding to the current calibration distance gear will be eliminated after the calibration image currently collected by the scanner on the calibrator is saved successfully at the calibration distance corresponding to the current calibration distance gear. By changing the size of the first virtual annular pattern, the pattern parts corresponding to each calibration distance gear of the second virtual annular pattern are all eliminated, that is, the calibration image collection process in the current calibration direction is completed. By converting the three-dimensional distance relationship between the calibrator and the scanner to the size of the two-dimensional pattern, all information (information represented by the virtual reference pattern and information displayed by the virtual calibration indication) is concentrated in the calibration image. The user does not need to pay attention to the area outside the calibration image during the calibration process, which further improves the calibration efficiency.

[0065] As an example, the virtual calibration indication can be displayed at the central position of the calibration image, or can be displayed at other positions of the calibration image, which is not limited specifically.

[0066] Please refer to FIGS. 8-11, and the following exemplary description of a virtual calibration indication of an embodiment of the present application:

[0067] As shown in FIG. 8, taking the virtual reference figure of the virtual marker figure as an example, when the virtual marker figure in the calibration image coincides with the size and pose of the marker, it can be determined that the marker is adjusted to the initial calibration pose indicated by the virtual marker figure, at which time the virtual calibration indication is displayed in the calibration image: the first virtual ring figure (such as the white ring in FIG. 8) and the second virtual ring figure (such as the gray ring in FIG. 8). The white ring represents the relative real-time distance between the scanner and the actual marker, as shown in FIG. 9, when the real-time distance becomes larger, the white ring also becomes larger accordingly, as shown in FIG. 10, when the real-time distance becomes smaller, the white ring also becomes smaller accordingly, and any part of the gray ring swept by the white ring will be eliminated when the calibration image corresponding to the part is saved, which means that the part of the figure has successfully collected and saved the calibration image. The gray ring represents the calibration distance range required between the scanner and the actual calibration board in the current calibration direction, and the scanner is kept moving within the calibration distance range in the current calibration direction, and the gray ring is completely eliminated using the white ring, which can complete the calibration image collection work in the current calibration direction.

[0068] As shown in FIG. 11, taking the virtual reference figure of the figure composed of the ring and the center cross of the ring shown in FIG. 7 as an example, after the white figure corresponding to the marker in the calibration image is aligned with the red figure in the central position of the calibration image, it is determined that the marker is adjusted to the initial calibration pose, and the virtual calibration indication is also displayed: the white ring and the gray ring.

[0069] As an example, after the virtual calibration indication is displayed, the virtual reference figure in the calibration image can be eliminated or continue to be saved and displayed, which is not limited.

[0070] Considering that after the virtual calibration indication is displayed, the scanner may deviate from the initial calibration pose during the calibration of the user based on the virtual calibration indication, which may cause a large calibration error in the initial calibration pose.

[0071] To solve the problem, as an example, in response to detecting that the pose of the calibration device deviates from the virtual reference figure after displaying the virtual calibration indication, the virtual calibration indication is eliminated, and an alignment indication is displayed in the calibration image, the alignment indication being used to instruct the user to align the calibration device with the virtual reference figure. As another example, the alignment indication can be a textual prompt or a graphical prompt; as another example, after detecting that the calibration device is re-aligned with the virtual reference figure in the calibration image, S403 described above can be performed again. Please refer to FIG. 12, as another example, if the virtual reference figure is in an eliminated state during calibration by the user based on the virtual calibration indication, the virtual calibration indication can be eliminated and the virtual reference figure can be re-displayed when it is detected that the pose of the calibration device deviates from the initial calibration pose corresponding to the virtual reference figure, so as to guide the user to re-align.

[0072] As an example, whether the position of the calibration device in the calibration image deviates from the initial calibration pose can be determined by comparing the distance between the two-dimensional coordinates of the center of the actual calibration device on the calibration image and the center point coordinates of the calibration image with a distance threshold value. When the distance is less than or equal to the distance threshold value, it can be considered that the position of the calibration device in the calibration image does not deviate from the initial calibration pose. When the distance is greater than the distance threshold value, it can be considered that the position of the calibration device in the calibration image deviates from the initial calibration pose. As another example, whether the angle of the calibration device in the calibration image deviates from the initial calibration pose can be determined by comparing the included angle between the actual rotation amount (for example, the rotation amount represented by a quaternion) of the calibration device relative to the scanner and the virtual rotation amount of the calibration device relative to the scanner corresponding to the initial calibration pose with an included angle threshold value. When the included angle is less than or equal to the included angle threshold value, it can be considered that the angle of the calibration device in the calibration image does not deviate from the initial calibration pose. When the included angle is greater than the included angle threshold value, it can be considered that the angle of the calibration device in the calibration image deviates from the initial calibration pose.

[0073] The calibration device described above can have various specific implementations. As an example, the calibration device can be a calibration board, a calibration ruler, or other objects with calibration functions, and the present application is not limited in this regard.

[0074] Corresponding to the method embodiments described above, the present application also provides a scanner calibration device, as shown in FIG. 13, which can include:

[0075] The display unit 1301 is configured to display the calibration image currently collected by the scanner on the calibration device in real time;

[0076] The display unit 1301 is further configured to superimpose and display a virtual reference figure in the calibration image, the virtual reference figure being used to indicate the initial calibration pose to which the scanner needs to be adjusted relative to the calibration device in the calibration image.

[0077] The calibration unit 1302 is configured to display a virtual calibration indication in the calibration image in response to detecting that the scanner is adjusted to the initial calibration pose indicated by the virtual reference graphic with respect to the calibration target, the virtual calibration indication being used to instruct the user to calibrate based on the initial calibration pose.

[0078] As an example, the virtual reference graphic is determined by determining a current calibration direction in preset calibration directions, the preset calibration directions including at least one calibration direction required for the scanner to calibrate based on the calibration target, and based on the current calibration direction and an initial calibration distance; the virtual calibration indication is used to instruct the user to calibrate based on the current calibration direction; and the calibration unit 1302 is specifically configured to display the virtual calibration indication in the calibration image in response to detecting that the calibration target coincides in size and pose with the virtual reference graphic in the calibration image.

[0079] As an example, the virtual calibration indication includes a current distance indication and a calibration distance range indication; the current distance indication is used to represent a real-time distance of the calibration target with respect to the scanner, and the calibration distance range indication is used to represent a calibration distance range determined based on a nearest calibration distance and a farthest calibration distance between the scanner and the calibration target.

[0080] As an example, the current distance indication can be represented by a first virtual ring-shaped graphic, and the calibration distance range indication can be represented by a second virtual ring-shaped graphic; the second virtual ring-shaped graphic is generated by generating the second virtual ring-shaped graphic with an inner ring and an outer ring representing the nearest calibration distance and the farthest calibration distance, respectively; and the first virtual ring-shaped graphic is generated by obtaining a real-time distance of the calibration target with respect to the scanner, and generating the first virtual ring-shaped graphic according to a proportional relationship between a size of the second virtual ring-shaped graphic and an actual calibration distance represented by the size based on the real-time distance.

[0081] As an example, the virtual calibration indication is used to instruct the user to calibrate at different calibration distances based on the current calibration direction; the center of the first virtual ring-shaped graphic coincides with the center of the second virtual ring-shaped graphic; and the apparatus further includes:

[0082] The division unit is configured to divide the second virtual ring-shaped graphic into a plurality of calibration distance ranges corresponding to a number of calibration distances required for the scanner to calibrate in the current calibration direction.

[0083] The storage unit is configured to, in response to detecting that the first virtual annular pattern is in a current calibration distance level of the plurality of calibration distance levels, save a calibration image of the calibrator currently captured by the scanner, and eliminate a part of the second virtual annular pattern corresponding to the current calibration distance level after successful saving.

[0084] As an example, the virtual reference pattern is a virtual calibrator pattern, and the virtual calibrator pattern is generated by: determining a virtual pose of the scanner relative to a virtual calibrator based on the current calibration direction and the initial calibration distance; and generating the virtual calibrator pattern based on the virtual pose and camera parameters of the scanner.

[0085] As an example, the pose of the calibrator is determined by: setting three-dimensional coordinates for a marker point on the calibrator; identifying two-dimensional coordinates of the marker point in the calibration image; and determining the pose of the scanner relative to the calibrator based on the three-dimensional coordinates and the two-dimensional coordinates.

[0086] As an example, the virtual calibrator pattern has a shape that is in a mapping relationship with the current calibration direction and the initial calibration distance, wherein the current calibration direction includes an angle component of the scanner relative to the actual calibrator, and the initial calibration distance is associated with a size of the virtual calibrator pattern.

[0087] As an example, the display unit 1301 is further configured to, in response to detecting that the pose of the calibrator deviates from the virtual reference pattern after displaying the virtual calibration indication, eliminate the virtual calibration indication and display an alignment indication in the calibration image, the alignment indication being used to instruct the user to align the calibrator to the virtual reference pattern.

[0088] The present application also provides an electronic device, as shown in FIG. 14, which includes: a processor 1401; a memory 1402 for storing processor-executable instructions; wherein the processor 1401 is configured to implement the scanner calibration method described in any of the above embodiments.

[0089] The present application also provides a computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the scanner calibration method described in any of the above embodiments.

[0090] Each of the embodiments in the present application is described in a progressive manner, and the same or similar parts of each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0091] Those skilled in the art can understand that each unit and algorithm step described in the embodiments of the present application can be realized by electronic hardware, computer software or a combination of the two. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0092] The above is only a specific embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A method for calibrating a scanner, comprising: displaying a calibration image currently captured by the scanner on a calibrator in real time; superimposing a virtual reference pattern on the calibration image, the virtual reference pattern being used to indicate an initial calibration pose to which the scanner needs to be adjusted relative to the calibrator in the calibration image; in response to detecting that the scanner is adjusted to the initial calibration pose indicated by the virtual reference pattern relative to the calibrator, displaying a virtual calibration indication in the calibration image, the virtual calibration indication being used to instruct a user to calibrate based on the initial calibration pose.

2. The method of claim 1, wherein, the virtual reference pattern is determined by determining a current calibration direction in a preset calibration direction, the preset calibration direction including at least one calibration direction needed by the scanner to calibrate based on the calibrator; and the virtual calibration indication is used to instruct the user to calibrate based on the current calibration direction; the detecting that the scanner is adjusted to the initial calibration pose indicated by the virtual reference pattern relative to the calibrator comprises: detecting that the calibrator and the virtual reference pattern are overlapped in size and pose in the calibration image.

3. The method of claim 2, wherein, the virtual calibration indication includes a current distance indication and a calibration distance range indication, the current distance indication being used to represent a real-time distance of the calibrator relative to the scanner, and the calibration distance range indication being used to represent a calibration distance range determined based on a nearest calibration distance and a farthest calibration distance between the scanner and the calibrator.

4. The method of claim 3, wherein, the current distance indication can be represented by a first virtual ring pattern, and the calibration distance range indication can be represented by a second virtual ring pattern; the second virtual ring pattern is generated by: generating the second virtual ring pattern by representing the nearest calibration distance and the farthest calibration distance by an inner ring and an outer ring, respectively; the first virtual ring pattern is generated by: obtaining a real-time distance of the calibrator relative to the scanner; generating the first virtual ring pattern according to a proportional relationship between a size of the second virtual ring pattern and an actual calibration distance represented by the size, based on the real-time distance.

5. The method of claim 4, wherein, the virtual calibration indication is used to instruct the user to calibrate at different calibration distances based on the current calibration direction; the center of the first virtual ring pattern is coincident with the center of the second virtual ring pattern; the method further comprises: dividing the second virtual ring pattern into a plurality of calibration distance gears corresponding to a number of calibration distances needed by the scanner to calibrate in the current calibration direction; in response to detecting that the first virtual ring pattern is in a current calibration distance gear of the plurality of calibration distance gears, saving a calibration image currently captured by the scanner on the calibrator, and eliminating a part of the second virtual ring pattern corresponding to the current calibration distance gear after saving successfully.

6. The method according to any one of claims 2-5, wherein, the virtual reference pattern is a virtual calibrator pattern, and the virtual calibrator pattern is generated by: determine a virtual pose of the scanner relative to a virtual calibrator based on the current calibration direction and the initial calibration distance; generate the virtual calibrator pattern based on the virtual pose and camera parameters of the scanner.

7. The method of claim 6, wherein, The pose of the calibrator is determined by: setting three-dimensional coordinates for marker points on the calibrator; identifying two-dimensional coordinates of the marker points in the calibration image; determining the pose of the scanner relative to the calibrator based on the three-dimensional coordinates and the two-dimensional coordinates.

8. The method of claim 6, wherein, The virtual calibrator pattern has a mapping relationship with the current calibration direction and the initial calibration distance, wherein the current calibration direction includes an angle component of the scanner relative to the actual calibrator, and the initial calibration distance is associated with the size of the virtual calibrator pattern.

9. The method of any one of claims 2-8, wherein, The method further comprises: in response to detecting that the pose of the calibrator deviates from the virtual reference pattern after displaying the virtual calibration indication, eliminating the virtual calibration indication and displaying an alignment indication in the calibration image, the alignment indication being used to instruct the user to align the calibrator to the virtual reference pattern.

10. A scanner calibration apparatus, comprising: a display unit configured to display a calibration image currently captured by a scanner on a calibrator in real time; the display unit is further configured to superimpose and display a virtual reference pattern in the calibration image, the virtual reference pattern being used to indicate an initial calibration pose to which the scanner needs to be adjusted relative to the calibrator in the calibration image; a calibration unit configured to, in response to detecting that the scanner is adjusted to the initial calibration pose indicated by the virtual reference pattern relative to the calibrator, display a virtual calibration indication in the calibration image, the virtual calibration indication being used to instruct a user to calibrate based on the initial calibration pose.

11. An electronic device, comprising: a processor; a memory configured to store processor-executable instructions; wherein the processor is configured to implement the method of any one of claims 1 to 9.

12. A computer-readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the steps in the method of any one of claims 1 to 9.

Images