Scanner calibration method and device, electronic equipment and computer readable storage medium

Abstract

The invention discloses a scanner calibration method and device, electronic equipment and a computer readable storage medium. The scanner calibration method comprises the following steps: displaying a calibration image currently acquired by a scanner for a calibrator in real time; a virtual reference graph is displayed in the calibration image in an overlapping mode, and the virtual reference graph is used for indicating the initial calibration posture to which the scanner needs to be adjusted relative to the calibrator in the calibration image; if it is detected that the scanner is adjusted to the initial calibration posture indicated by the virtual reference graph relative to the calibrator, a virtual calibration instruction is displayed in the calibration image, and the virtual calibration instruction is used for instructing a user to conduct calibration based on the initial calibration posture. The efficiency of the calibration process can be improved.

Description

Technical Field

[0001] The present application relates to the field of scanner calibration, and in particular to a scanner calibration method, device, electronic device and computer-readable storage medium. Background Art

[0002] Calibrators are used in machine vision, image measurement, photogrammetry, 3D reconstruction and other fields. They need to establish a geometric model of camera imaging to correct lens distortion, determine the conversion relationship between physical size and pixels, and determine the relationship between the 3D geometric position of a point on the surface of a spatial object and its corresponding point in the image. By shooting a flat plate with a fixed-pitch pattern array with a camera and calculating it through a calibration algorithm, the geometric model of the camera can be derived, thereby obtaining high-precision measurement and reconstruction results. The flat plate with a fixed-pitch pattern array is a calibrator.

[0003] The calibrator can be used in a variety of scenarios. Take the scanning scenario as an example. In order to ensure the scanning accuracy of the scanner, the user needs to use the calibrator to calibrate the scanner. When performing the calibration operation, the traditional calibration method is not friendly to novice users. In the user display interface, the main information representing the calibration direction is often outside the screen captured by the scanner's camera. During the calibration process for the current calibration direction in the screen of the display interface, the user needs to pay attention to the calibration direction information outside the camera screen so that the scanner can be calibrated under the premise of maintaining the current calibration direction. On the one hand, it is difficult for the user to operate the scanner to align with the current calibration direction. On the other hand, when performing the calibration operation based on the current calibration direction, the projection center of the scanner may deviate from the center of the calibrator due to the need to distract attention to the calibration direction information outside the screen, resulting in inefficient calibration process. Summary of the invention

[0004] In view of the above technical problems, the present application provides a scanner calibration method, device, electronic device and computer-readable storage medium, and the technical solutions are as follows:

[0005] According to a first aspect of the present application, a scanner calibration method is provided, the method comprising:

[0006] Display the calibration image currently acquired by the scanner to the calibrator in real time;

[0007] Overlaying and displaying a virtual reference graphic in the calibration image, the virtual reference graphic is used to indicate an initial calibration posture to which the scanner needs to be adjusted relative to the calibrator in the calibration image;

[0008] If it is detected that the scanner is adjusted to the initial calibration posture indicated by the virtual reference figure relative to the calibrator, a virtual calibration indication is displayed in the calibration image, and the virtual calibration indication is used to instruct the user to perform calibration based on the initial calibration posture.

[0009] According to a second aspect of the present application, a scanner calibration device is provided, the device comprising:

[0010] A display unit, used for displaying in real time the calibration image currently acquired by the scanner to the calibrator;

[0011] The display unit is further used to overlay and display a virtual reference graphic in the calibration image, wherein the virtual reference graphic is used to indicate an initial calibration posture to which the scanner needs to be adjusted relative to the calibrator in the calibration image;

[0012] A calibration unit is used to display a virtual calibration indication in the calibration image if it is detected that the scanner is adjusted to the initial calibration posture indicated by the virtual reference figure relative to the calibrator, wherein the virtual calibration indication is used to instruct the user to perform calibration based on the initial calibration posture.

[0013] According to a third aspect of the present application, an electronic device is provided, the electronic device comprising:

[0014] processor;

[0015] a memory for storing processor-executable instructions;

[0016] The processor is configured to implement the method as described in the first aspect.

[0017] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the method described in the first aspect are implemented.

[0018] The technical solution provided by the present application uses AR graphics to intuitively display the calibration direction and corresponding position of the scanner, so as to guide the user to efficiently complete the entire calibration process. Specifically, by superimposing a virtual reference graphic indicating the initial calibration posture to which the scanner needs to be adjusted relative to the calibrator in the calibration image, and a virtual calibration indication for instructing the user to calibrate based on the initial calibration posture, the information representing the calibration direction can be concentrated in the calibration image, and the indication of the posture required for calibration is intuitive and easy to understand. The user only needs to focus on the information in the calibration image for calibration as much as possible, without being distracted by the information in the area outside the calibration image, thereby improving the efficiency of the calibration process.

[0019] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0021] Figure 1 It is a schematic diagram of a calibrator application scenario of the related technology;

[0022] Figure 2 is a schematic diagram of a scanner calibration scene in the related art;

[0023] Figure 3 It is a schematic diagram of a calibration direction in the related art;

[0024] Figure 4 is a flowchart of a scanner calibration method of the present application;

[0025] Figure 5 is a schematic diagram of a virtual reference graphic included in a calibration image according to an embodiment of the present application;

[0026] Figure 6 is a schematic diagram of a virtual reference graphic included in a calibration image according to another embodiment of the present application;

[0027] Figure 7 is a schematic diagram of a virtual reference graphic included in a calibration image according to another embodiment of the present application;

[0028] Figure 8 is a calibration indication diagram included in a calibration image of an embodiment of the present application;

[0029] Fig. 9 is a calibration indication diagram included in a calibration image according to another embodiment of the present application;

[0030] Fig.10 is a calibration indication diagram included in a calibration image according to another embodiment of the present application;

[0031] Fig.11 is a calibration indication diagram included in a calibration image according to another embodiment of the present application;

[0032] Fig.12 is a schematic diagram of a calibrator deviating from a virtual reference pattern according to an embodiment of the present application;

[0033] Fig.13is a schematic structural diagram of a scanner calibration device according to an embodiment of the present application;

[0034] Fig.14 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described in detail below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of this application.

[0036] Calibrators are used in machine vision, image measurement, photogrammetry, 3D reconstruction and other fields. They are used to correct lens distortion, determine the conversion relationship between physical size and pixels, and determine the relationship between the 3D geometric position of a point on the surface of a space object and its corresponding point in the image. By shooting a flat plate with a fixed-pitch pattern array and calculating it with a calibration algorithm, the geometric model of the camera can be obtained, thereby obtaining high-precision measurement and reconstruction results. Figure 1 Taking the marker shown as an example, the marker can be a flat plate with a fixed-pitch pattern array.

[0037] The calibrator can be used in a variety of scenarios. Take the scanning scenario as an example. In order to ensure the scanning accuracy of the scanner, the user needs to use the calibrator to calibrate the scanner. When performing the calibration operation, the traditional calibration method is not friendly to novice users. In the user display interface, the main information representing the calibration direction is often outside the screen captured by the scanner's camera. During the calibration process for the current calibration direction in the camera screen, the user needs to pay attention to the calibration direction information outside the camera screen so that the scanner can be calibrated under the premise of maintaining the current calibration direction. On the one hand, it is difficult for the user to operate the scanner to align with the current calibration direction. On the other hand, when performing the calibration operation based on the current calibration direction, since it is necessary to distract and pay attention to the calibration direction information outside the screen, it is easy for the scanner projection center to deviate from the calibrator center, resulting in inefficient calibration process.

[0038] See also Figure 2-Figure 3 , the following is an exemplary introduction to the scanner calibration scenario in the related art:

[0039] like Figure 2 As shown, a calibration operation method in the related art includes:

[0040] (1) The user observes the "left-tilted" / "right-tilted" area above the camera screen, and adjusts the tilt angle of the scanner relative to the calibrator in the horizontal direction based on a given tilt angle (such as the position shown by the dark gray figure). When the black frame figure in the "left-tilted" / "right-tilted" area overlaps with the dark gray figure, it is considered that the scanner has reached the appropriate initial calibration position in the horizontal direction relative to the calibrator.

[0041] (2) The user observes the "forward tilt" / "backward tilt" area on the left side outside the camera screen, and adjusts the tilt angle of the scanner in the vertical direction relative to the calibrator based on a given tilt angle (such as the position shown by the dark gray figure). When the black frame figure in the "forward tilt" / "backward tilt" area overlaps with the dark gray figure, it is considered that the scanner has reached the appropriate initial calibration position in the vertical direction relative to the calibrator.

[0042] (3) The user observes the central area of ​​the calibrator in the image captured by the scanner's camera (i.e. Figure 2 Adjust the position of the scanner's projection center (such as the medium gray circular area) Figure 2 The central red circular area) makes the scanner projection center point to the center of the calibrator, that is Figure 2 The central red circle coincides with the gray circle.

[0043] As an example, the actual direction of the scanner relative to the calibrator, "left" / "right", "forward" / "backward", can be referred to as follows: Figure 3 As shown, Figure 3 The scanner and the calibrator are kept level and the calibrator is in the correct direction.

[0044] Only after the scanner calibration positions indicated by the above steps (1)-(3) are achieved, can it be considered that the scanner is in the initial position corresponding to the current calibration direction of the calibrator. At this time, it is necessary to keep the angle between the line connecting the scanner projection center and the center of the calibrator and the straight line perpendicular to the center of the calibrator unchanged, and it is also necessary to keep the scanner projection center aligned with the center of the calibrator. Then, it is necessary to move the scanner on the line connecting its projection center and the center of the calibrator, and finally light up all the height indicator bars on the right side outside the back of the camera. The height indicated by the height indicator bar refers to the height in the vertical direction between the scanner and the calibrator.

[0045] The calibration operation method of steps (1) to (3) in the related art is not user-friendly for novice users. During calibration, the user needs to hold the device in a certain posture and move in one direction, which is time-consuming and laborious, and easily causes fatigue.

[0046] Moreover, the operation is complicated and requires attention to information in multiple different areas at the same time, which increases the calibration time and requires users to be highly concentrated. It is difficult for users to accurately find the correct position where the scanner should be based on the information on the interface. Position deviation is prone to occur during the calibration process, making it difficult to successfully complete the entire calibration process.

[0047] In view of the above problems, the present application provides a scanner calibration method, which can improve the efficiency of the calibration process. Figure 4 As shown, the method comprises the following steps:

[0048] S401, displaying in real time the calibration image currently acquired by the scanner to the calibrator.

[0049] S402: Overlay and display a virtual reference pattern in the calibration image.

[0050] The virtual reference graphic is used to indicate the initial calibration posture to which the scanner needs to be adjusted relative to the calibrator in the calibration image.

[0051] S403: If it is detected that the scanner is adjusted to the initial calibration posture indicated by the virtual reference figure relative to the calibrator, a virtual calibration indication is displayed in the calibration image.

[0052] The virtual calibration instruction is used to instruct the user to perform calibration based on the initial calibration posture.

[0053] The technical solution provided in the embodiment of the present application uses AR graphics to intuitively display the calibration direction and corresponding position of the scanner, so as to guide the user to efficiently complete the entire calibration process. Specifically, a virtual reference graphic indicating the initial calibration posture to which the scanner needs to be adjusted relative to the calibrator in the calibration image, and a virtual calibration indication for instructing the user to calibrate based on the initial calibration posture are superimposed and displayed in the calibration image. Information representing the calibration direction can be concentrated in the calibration image, and the indication of the posture required for calibration is intuitive and easy to understand. The user only needs to focus on the information in the calibration image for calibration as much as possible, without being distracted by the information in the area outside the calibration image, thereby improving the efficiency of the calibration process.

[0054] The calibration image in S401 can be collected in a variety of ways. As an example, the calibration image can be a calibration image collected in real time by a camera configured on the scanner; as another example, the position of the scanner projection center can be located at the center of the calibration image; as another example, the center position of the virtual reference figure displayed in S402 can be located at the center of the calibration image, so that the center position of the virtual reference figure is represented as the position of the projection center of the projector.

[0055] It is understandable that, since the calibration image acquired in real time includes the image of the actual calibrator, the position and size of the calibrator in the calibration image will also change with the position and distance of the scanner relative to the actual calibrator. As an example, if the center position of the virtual reference figure displayed in S402 needs to be represented as the position of the projection center of the projector, the center position of the virtual reference figure can be kept at the center position of the calibration image without changing. At the same time, if the initial calibration position that the calibrator needs to be adjusted to in the calibration image remains unchanged, the size and position of the virtual reference figure can also remain unchanged.

[0056] There are many ways to detect that the scanner is adjusted relative to the calibrator to the initial calibration posture indicated by the virtual reference figure. As an example, the virtual reference figure can be determined by determining the current calibration direction in a preset calibration direction and based on the current calibration direction and the initial calibration distance. The preset calibration direction may include at least one calibration direction required for the scanner to calibrate based on the calibrator; the virtual calibration indication displayed in S403 can be used to instruct the user to calibrate based on the current calibration direction; when it is detected that the size and posture of the calibrator and the virtual reference figure in the calibration image coincide, it can be determined that the calibrator is adjusted to the initial calibration posture indicated by the virtual reference figure; as another example, the initial calibration posture indicated by the virtual reference figure may include the above-mentioned current calibration direction, and the initial calibration distance between the scanner and the actual calibrator (the distance can be displayed by the size of the virtual reference figure).

[0057] The preset calibration direction may include at least one calibration direction required for the scanner to calibrate based on the calibrator, and the at least one calibration direction required for the scanner to calibrate based on the calibrator is determined according to the actual calibration process, in which postures the scanner needs to be calibrated relative to the calibrator, such as vertically upward, forward tilted, backward tilted, left tilted, and right tilted, and the preset different calibration directions can uniquely identify a calibration posture. It is understandable that if there are multiple preset calibration directions, the scanner needs to be calibrated in each of the calibration directions to complete the complete calibration process.

[0058] As an example, the above-mentioned initial calibration distance between the scanner and the actual calibrator may be the farthest calibration distance required between the scanner and the actual calibrator; as another example, the initial calibration distance may also be the nearest calibration distance, or any calibration distance between the farthest calibration distance and the nearest calibration distance; therefore, there is no limitation on the specific implementation of the initial calibration distance.

[0059] The virtual reference graph displayed in S402 can be generated in a variety of ways. As an example, the virtual reference graph can be a virtual calibrator graph. Based on the current calibration direction determined in the preset calibration direction and the initial calibration distance, the virtual position and posture of the scanner relative to the virtual calibrator can be determined, and then the virtual calibrator graph is generated based on the determined virtual position and posture and the camera parameters of the scanner. As another example, the intrinsic parameter matrix of the camera that collects the calibration image on the scanner can be obtained first. After determining the virtual position and posture of the virtual calibrator relative to the scanner, the coordinate transformation matrix of the virtual calibrator to the camera is obtained, and the three-dimensional coordinates of the key points on the virtual calibrator in the camera coordinate system are calculated according to the coordinate transformation matrix and the virtual position and posture, and then the two-dimensional coordinates of the key points on the calibrator on the camera pixel plane are calculated according to the three-dimensional coordinates and the intrinsic parameter matrix of the camera, and the virtual calibrator graph is generated and displayed based on the calculated two-dimensional coordinates. It is worth noting that the above introduction to the virtual calibrator graph generation method is only an exemplary display. In practical applications, other generation methods are not excluded, and there is no specific limitation on this.

[0060] As an example, there is a mapping relationship between the shape of the virtual calibrator graphic and the current calibration direction and the initial calibration distance, wherein the current calibration direction includes the angle component of the scanner relative to the actual calibrator, and the initial calibration distance can be associated with the size of the virtual calibrator graphic. As another example, the angle component of the scanner relative to the actual calibrator can be the tilt angle component of the scanner relative to the actual calibrator, such as the left / right tilt angle, and / or the forward / backward tilt angle. It can be understood that the current calibration direction can determine the degree of tilt of the virtual calibrator graphic in the calibration image, and the initial calibration distance can determine the size of the virtual calibrator graphic in the calibration image. The shape of the virtual calibrator graphic can intuitively reflect the initial calibration distance and the angle component of the scanner relative to the actual calibrator, and convert the angle component that the user originally needs to align during calibration into an alignment that coincides with the graphic posture and size in the calibration image, so that the user can intuitively perform calibration and alignment.

[0061] See also Figure 5 , Figure 6 , the virtual calibrator graphics and calibrator in the calibration image of the embodiment of the present application are exemplarily introduced as follows:

[0062] like Figure 5As shown in the figure, the black area shows the calibration image collected in real time by the calibrator (specifically, it can be taken by the black and white camera of the calibrator). The calibration image contains the actual calibrator image (gray with white dots) and the red figure in the center of the calibration image. The red figure is the virtual calibrator figure mentioned above. Taking the initial calibration distance as the farthest calibration distance of the scanner relative to the calibrator as an example, the red figure can be the size and posture of the calibrator that should be seen in the calibration image at the farthest calibration distance. The user needs to adjust the relative position between the scanner and the actual calibrator to align and overlap the actual calibrator (gray with white dots) in the calibration image with the red figure (virtual calibrator figure), including the overlap of size and posture.

[0063] like Figure 6 As shown, if the current calibration direction is a calibration direction that requires the scanner to have an inclination angle relative to the actual calibrator (e.g., left / right, forward / backward), the red graphic (virtual calibrator graphic) can be displayed as a perspective effect and form a tilted posture (e.g., Figure 6 to guide the user to move and / or rotate the scanner so that a corresponding tilted posture and a required distance position are formed between the scanner and the actual calibrator, thereby adjusting the calibrator part in the calibration image to a posture corresponding to the red graphic.

[0064] See also Figure 7 The virtual reference pattern can also be other patterns. As an example, the virtual reference pattern can also be a pattern composed of a ring and a cross at the center of the ring, such as Figure 7 As shown, Figure 7 The red figure formed by the circle at the center position of the calibration image and the center cross is the virtual reference figure. The calibration image may also include a white figure drawn in real time based on the actual calibration plate relative to the scanner (that is, the image composed of the white circle and the center cross located at the center position of the calibration plate in the calibration image); as another example, another way to detect that the calibrator is adjusted to the initial calibration posture indicated by the virtual reference figure may include: detecting in the calibration image whether the white figure corresponding to the calibrator is aligned with the red figure at the center position of the calibration image (for example, whether the center crosses of the two are aligned); if the two are aligned, it is determined that the calibrator is adjusted to the above-mentioned initial calibration posture.

[0065] The actual position and pose of the calibrator can be determined in a variety of ways. As an example, the three-dimensional coordinates of the marker points on the calibrator can be set first, and then the two-dimensional coordinates of the marker points of the calibrator can be identified in the calibration image, and then the position and pose of the scanner relative to the calibrator can be determined based on the set three-dimensional coordinates and the identified two-dimensional coordinates. As another example, after identifying the two-dimensional coordinates of the marker points, the identified two-dimensional coordinates can be aligned with the set three-dimensional coordinates, and the actual position and pose of the calibrator relative to the scanner can be calculated according to the perspective n-point problem algorithm (Perspective-n-Point, PnP), wherein the basic idea of ​​the PnP algorithm is to use the coordinates of some known points on the object (calibrator) (usually the corner points or feature points of the object, such as the marker points of the calibrator) in the three-dimensional space, and the projection coordinates of these points on the two-dimensional image, to calculate the rotation matrix (R) and displacement vector (t) of the camera that collects the two-dimensional image, that is, the external parameters of the camera, so that the object model in the three-dimensional space can be mapped to the two-dimensional image, thereby determining the position and pose of the camera relative to the three-dimensional object (marker).

[0066] There are many ways to set the three-dimensional coordinates for the marker points on the calibrator. As an example, the calibrator plane can be taken as the plane with Z=0, and the three-dimensional coordinates can be generated according to the two-dimensional coordinates of the marker points on the calibrator on the plane formed by the X-axis and the Y-axis (the two-dimensional coordinates are obtained by measurement), and the three-dimensional coordinates are used as the set coordinates. As another example, the three-dimensional coordinates can be the coordinates relative to the center of the actual calibrator. The center of the calibrator is generally the geometric center of the calibrator. An orthogonal right-handed coordinate system can be established perpendicular to the calibrator with the positive direction of the Z axis pointing upward.

[0067] The virtual calibration indication displayed in S403 can have multiple specific implementations. As an example, the virtual calibration indication can include a current distance indication and a calibration distance range indication; the current distance indication is used to represent the real-time distance of the calibrator relative to the scanner, and the calibration distance range indication is used to represent the calibration distance range determined based on the nearest calibration distance and the farthest calibration distance between the scanner and the calibrator.

[0068] The current distance indication and the calibrated distance range indication can be displayed in the calibration image in a variety of ways. As an example, the current distance indication can be represented by a first virtual ring figure, and the calibrated distance range indication can be represented by a second virtual ring figure; the second virtual ring figure can be generated in the following way: the inner ring and the outer ring respectively represent the closest calibration distance and the farthest calibration distance between the scanner and the calibrator, and a second virtual ring figure is generated, the inner ring of the generated second virtual ring figure represents the closest calibration distance, and the outer ring represents the farthest calibration distance; the above-mentioned first virtual ring figure can be generated in the following way: first obtain the real-time distance of the calibrator relative to the scanner, and then based on the real-time distance of the calibrator relative to the scanner, generate the first virtual ring figure according to the proportional relationship between the size of the above-mentioned second virtual ring figure and the actual calibration distance represented by the size.

[0069] The first virtual ring shape and the second virtual ring shape can have multiple specific implementations. As an example, the first virtual ring shape and the second virtual ring shape can both be circular ring shapes, or can both be other types of shapes, such as rectangular ring shapes, triangular ring shapes, etc., so there is no limitation on the specific implementation of the two.

[0070] The first virtual annular figure can be generated in a variety of ways. As an example, when the first virtual annular figure and the second virtual annular figure are both circular annular figures, the outer ring radius R and the inner ring radius r of the second virtual annular figure displayed in the calibration image can be first determined, and then the farthest calibration distance H corresponding to the outer ring and the closest calibration distance h corresponding to the inner ring can be determined. The proportional relationship between the size of the second virtual annular figure and the actual calibration distance represented by the size can be calculated by the following formula:

[0071] K=(Rr) / (Hh) (1)

[0072] Wherein, K is a proportionality coefficient between the size of the second virtual annular figure and the actual calibration distance represented by the size.

[0073] The real-time distance of the calibrator relative to the scanner is recorded as height, and the radius of the first virtual annular figure can be calculated by the following formula:

[0074]

[0075] Based on the calculated radius, a first virtual ring-shaped figure may be generated.

[0076] The virtual calibration indication can be used to instruct the user to calibrate at different calibration distances based on the current calibration direction. As an example, the first virtual ring figure can coincide with the center of the second virtual ring figure. Based on the number of calibration distances required for the scanner to calibrate in the current calibration direction, the second virtual ring figure can be divided into multiple calibration distance positions corresponding to the number. If it is detected that the first virtual ring figure is in the current calibration distance position among the multiple calibration distance positions, the calibration image currently captured by the scanner for the calibrator is saved, and after the saving is successful, the part of the second virtual ring figure corresponding to the current calibration distance position is eliminated. It can be understood that the size of the first virtual annular figure can change with the change of the real-time distance of the calibrator relative to the scanner. When the real-time distance increases, the first virtual annular figure increases, and when the real-time distance decreases, the first virtual annular figure decreases. In the process of the size change of the first virtual annular figure, the first virtual annular figure will stay at the graphic part corresponding to each calibration distance gear among the multiple calibration distance gears of the second virtual annular figure. Whenever the first virtual annular figure stays at the graphic part corresponding to the current calibration distance gear, after the calibration image currently collected by the calibrator for the scanner at the calibration distance corresponding to the current calibration distance gear is successfully saved, the graphic part corresponding to the current calibration distance gear will be eliminated. Through the size change of the first virtual annular figure, the graphic parts corresponding to each calibration distance gear of the second virtual annular figure are eliminated, and the calibration image acquisition process in the current calibration direction can be completed. By converting the three-dimensional distance relationship between the calibrator and the scanner into the size of a two-dimensional figure, all information (the information represented by the virtual reference figure and the information displayed by the virtual calibration indication) is concentrated in the calibration image. The user does not need to be distracted by areas outside the calibration image during the calibration process, thereby further improving the calibration efficiency.

[0077] As an example, the virtual calibration indication may be displayed at the center of the calibration image, or may be displayed at other locations of the calibration image, which is not specifically limited.

[0078] See also Figures 8 to 11 , the following is an exemplary introduction to a virtual calibration indication of an embodiment of the present application:

[0079] like Figure 8 As shown, taking the virtual reference figure as the virtual calibrator figure as an example, when the size and posture of the virtual calibrator figure in the calibration image coincide with the calibrator, it can be determined that the calibrator is adjusted to the initial calibration posture indicated by the virtual calibrator figure. At this time, the virtual calibration indication is displayed in the calibration image: the first virtual annular figure (such as Figure 8 The white circle in the figure) and the second virtual ring figure (such as Figure 8 The blue circle in the figure). The white circle represents the relative real-time distance between the scanner and the actual calibrator, such as Fig. 9 As shown in , when the real-time distance increases, the white ring also increases accordingly, such as Fig.10 As shown in the figure, when the real-time distance becomes smaller, the white ring also becomes smaller accordingly, and any part of the blue ring swept by the white ring will be eliminated when the calibration image corresponding to the part of the figure is saved. Elimination means that the calibration distance gear corresponding to the part of the figure has successfully collected and saved the calibration image. The blue ring represents the calibration distance range required between the scanner and the actual calibration plate in the current calibration direction. Keep the scanner moving within the calibration distance range in the current calibration direction, and use the white ring to eliminate all the blue rings to complete the calibration image acquisition work in the current calibration direction.

[0080] like Fig.11 As shown, the virtual reference figure is as follows Figure 7 Taking the figure composed of the circle and the central cross shown in the figure as an example, after the white figure corresponding to the calibrator in the calibration image is aligned with the red figure in the center of the calibration image, it is determined that the calibrator is adjusted to the initial calibration posture, and the virtual calibration indication is also displayed: a white ring and a blue ring.

[0081] As an example, after the virtual calibration indication is displayed, the virtual reference graphic in the calibration image may be eliminated or may continue to be stored and displayed, and this is not specifically limited.

[0082] Considering that after the virtual calibration indication is displayed, the scanner may deviate from the initial calibration posture when the user is calibrating based on the virtual calibration indication, resulting in a large calibration error under the initial calibration posture.

[0083] For this problem, as an example, if it is detected that the position of the calibrator deviates from the virtual reference figure after the virtual calibration indication is displayed, the virtual calibration indication is eliminated, and an alignment indication is displayed in the calibration image, and the alignment indication is used to instruct the user to align the calibrator with the virtual reference figure. As another example, the alignment indication can be a text prompt or a graphic prompt; as another example, after it is detected in the calibration image that the calibrator is realigned with the virtual reference figure, the above S403 can be executed again. See Fig.12 As another example, if the virtual reference graphic is in an eliminated state during the user's calibration based on the virtual calibration indication, when it is detected that the posture of the calibrator deviates from the initial calibration posture corresponding to the virtual reference graphic, the virtual calibration indication can be eliminated and the virtual reference graphic can be redisplayed to guide the user to realign.

[0084] As an example, whether the position of the calibrator in the calibration image deviates from the initial calibration pose relative to the scanner can be judged by comparing the distance between the two-dimensional coordinates of the center of the actual calibrator on the calibration image and the coordinates of the center point of the calibration image with a distance threshold. When the distance is less than or equal to the distance threshold, it can be considered that the position of the calibrator in the calibration image has not deviated from the initial calibration pose. When the distance is greater than the distance threshold, it can be considered that the position of the calibrator in the calibration image has deviated from the initial calibration pose. As another example, whether the angle of the calibrator in the calibration image deviates from the initial calibration pose relative to the scanner can be judged by comparing the angle between the actual rotation amount of the calibration plate relative to the scanner (for example, the rotation amount represented by the quaternion) and the rotation amount of the virtual calibration plate relative to the scanner corresponding to the initial calibration pose with an angle threshold. When the angle is less than or equal to the angle threshold, it can be considered that the angle of the calibrator in the calibration image has not deviated from the initial calibration pose. When the angle is greater than the angle threshold, it can be considered that the angle of the calibrator in the calibration image has deviated from the initial calibration pose.

[0085] The above-mentioned calibrator can be implemented in many ways. For example, the calibrator can be a calibration plate, a calibration ruler, or other objects with calibration functions, without limitation.

[0086] Corresponding to the above method embodiment, the present application embodiment also provides a scanner calibration device, see Fig.13 As shown, the device may include:

[0087] The display unit 1301 is used to display the calibration image currently acquired by the scanner to the calibrator in real time;

[0088] The display unit 1301 is further used to overlay and display a virtual reference graphic in the calibration image, where the virtual reference graphic is used to indicate the initial calibration posture to which the scanner needs to be adjusted relative to the calibrator in the calibration image;

[0089] The calibration unit 1302 is used to display a virtual calibration indication in the calibration image if it is detected that the scanner is adjusted to the initial calibration posture indicated by the virtual reference figure relative to the calibrator, and the virtual calibration indication is used to instruct the user to calibrate based on the initial calibration posture.

[0090] As an example, the virtual reference figure is determined by determining a current calibration direction in a preset calibration direction and based on the current calibration direction and an initial calibration distance, and the preset calibration direction includes at least one calibration direction required for the scanner to calibrate based on the calibrator; the virtual calibration indication is used to instruct the user to calibrate based on the current calibration direction; the calibration unit 1302 is specifically used to display a virtual calibration indication in the calibration image if it is detected that the size and posture of the calibrator and the virtual reference figure in the calibration image coincide.

[0091] As an example, the virtual calibration indication includes a current distance indication and a calibration distance range indication; the current distance indication is used to represent the real-time distance of the calibrator relative to the scanner, and the calibration distance range indication is used to represent the calibration distance range determined based on the nearest calibration distance and the farthest calibration distance between the scanner and the calibrator.

[0092] As an example, the current distance indication can be represented by a first virtual ring-shaped graph, and the calibrated distance range indication can be represented by a second virtual ring-shaped graph; the second virtual ring-shaped graph is generated in the following manner: an inner ring and an outer ring represent the nearest calibrated distance and the farthest calibrated distance respectively, thereby generating the second virtual ring-shaped graph; the first virtual ring-shaped graph is generated in the following manner: the real-time distance of the calibrator relative to the scanner is obtained; based on the real-time distance, the first virtual ring-shaped graph is generated according to the proportional relationship between the size of the second virtual ring-shaped graph and the actual calibrated distance represented by the size.

[0093] As an example, the virtual calibration indication is used to instruct the user to calibrate at different calibration distances based on the current calibration direction; the center of the first virtual annular figure coincides with the center of the second virtual annular figure; the device further includes:

[0094] a dividing unit, configured to divide the second virtual annular figure into a plurality of calibration distance gears corresponding to the number of calibration distances required for the scanner to calibrate in the current calibration direction;

[0095] A storage unit is used to save the calibration image currently acquired by the scanner on the calibrator if it is detected that the first virtual annular figure is in the current calibration distance position among the multiple calibration distance positions, and to eliminate the portion of the second virtual annular figure corresponding to the current calibration distance position after successful saving.

[0096] As an example, the virtual reference graphic is a virtual calibrator graphic, and the virtual calibrator graphic is generated in the following manner: determining the virtual pose of the scanner relative to the virtual calibrator based on the current calibration direction and the initial calibration distance; generating the virtual calibrator graphic based on the virtual pose and the camera parameters of the scanner.

[0097] As an example, the position and posture of the calibrator is determined in the following manner: setting three-dimensional coordinates for the marker points on the calibrator; identifying the two-dimensional coordinates of the marker points in the calibration image; and determining the position and posture of the scanner relative to the calibrator based on the three-dimensional coordinates and the two-dimensional coordinates.

[0098] As an example, there is a mapping relationship between the shape of the virtual calibrator graphic and the current calibration direction and the initial calibration distance, wherein the current calibration direction includes an angular component of the scanner relative to the actual calibrator, and the initial calibration distance can be associated with the size of the virtual calibrator graphic.

[0099] As an example, the display unit 1301 is also used to eliminate the virtual calibration indication and display an alignment indication in the calibration image if it is detected that the posture of the calibrator deviates from the virtual reference figure after the virtual calibration indication is displayed, and the alignment indication is used to instruct the user to align the calibrator with the virtual reference figure.

[0100] The present application also provides an electronic device, such as Fig.14 As shown, the electronic device includes: a processor 1401; a memory 1402 for storing processor executable instructions; wherein the processor 1401 is configured to implement the scanner calibration method described in any of the above embodiments.

[0101] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the scanner calibration method described in any of the above embodiments is implemented.

[0102] The above is only a specific implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A scanner calibration method, characterized in that: include: Display the calibration image currently acquired by the scanner to the calibrator in real time; Overlaying and displaying a virtual reference graphic in the calibration image, the virtual reference graphic is used to indicate an initial calibration posture to which the scanner needs to be adjusted relative to the calibrator in the calibration image; If it is detected that the scanner is adjusted to the initial calibration posture indicated by the virtual reference figure relative to the calibrator, a virtual calibration indication is displayed in the calibration image, and the virtual calibration indication is used to instruct the user to perform calibration based on the initial calibration posture.

2. The method according to claim 1, characterized in that The virtual reference pattern is determined by determining a current calibration direction in a preset calibration direction and based on the current calibration direction and an initial calibration distance, wherein the preset calibration direction includes at least one calibration direction required for the scanner to calibrate based on the calibrator; the virtual calibration indication is used to instruct the user to calibrate based on the current calibration direction; The detecting that the scanner is adjusted relative to the calibrator to the initial calibration posture indicated by the virtual reference figure includes: It is detected that in the calibration image, the size and the posture of the calibrator and the virtual reference figure coincide with each other.

3. The method according to claim 2, characterized in that The virtual calibration indication includes a current distance indication and a calibration distance range indication; the current distance indication is used to represent the real-time distance of the calibrator relative to the scanner, and the calibration distance range indication is used to represent the calibration distance range determined based on the nearest calibration distance and the farthest calibration distance between the scanner and the calibrator.

4. The method according to claim 3, characterized in that The current distance indication can be represented by a first virtual annular graphic, and the calibrated distance range indication can be represented by a second virtual annular graphic; The second virtual annular graphic is generated by: The inner ring and the outer ring represent the closest calibrated distance and the farthest calibrated distance respectively, and generate the second virtual ring graph; The first virtual annular graphic is generated by: Obtaining a real-time distance between the calibrator and the scanner; Based on the real-time distance, the first virtual annular figure is generated according to the proportional relationship between the size of the second virtual annular figure and the actual calibrated distance represented by the size.

5. The method according to claim 4, characterized in that The virtual calibration indication is used to instruct the user to perform calibration at different calibration distances based on the current calibration direction; The centers of the first virtual annular figure and the second virtual annular figure coincide with each other; The method further comprises: Based on the number of calibration distances required for the scanner to calibrate in the current calibration direction, dividing the second virtual annular figure into a plurality of calibration distance gears corresponding to the number; If it is detected that the first virtual annular figure is at the current calibration distance position among the multiple calibration distance positions, the calibration image currently captured by the scanner on the calibrator is saved, and after successful saving, the part of the second virtual annular figure corresponding to the current calibration distance position is eliminated.

6. The method according to claim 2, characterized in that The virtual reference graph is a virtual calibrator graph, and the virtual calibrator graph is generated in the following manner: Determining a virtual position and posture of the scanner relative to a virtual calibrator based on the current calibration direction and the initial calibration distance; The virtual calibrator graphic is generated based on the virtual pose and the camera parameters of the scanner.

7. The method according to claim 6, characterized in that The position and posture of the calibrator are determined by: Setting the three-dimensional coordinates for the marking points on the calibrator; Identifying the two-dimensional coordinates of the marker points in the calibration image; Based on the three-dimensional coordinates and the two-dimensional coordinates, the position and posture of the scanner relative to the calibrator is determined.

8. The method according to claim 6, characterized in that There is a mapping relationship between the shape of the virtual calibrator graphic and the current calibration direction and the initial calibration distance, wherein the current calibration direction includes an angle component of the scanner relative to the actual calibrator, and the initial calibration distance can be associated with the size of the virtual calibrator graphic.

9. The method according to claim 2, characterized in that: The method further comprises: If it is detected that the position of the calibrator deviates from the virtual reference figure after the virtual calibration indication is displayed, the virtual calibration indication is eliminated and an alignment indication is displayed in the calibration image, wherein the alignment indication is used to instruct the user to align the calibrator with the virtual reference figure.

10. A scanner calibration device, characterized in that: include: A display unit, used for displaying in real time the calibration image currently acquired by the scanner to the calibrator; The display unit is further used to overlay and display a virtual reference graphic in the calibration image, wherein the virtual reference graphic is used to indicate an initial calibration posture to which the scanner needs to be adjusted relative to the calibrator in the calibration image; A calibration unit is used to display a virtual calibration indication in the calibration image if it is detected that the scanner is adjusted to the initial calibration posture indicated by the virtual reference figure relative to the calibrator, wherein the virtual calibration indication is used to instruct the user to perform calibration based on the initial calibration posture.

11. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images