Visual calibration method, computer equipment, storage medium and program product

Abstract

The invention relates to a visual calibration method, computer equipment, a storage medium and a program product. The method comprises the following steps: displaying a calibration image acquisition interface, wherein the calibration image acquisition interface comprises a target pose identifier; in response to the pose change of the to-be-calibrated equipment, controlling the first movable identifier to execute an action on the calibration image acquisition interface along with the position change, and controlling the second movable identifier to execute an action on the calibration image acquisition interface along with the pose change; the pose change at least comprises one of the position change and the pose change; when the first movable identifier and the second movable identifier are both matched with the target pose identifier, acquiring a calibration image of a current pose through the to-be-calibrated device; the calibration image is used for calibrating the equipment to be calibrated. The method can improve the calibration efficiency.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a visual calibration method, computer equipment, storage medium, and program product. Background Art

[0002] To ensure high-precision measurement results from handheld scanners, it's necessary to compensate for structural deformation caused by heat generation during operation or mechanical vibration during transportation. Calibration compensates for this structural deformation. Calibration is essential for handheld scanners to capture high-quality 3D data. A mature and reliable solution currently available uses a calibration plate with geometric features. Scanning the plate determines the correspondence between physical coordinates and image coordinates. Calibration using the plate indicates different heights and postures at different locations. Traditionally, the desired posture is displayed on the user interface, requiring the user to repeatedly adjust the calibration plate's position, both coarsely and finely, to achieve the desired posture, resulting in low calibration efficiency. Summary of the Invention

[0003] Based on this, it is necessary to provide a visual calibration method, computer equipment, storage medium and program product that can improve calibration efficiency in response to the above technical problems.

[0004] A visual calibration method, comprising: Displaying a calibration image acquisition interface, wherein the calibration image acquisition interface includes a target posture identifier; In response to a change in the posture of the device to be calibrated, controlling the first movable marker to perform an action on the calibration image acquisition interface as the position changes, and controlling the second movable marker to perform an action on the calibration image acquisition interface as the posture changes; the posture change includes at least one of the position change and the posture change; When both the first movable marker and the second movable marker match the target posture marker, a calibration image of the current posture is collected by the device to be calibrated; the calibration image is used to calibrate the device to be calibrated.

[0005] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of each embodiment of a visual calibration method when executing the computer program.

[0006] A computer-readable storage medium stores a computer program, which implements the steps of each visual calibration method embodiment when executed by a processor.

[0007] A computer program product includes a computer program, which implements the steps of each visual calibration method embodiment when executed by a processor.

[0008] The above-mentioned visual calibration method, computer equipment, storage medium and program product display the target posture identifier in the calibration image acquisition interface, and in response to the posture change of the device to be calibrated, control the first movable identifier to perform an action as the position changes, and control the second movable identifier to perform an action as the posture changes, thereby decomposing the posture of the device to be calibrated into two parts, position and posture, and visualizing them separately; when the first movable identifier and the second movable identifier both match the target posture identifier, a calibration image of the current posture is collected for calibrating the device to be calibrated, and the abstract three-dimensional posture is converted into an intuitive and easy-to-understand two-dimensional image interface, so that the user can perform posture matching in steps and parts, greatly reducing the difficulty of matching the target posture, and can perform calibration within the same interface, thereby significantly improving the calibration efficiency and posture accuracy, thereby improving the accuracy of calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A diagram illustrating an application environment of a visual calibration method according to an embodiment; Figure 2 Schematic diagram of a flow chart of a visual calibration method in one embodiment; Figure 3 A schematic diagram of a calibration position trend in one embodiment; Figure 4 Schematic diagram of the coordinate orientation of a binocular camera in one embodiment; Figure 5 A schematic diagram of positioning trend in another embodiment; Figure 6 A schematic diagram of a target pose setting process in one embodiment; Figure 7 A schematic diagram of a movable marker posture matching process in one embodiment; FIG8( a ) is a schematic diagram of a calibration image acquisition interface in one embodiment; FIG8( b ) is a schematic diagram of target pose identification and tilted area in one embodiment; Figure 9 A schematic diagram of an interface for calibrating identifier matching of an image acquisition interface in one embodiment; Figure 10 A schematic diagram of a flow chart of identification matching in one embodiment; Figure 11 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0010] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0011] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0012] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly. The connection can be a direct connection or an indirect connection.

[0013] In addition, the descriptions of "first", "second", etc. in this application are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0014] As used herein, the terms "first," "second," and the like may be used to describe various data, but these data are not limited by these terms. These terms are used solely to distinguish a first data item from another data item. For example, without departing from the scope of this application, a first movable marker may be referred to as a second movable marker, and similarly, a second movable marker may be referred to as a first movable marker. Both the first movable marker and the second movable marker are movable markers, but they are not the same movable marker.

[0015] It can be understood that the “connection” in the following embodiments should be understood as “electrical connection”, “communication connection”, etc. if there is transmission of electrical signals or data between the connected circuits, modules, units, etc.

[0016] It can be understood that the operation of "obtaining data" in the embodiments of the present application includes but is not limited to the following implementation methods: directly reading the original data pre-stored in the device; or indirect acquisition after data collection, conversion and processing.

[0017] The visualization calibration method provided in this application can be applied to Figure 1 application environment. Figure 1Figure 1 is an application environment diagram of a visual calibration method in one embodiment. Terminal device 110 may be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices. The server may be implemented as a standalone server or a server cluster consisting of multiple servers. Device to be calibrated 120 may be a binocular vision scanner, camera, or the like. Terminal device 110 and device to be calibrated 120 are connected by wire or wirelessly.

[0018] In one embodiment, Figure 2 FIG. 1 is a flow chart of a visual calibration method in an embodiment, wherein the method is applied to Figure 1 The terminal device is used as an example to illustrate the process, including the following steps: Step 202: Display a calibration image acquisition interface, which includes a target posture identifier.

[0019] Among them, the calibration image acquisition interface is a user interaction interface that integrates visual posture guidance and real-time dynamic recognition functions. It maps the three-dimensional spatial posture relationship in real time through two-dimensional graphical elements, guiding the operator to complete the image acquisition work during the calibration process.

[0020] The target pose indicator is located on the calibration image acquisition screen and is used to indicate the target pose and align the device to be calibrated in 3D space. The target pose is a predefined 3D spatial pose that needs to be achieved during the calibration process.

[0021] Specifically, a calibration image acquisition interface is displayed on the terminal interface, and the calibration image interface includes a target pose identifier. The target pose identifier can be a single identifier or two identifiers. For example, the target pose identifier is a circular surface contained in a ring, where the ring is the target pose identifier corresponding to the first movable identifier, and the circular surface is the target pose identifier corresponding to the second movable identifier. Alternatively, the circular surface contained in the ring can be considered the same identifier.

[0022] Step 204, in response to the posture change of the device to be calibrated, control the first movable marker to perform an action in the calibration image acquisition interface as the position changes, and control the second movable marker to perform an action in the calibration image acquisition interface as the posture changes; the posture change includes at least one of a position change and a posture change.

[0023] The first and second movable markers are markers that can perform actions on the image acquisition interface. The first and second movable markers can take any shape, including crosshairs, arrows, dynamic cursors, or geometric shapes (such as circles, rectangles, and donuts). Their display properties (such as color, transparency, and size) can be dynamically adjusted based on the calibration status.

[0024] The first movable marker is used to indicate position changes and can also be used to indicate plane rotation changes. The second movable marker is used to indicate posture changes. Actions performed on the image acquisition interface can include movement, rotation, and deformation, and deformation can include reduction, enlargement, and missing corners.

[0025] Specifically, the posture change may be only a position change, only a posture change, or both a position and a posture change. The device to be calibrated can collect its own displacement. In response to the position change of the device to be calibrated, the terminal device controls the first movable marker to perform actions in the image acquisition interface as the position changes, such as movement and scaling, etc., but not limited to this. For example, when the device to be calibrated produces a translational displacement along the X-axis or Y-axis direction, the first movable marker in the calibration image acquisition interface will move synchronously along the corresponding direction, and its scaling ratio will be dynamically adjusted according to the preset scaling coefficient based on the displacement. When the device to be calibrated produces a translational displacement along the Z-axis direction, the first movable marker will scale as the Z-axis displacement changes.

[0026] In response to the posture change of the device to be calibrated, the terminal device controls the second movable marker to perform an action on the image acquisition interface as the posture changes, such as movement and rotation, etc., but not limited to this. For example, the distance between the second movable marker and the target posture marker is used to represent the deviation angle of the device to be calibrated in the corresponding direction. The greater the deviation angle, the greater the distance. When they coincide, it means that the target posture has been reached. Alternatively, the second movable marker can be a quasi-three-dimensional marker that is synchronized with the posture of the device to be calibrated. The terminal device can match the posture of the second movable marker with the posture of the target posture marker.

[0027] Step 206: When both the first movable marker and the second movable marker match the target posture marker, a calibration image of the current posture is collected by the device to be calibrated; the calibration image is used to calibrate the device to be calibrated.

[0028] The current posture is the posture when the first movable marker and the second movable marker match the target posture marker.

[0029] Specifically, the target posture identifier includes a target position identifier and a target posture identifier, the target position identifier is used to match the first movable identifier, and the target posture identifier is used to match the second movable identifier. It can be understood that the target position identifier and the target posture identifier can be integrated into one, or can be two separate ones. When the first movable identifier moves to the target position identifier and the size coincides with the part, it is determined that the first movable identifier matches the target position identifier. When the distance between the second movable identifier and the target posture identifier is used to represent the offset angle of the device to be calibrated in the corresponding direction, the second movable identifier moves to the target posture identifier, and then it is determined that the two match. Optionally, the second movable identifier is a three-dimensional-like identifier representing the posture, and the posture of the second movable identifier is required to match the target posture identifier.

[0030] When both the first and second movable markers match the target pose marker, the terminal device displays an image acquisition marker and captures a calibration image of the current pose. After preprocessing the calibration image, the terminal device inputs the coordinates of the calibration points in the image and on the calibration object to perform an adjustment. This adjustment iterates the relative position relationship of the binocular vision cameras to correct for errors caused by vibration.

[0031] In this embodiment, the target posture identifier is displayed in the calibration image acquisition interface. In response to the posture change of the device to be calibrated, the first movable identifier is controlled to perform an action as the position changes, and the second movable identifier is controlled to perform an action as the posture changes. The posture of the device to be calibrated is decomposed into two parts, position and posture, which are visualized separately. When the first movable identifier and the second movable identifier both match the target posture identifier, a calibration image of the current posture is collected for calibrating the device to be calibrated. The abstract three-dimensional posture is converted into an intuitive and easy-to-understand two-dimensional image interface, so that the user can perform posture matching in steps and parts, which greatly reduces the difficulty of matching the target posture, and can be calibrated in the same interface, thereby significantly improving the calibration efficiency and posture accuracy, thereby improving the accuracy of calibration.

[0032] In one embodiment, in response to a change in the posture of the device to be calibrated, controlling the first movable marker to perform an action on the calibration image acquisition interface as the position changes includes: In response to a horizontal translation change of the device to be calibrated, controlling the first movable marker to move on the calibration image acquisition interface along with the horizontal translation change; In response to a change in the distance of the device to be calibrated, the first movable marker is controlled to zoom in and out on the calibration image acquisition interface as the distance changes.

[0033] The horizontal translation change refers to the coordinate change on a certain plane, such as the change in the X-axis direction and the change in the Y-axis direction on the XY plane, or the translation change on a plane that matches the plane of the calibration object, such as a parallel plane.

[0034] The distance change can be the change in distance from the calibration object, and the direction represented by the distance change is perpendicular to the horizontal plane translation change. The horizontal plane translation change and the distance change represent the three directions of the three-dimensional coordinates.

[0035] Specifically, in response to the horizontal translation change of the device to be calibrated, the terminal device controls the first movable marker to move on the calibration image acquisition interface along with the horizontal translation change; in response to the distance change of the device to be calibrated, the terminal device controls the first movable marker to scale on the calibration image acquisition interface along with the distance change. For example, if the change on the XY plane is the horizontal translation change and the change in the Z direction is the distance change, then when the XY coordinates of the device to be calibrated change, the first movable marker moves on the calibration image acquisition interface, and when the Z coordinate of the device to be calibrated changes, the first movable marker scales. The scaling direction is set according to demand, and one of the situations of being greater than the target distance and less than the target distance is zooming in, and the other situation is zooming out.

[0036] In this embodiment, in response to the horizontal translation change of the device to be calibrated, the first movable marker is controlled to move in the calibration image acquisition interface along with the horizontal translation change; in response to the distance change of the device to be calibrated, the second movable marker is controlled to move in the calibration image acquisition interface along with the distance change. This can truthfully reflect the changes of the device to be calibrated in three-dimensional space, visualize the posture data, help to quickly adjust the device posture, and greatly improve the calibration efficiency.

[0037] In one embodiment, the second movable marker includes a front-back tilt marker and a left-right tilt marker; In response to the posture change of the device to be calibrated, controlling the second movable marker to perform an action on the calibration image acquisition interface as the posture changes, including: In response to the left-right tilt change of the device to be calibrated, controlling the left-right tilt marker to move within the left-right tilt area of ​​the calibration image acquisition interface; In response to the front-back tilt change of the device to be calibrated, the front-back tilt mark is controlled to move within the front-back tilt area of ​​the calibration image acquisition interface.

[0038] The movement of the left and right tilt indicator is used to indicate the left and right tilt degree of the device to be calibrated. The indicator can be a horizontal arrow, a slider, a scale bar, etc., without limitation. Similarly, the movement of the front and back tilt indicator is used to indicate the front and back tilt degree of the device to be calibrated. The indicator can be a vertical arrow, a slider, a scale bar, etc., without limitation.

[0039] Left-right tilt refers to the change in the device's rotation angle around its own Y-axis (longitudinal axis), for example, if the device is tilted left or right. Forward-backward tilt refers to the change in the device's rotation angle around its own X-axis (transverse axis), for example, if the device is tilted forward (pitch angle) or backward (elevation angle).

[0040] The left and right tilt area is used to limit the movement range of the left and right tilt marker, and can be, for example, a rectangular box in the left and right directions. The front and back tilt area is used to limit the movement range of the front and back tilt marker, and can be, for example, a rectangular box in the front and back directions. It is understood that when the device to be calibrated exceeds the tilt range, the marker stops or prompts an out-of-limit message.

[0041] Specifically, when the device to be calibrated is tilted to the left, the terminal device controls the left and right tilt indicators to move leftward within the left and right tilt area. When the device to be calibrated is tilted to the right, the terminal device controls the left and right tilt indicators to move rightward within the left and right tilt area. When the device to be calibrated is tilted forward, the terminal device controls the front and back tilt indicators to move upward within the front and back tilt area. When the device to be calibrated is tilted backward, the terminal device controls the front and back tilt indicators to move downward within the front and back tilt area.

[0042] In this embodiment, in response to the left and right tilt changes of the device to be calibrated, the left and right tilt markers are controlled to move within the left and right tilt areas. In response to the front and back tilt changes of the device to be calibrated, the front and back tilt markers are controlled to move within the front and back tilt areas. Through the separated tilt areas and tilt markers, the three-dimensional tilt changes of the device to be calibrated are converted into visual interface feedback, which helps to quickly adjust the device posture, thereby achieving efficient device calibration.

[0043] In one embodiment, the visual calibration method further includes: in response to a plane rotation change of the device to be calibrated, controlling the first movable marker to rotate on the calibration image acquisition interface along with the plane rotation change.

[0044] In-plane rotation refers to rotation of the device about an axis (e.g., the Z-axis) perpendicular to the plane (e.g., the horizontal plane) in which it resides, such as clockwise or counterclockwise rotation. The first movable marker can be a marker that can identify the direction of rotation, such as a missing-corner ring, an arrow, etc., but is not limited to these.

[0045] Specifically, when the device to be calibrated rotates on the horizontal plane, the terminal device controls the first movable marker to rotate on the calibration image acquisition interface along with the plane rotation. For example, if the device to be calibrated rotates clockwise on the horizontal plane, the terminal device controls the first movable marker to rotate clockwise on the calibration image acquisition interface.

[0046] In this embodiment, in response to the plane rotation change of the device to be calibrated, the first movable marker is controlled to rotate on the calibration image acquisition interface along with the plane rotation change, so that the posture of the device to be calibrated can be reflected on the visual interface.

[0047] In one embodiment, capturing the spatial position of a calibration object such as a calibration plate is crucial for calibration, so the spatial position of the calibration plate needs to be designed. Based on the coordinates of the calibration plate, the relative position of the calibration plate and the scanner is established through the factory parameters of the scanner, including the camera internal parameters and the binocular structure parameters. In this embodiment, N postures are preset, where N is set according to demand, such as 30, which includes height changes (characterizing the change in distance between the device and the calibration plate), forward tilt, backward tilt, left tilt and right tilt. Among them, 18 positions are height changes, and the remaining positions are evenly divided into the other four directions. When designing the position, the convenience of use by the operator and the continuity of the spatial position distribution are taken into consideration, and the positions are all changed in sequence. As Figure 3 FIG. 1 is a schematic diagram of a calibration position trend in an embodiment. Figure 3 The direction of increasing height (i.e. the direction of distance change between the device to be calibrated and the calibration plate), left tilt direction, right tilt direction, forward tilt direction and backward tilt direction are indicated in . Taking the device to be calibrated as a binocular camera as an example, Figure 4 Schematic diagram of the coordinate orientation of a binocular camera in one embodiment. Figure 4 The blue dot in the figure represents the calibration point in three-dimensional space. The two coordinates are the coordinates of the two cameras of the binocular camera, where the red line represents the X-axis, the green line represents the Y-axis, and the blue and black lines represent the Z-axis. Figure 5 A schematic diagram of positioning posture trend in another embodiment. Figure 5 The figure shows the pose diagrams of 30 positions. Figure 5 The yellow dots in the middle represent the calibration points. Similarly, the red line represents the X-axis, the green line represents the Y-axis, and the blue and black lines represent the Z-axis. Figure 6 The following is a schematic diagram of the target pose setting process in one embodiment. The terminal device inputs the coordinates of the calibration points on the calibration plate, the camera intrinsic parameters, and the binocular camera structure parameters. Based on the input data, a rotation matrix is ​​established from the camera coordinate system to the world coordinate system. The camera's 30 spatial poses (X, Y, Z, phi, w, k) are set in the camera coordinate system, where X, Y, and Z represent the spatial position, and phi, w, and k represent the spatial posture, which are the plane rotation angle, front-back tilt angle, and left-right tilt angle, respectively. The spatial pose in the camera coordinate system is converted to the spatial pose in the world coordinate system using the rotation matrix. The terminal device uses the spatial pose in the world coordinate system as the target pose.

[0048] After acquiring the calibration image of the current posture, the visual calibration method further includes: When the calibration image is not the last calibration image, update the target pose identifier; Returning to the step of controlling the first movable marker to perform an action on the calibration image acquisition interface as the position changes, and controlling the second movable marker to perform an action on the calibration image acquisition interface as the posture changes, in response to the change in the position of the device to be calibrated; When the calibration image is the last calibration image, it shows that the image acquisition is completed.

[0049] Specifically, since there are multiple target postures represented by the target posture identifier, after collecting a calibration image, the target posture identifier is updated, that is, the target posture is updated, the collection of the next calibration image is started, and the execution is returned to the step of responding to the posture change of the device to be calibrated, controlling the first movable identifier to perform an action in the image acquisition interface as the position changes, and controlling the second movable identifier to perform an action in the calibration image acquisition interface as the posture changes, and collecting the next calibration image. When the calibration image is the last calibration image, the image acquisition is completed. It can be understood that updating the target posture identifier includes updating the target posture, and can also include updating the position, size, orientation, etc. of the target posture identifier in the calibration image acquisition interface. The mark of image acquisition completion can be "image acquisition completed", a pop-up window, a prompt light, the serial number of the last calibration image, etc., but is not limited to this.

[0050] In this embodiment, when the calibration image is not the last calibration image, the target posture identifier is updated, and the process returns to execute the steps in response to the posture change of the device to be calibrated, controlling the first movable identifier to perform an action in the calibration image acquisition interface as the position changes, and controlling the second movable identifier to perform an action in the calibration image acquisition interface as the posture changes. When the swinging image is the last calibration image, the image acquisition is displayed as completed, and multiple calibration images can be visually calibrated according to the process, reducing the difficulty of calibration and improving the calibration efficiency.

[0051] In one embodiment, the visual calibration method further comprises: displaying a sequence number of the calibration image; When the calibration image is the last one, it shows that the image acquisition is complete, including: When the sequence number of the calibration image is the target sequence number, it indicates that the image acquisition is completed.

[0052] Specifically, the target sequence number matches the number of preset calibration poses. For example, if the number of preset calibration poses is 30, then the target sequence number is also 30. It is understood that since the target pose identifier represents at least two target poses, after acquiring the calibration image, displaying the sequence number of the calibration image can intuitively display the calibration progress. When the sequence number of the calibration image is the target sequence number, it indicates that the calibration image is the last image, indicating that image acquisition is complete.

[0053] In this embodiment, the serial number of the calibration image is displayed. When the serial number of the calibration image is the target serial number, it means that the calibration image is the last image, and the image acquisition is completed. This allows the user to know the number of calibrated poses, facilitates immediate adjustment of the pose, and improves calibration efficiency.

[0054] In one embodiment, Figure 7FIG. 1 is a flow chart of movable marker pose matching in one embodiment. The visual calibration method further includes the following steps: Step 702: Capture a reference image of the calibration point using the device to be calibrated.

[0055] Specifically, the device to be calibrated uses binocular vision, such as a binocular camera or binocular laser scanner. Each binocular vision device captures a pair of reference images. The terminal device is connected to the device to be calibrated, and the device collects reference images of the calibration points on the object in real time.

[0056] Step 704: extract feature points from the reference image to obtain the two-dimensional image coordinates of the calibration points.

[0057] The projection matrix of the device to be calibrated has been calibrated before the device leaves the factory.

[0058] Specifically, the terminal device performs feature extraction on the reference image using corner detection algorithms, feature detection algorithms, etc., extracts feature points in the reference image, and obtains the two-dimensional image coordinates of the calibration points. The two-dimensional image coordinates are pixel coordinates.

[0059] Step 706 : Determine the three-dimensional coordinates of the calibration points according to the two-dimensional image coordinates of the calibration points and the projection matrix of the device to be calibrated.

[0060] Specifically, the projection matrix can convert the 2D image coordinates captured by the device to be calibrated into 3D coordinates. The terminal device then performs a conversion based on the 2D image coordinates of the calibration point and the projection matrix to determine the 3D coordinates of the calibration point.

[0061] Step 708: Determine the current posture of the device to be calibrated based on the correspondence between the three-dimensional coordinates and the two-dimensional image coordinates.

[0062] Specifically, by knowing the correspondence between 3D spatial points and the 2D projections of calibration points in the image, the device's position and posture in space can be calculated. The terminal device can determine the current posture of the device to be calibrated based on the 3D coordinates and 2D image coordinates using quaternions, SVD decomposition, and other methods.

[0063] Step 710: When the current posture matches the target posture indicated by the target posture identifier, the first movable identifier and the second movable identifier both match the target posture identifier, and a calibration image of the current posture is collected by the device to be calibrated.

[0064] Specifically, when the error between the current pose and the target pose indicated by the target pose identifier is within a preset range, the current pose matches the target pose. Then, on the calibration image acquisition interface, the first and second movable markers are matched with the target pose identifier, and a calibration image of the current pose is automatically acquired by the device to be calibrated. The terminal device then returns to the step of extracting feature points from the reference image until the acquired calibration image is the last one, ending the calibration image acquisition process.

[0065] In this embodiment, a reference image of the calibration point is collected by the device to be calibrated, feature points in the reference image are extracted, and the two-dimensional image coordinates of the calibration point are obtained. The three-dimensional coordinates of the calibration point are determined according to the two-dimensional image coordinates of the calibration point and the projection matrix. The current posture of the device to be calibrated is determined according to the three-dimensional coordinates and the two-dimensional image coordinates. When the current posture matches the target posture represented by the target posture identifier, the first movable identifier and the second movable identifier on the interface match the target posture identifier, indicating that the device posture has reached the standard. By collecting the calibration image of the current posture of the device to be calibrated, accurate calibration of the device posture can be achieved without the use of other instruments, thereby ensuring the accuracy and consistency of subsequent operations.

[0066] In one embodiment, the current pose is represented by a rotation matrix and a translation matrix; Controlling the first movable marker to perform an action on the calibration image acquisition interface as the position changes, including: Determining a display position and a display size of the first movable marker on the calibration image acquisition interface according to the translation matrix; Controlling the second movable marker to perform actions on the calibration image acquisition interface as the posture changes, including: The execution action of the second movable marker on the calibration image acquisition interface is determined according to the rotation matrix.

[0067] Among them, the rotation matrix is ​​used to represent the posture of the device to be calibrated, and the translation matrix is ​​used to represent the position of the device to be calibrated.

[0068] Specifically, the terminal device converts the translation data of the device to be calibrated into the display position and size of the first movable marker on the calibration image acquisition interface based on the translation matrix and a certain conversion coefficient. Based on the rotation matrix, the terminal device can determine the display position of the second movable marker (e.g., a slider) on the calibration image acquisition interface. Alternatively, the rotation matrix can be used to determine the posture of the second movable marker (e.g., a 3D icon) on the calibration image acquisition interface.

[0069] In this embodiment, the display position and display size of the first movable marker in the calibration image acquisition interface are determined according to the translation matrix, and the execution action of the second movable marker in the calibration image acquisition interface is determined according to the rotation matrix. This can achieve dynamic visual feedback on the device posture, help users intuitively adjust to the target posture, and thus improve the accuracy and efficiency of device calibration.

[0070] In one embodiment, Figure 5 The spatial distribution of 30 positions is shown, with the first 18 positions varying in height, and the remaining positions evenly distributed across the other four directions. The positions were designed with user convenience and continuity in the spatial position distribution in mind. Figure 8(a) is a schematic diagram of the calibration image acquisition interface in one embodiment. The left and right sides of Figure 8(a) are both prompt areas, with the center area being the image acquisition area. The prompt area indicates the number of positions to be calibrated (target poses to be calibrated), 30, and the number of calibrated positions (poses), 0, for a total of 30. The scanning process is also explained: Scan points 1 to 18 at a vertical height. Position the scanner as shown to scan points 1 to 18, aligning the green coordinate system box with the red coordinate system box. The calibration steps are displayed in the right prompt area. ① Remove the calibration plate and lay it flat, then place the handheld device on top of the plate. ② Follow the diagram to align the green coordinate system box with the red coordinate system box. ③ During the calibration process, a total of 30 points are calibrated: points 1-18 for vertical height (distance change); points 19-21 for forward tilt; points 22-24 for backward tilt; points 25-27 for left tilt; and points 28-30 for right tilt. The green notched ring in the image acquisition area is the first movable marker, while the red and green small circles are the second movable markers. The red circle indicates forward tilt, while the green circle indicates left and right tilt. The size of the green notched ring represents changes in height, while the orientation of the notch indicates changes in planar rotation. The red and green circles sliding between the crosses represent changes in forward and backward tilt and left and right tilt. The icon design brings all posture changes together and transforms them into a two-dimensional representation, making the three-dimensional spatial posture more vivid and easier to use.

[0071] Figure 8 (b) is a schematic diagram of the target posture marker and tilt area in one embodiment. It can be understood that the target posture marker in Figure 8 (b) includes a central black circle and a yellow notched ring, and the tilt area is a "cross" area, where the horizontal line is the left and right tilt area, and the vertical line is the front and back tilt area. The central black circle is used to match the second movable marker, and the yellow notched ring is used to match the first movable marker. When the green notched ring coincides with the yellow notched ring, it means that the position and plane rotation posture of the scanner match the target posture. When both the red and green small circles move to the central black circle, it means that the left and right tilt angles and the front and back tilt angles of the scanner match the target posture.

[0072] Figure 9 This is a diagram illustrating the interface for matching markers in the calibration image acquisition interface in one embodiment. When the green notched circle overlaps with the yellow notched circle, the overlapping portion changes color to orange. When both the red and green small circles overlap with the central black circle, the overlapping portion changes color to orange, indicating that the current pose acquisition is complete. Figure 9 The ⑥ in the text box indicates the sequence number of the calibration image, indicating that a calibration image has been acquired for the current pose. Currently, there are 25 positions to be calibrated, 5 positions already calibrated, and the sixth position currently being calibrated. To terminate calibration at this point, click the End Calibration control to end the acquisition of the calibration image.

[0073] Then, taking the above interface diagram as an example, Figure 10 The figure shows a schematic diagram of the marker matching process in one embodiment, which includes the following steps: Start → Align the size of the notched ring (height alignment) → Align the notches of the notched ring (align the plane rotation angle) → Align the front and back tilted small circles with the center black circle → Align the left and right tilted small circles with the center black circle → End. It should be understood that the above process is only a suggestion and can be performed in any order, as long as the notched ring and two small circles are aligned with the corresponding markers.

[0074] During the position alignment process, the marker points will be dynamically identified in real time. The scanner captures images at a rate of 40 frames, and then performs image processing to extract image features. The two-dimensional image coordinates of the ellipse are obtained through image features, and the three-dimensional coordinates of the current marker point are calculated based on the projection matrix. The two-dimensional ellipse is matched one-to-one with the calibration plate according to the three-dimensional coordinates, and the rotation and translation matrix is ​​calculated to display the identified marker points in real time. The current device posture can be obtained through the rotation and translation matrix, and this posture can be used to determine whether it is consistent with the set target position range. If it meets the requirements, the acquisition conditions are met. The specific process diagram is as follows Figure 7 , which will not be described here. After completing the image acquisition process and processing the image, the image coordinates and calibration plate coordinates are input for adjustment and calibration. Adjustment is used to iteratively calculate the relative position of the two cameras and correct for errors caused by vibration.

[0075] In this embodiment, the target posture identifier is displayed in the calibration image acquisition interface. In response to the posture change of the device to be calibrated, the first movable identifier is controlled to perform an action as the position changes, and the second movable identifier is controlled to perform an action as the posture changes. The posture of the device to be calibrated is decomposed into two parts, position and posture, which are visualized separately. When the first movable identifier and the second movable identifier both match the target posture identifier, a calibration image of the current posture is collected for calibrating the device to be calibrated. The abstract three-dimensional posture is converted into an intuitive and easy-to-understand two-dimensional image interface, so that the user can perform posture matching in steps and parts, and can perform calibration in the same interface, which is convenient for operation and has fun and interactivity, greatly reducing the difficulty of matching the target posture, thereby significantly improving the calibration efficiency and posture accuracy, and improving the accuracy of calibration.

[0076] In one embodiment, a visual calibration method includes: Step (a1) displays a calibration image acquisition interface, which includes a target posture identifier.

[0077] Step (a2): in response to the horizontal translation change of the device to be calibrated, controlling the first movable marker to move on the calibration image acquisition interface along with the horizontal translation change; the display position of the first movable marker is determined according to the translation matrix of the device to be calibrated.

[0078] Step (a3): in response to the distance change of the device to be calibrated, controlling the first movable marker to scale on the calibration image acquisition interface as the distance changes; the display size of the first movable marker is determined according to the translation matrix of the device to be calibrated.

[0079] Step (a4) controls the left and right tilt markers to move within the left and right tilt areas of the calibration image acquisition interface in response to the left and right tilt changes of the device to be calibrated; the display positions of the left and right tilt markers are determined according to the rotation matrix of the device to be calibrated.

[0080] Step (a5) controls the front and rear tilt marker to move within the front and rear tilt area of ​​the calibration image acquisition interface in response to the front and rear tilt change of the device to be calibrated; the display position of the front and rear tilt marker is determined according to the rotation matrix of the device to be calibrated.

[0081] Step (a6): in response to the plane rotation change of the device to be calibrated, controlling the first movable marker to rotate on the calibration image acquisition interface along with the plane rotation change.

[0082] Step (a7): collecting reference images of the calibration points through the device to be calibrated.

[0083] Step (a8) extracts feature points from the reference image and obtains the two-dimensional image coordinates of the calibration points.

[0084] Step (a9) determines the three-dimensional coordinates of the calibration points according to the two-dimensional image coordinates of the calibration points and the projection matrix of the device to be calibrated.

[0085] Step (a10) determines the current posture of the device to be calibrated based on the correspondence between the three-dimensional coordinates and the two-dimensional image coordinates.

[0086] In step (a11), when the current posture matches the target posture indicated by the target posture identifier, the first movable identifier and the second movable identifier both match the target posture identifier, and a calibration image of the current posture is collected by the device to be calibrated. The calibration image is used to calibrate the device to be calibrated.

[0087] Step (a12), display the serial number of the calibration image.

[0088] In step (a13), when the calibration image is not the last calibration image, the target pose identifier is updated.

[0089] Step (a14) returns to executing the step of controlling the first movable marker to perform an action on the calibration image acquisition interface as the position changes in response to the posture change of the device to be calibrated, and controlling the second movable marker to perform an action on the calibration image acquisition interface as the posture changes.

[0090] In step (a15), when the sequence number of the calibration image is the target sequence number, it is displayed that the image acquisition is completed.

[0091] In this embodiment, the target posture identifier is displayed in the calibration image acquisition interface. In response to the posture change of the device to be calibrated, the first movable identifier is controlled to perform an action as the position changes, and the second movable identifier is controlled to perform an action as the posture changes. The posture of the device to be calibrated is decomposed into two parts, position and posture, which are visualized separately; when the first movable identifier and the second movable identifier both match the target posture identifier, the calibration image of the current posture is collected for calibrating the device to be calibrated, and the abstract three-dimensional posture is converted into an intuitive and easy-to-understand two-dimensional image interface, so that the user can perform posture matching in steps and parts, and can perform calibration in the same interface, thereby significantly improving the calibration efficiency and posture accuracy, thereby improving the accuracy of calibration.

[0092] It should be understood that although the above Figure 2 、 6 The steps in the flowcharts of , 7 and 10 are shown in sequence as indicated by arrows, and the steps in steps (a1) to (a15) are shown in sequence as indicated by numbers, but these steps are not necessarily performed in the order indicated by arrows or numbers. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be performed in other orders. Moreover, Figure 2 、 6At least part of the steps in , 7 and 10 may include multiple steps or multiple stages. These steps or stages do not necessarily have to be performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0093] In one embodiment, a visual calibration device is provided. The device may be a software module or a hardware module, or a combination of both, and may be part of a computer device. The device specifically includes: a display module, a control module, and an acquisition module, wherein: A display module is used to display a calibration image acquisition interface, which includes a target posture identifier; A control module, configured to control the first movable marker to perform an action on the calibration image acquisition interface as the position changes, and to control the second movable marker to perform an action on the calibration image acquisition interface as the posture changes, in response to a posture change of the device to be calibrated; the posture change includes at least one of a position change and a posture change; The acquisition module is used to acquire a calibration image of the current posture through the device to be calibrated when the first movable marker and the second movable marker both match the target posture marker; the calibration image is used to calibrate the device to be calibrated.

[0094] In one embodiment, the control module is configured to: In response to a horizontal translation change of the device to be calibrated, controlling the first movable marker to move on the calibration image acquisition interface along with the horizontal translation change; In response to a change in the distance of the device to be calibrated, the first movable marker is controlled to zoom in and out on the calibration image acquisition interface as the distance changes.

[0095] In one embodiment, the second movable marker includes a front-back tilt marker and a left-right tilt marker; Control module for: In response to the left-right tilt change of the device to be calibrated, controlling the left-right tilt marker to move within the left-right tilt area of ​​the calibration image acquisition interface; In response to the front-back tilt change of the device to be calibrated, the front-back tilt mark is controlled to move within the front-back tilt area of ​​the calibration image acquisition interface.

[0096] In one embodiment, the control module is further configured to: In response to the plane rotation change of the device to be calibrated, the first movable marker is controlled to rotate on the calibration image acquisition interface along with the plane rotation change.

[0097] In one embodiment, the acquisition module is further configured to update the target pose identifier when the calibration image is not the last calibration image; The control module is further configured to return and execute in response to the posture change of the device to be calibrated, control the first movable marker to perform an action on the calibration image acquisition interface as the position changes, and control the second movable marker to perform an action on the calibration image acquisition interface as the posture changes; The acquisition module is also used to display that image acquisition is completed when the calibration image is the last calibration image.

[0098] In one embodiment, the display module is further configured to: Display the serial number of the calibration image; When the sequence number of the calibration image is the target sequence number, it indicates that the image acquisition is completed.

[0099] In one embodiment, the acquisition module is further configured to: Collect reference images of calibration points through the device to be calibrated; The control module is also used to: Extract feature points from the reference image and obtain the two-dimensional image coordinates of the calibration points; Determine the three-dimensional coordinates of the calibration points according to the two-dimensional image coordinates of the calibration points and the projection matrix of the device to be calibrated; Determine the current pose of the device to be calibrated based on the correspondence between the three-dimensional coordinates and the two-dimensional image coordinates; The acquisition module is also used to: When the current posture matches the target posture indicated by the target posture identifier, both the first movable identifier and the second movable identifier match the target posture identifier, and a calibration image of the current posture is collected by the device to be calibrated.

[0100] In one embodiment, the current pose is represented by a rotation matrix and a translation matrix; Control module for: Determining a display position and a display size of the first movable marker on the calibration image acquisition interface according to the translation matrix; The execution action of the second movable marker on the calibration image acquisition interface is determined according to the rotation matrix.

[0101] For the specific definition of the visual calibration device, please refer to the definition of the visual calibration method above, which will not be repeated here. The various modules in the above-mentioned visual calibration device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0102] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 11As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be achieved via Wi-Fi, mobile cellular networks, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a visual calibration method. The display unit of the computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0103] Those skilled in the art will understand that Figure 11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0104] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.

[0105] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method embodiments are implemented.

[0106] In one embodiment, a computer program product is provided, including a computer program, which implements the steps of the above method embodiments when executed by a processor.

[0107] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0108] The above description is only a preferred embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A visual calibration method, characterized in that: The method comprises: Displaying a calibration image acquisition interface, wherein the calibration image acquisition interface includes a target posture identifier; In response to a change in the posture of the device to be calibrated, controlling the first movable marker to perform an action on the calibration image acquisition interface as the position changes, and controlling the second movable marker to perform an action on the calibration image acquisition interface as the posture changes; the posture change includes at least one of the position change and the posture change; When both the first movable marker and the second movable marker match the target posture marker, a calibration image of the current posture is collected by the device to be calibrated; the calibration image is used to calibrate the device to be calibrated.

2. The method according to claim 1, characterized in that In response to the change in the posture of the device to be calibrated, controlling the first movable marker to perform an action on the calibration image acquisition interface as the position changes, includes: In response to a horizontal translation change of the device to be calibrated, controlling the first movable marker to move on the calibration image acquisition interface along with the horizontal translation change; In response to a distance change of the device to be calibrated, the first movable marker is controlled to zoom in and out on the calibration image acquisition interface as the distance changes.

3. The method according to claim 1, characterized in that The second movable mark includes a front-back tilt mark and a left-right tilt mark; In response to a change in the posture of the device to be calibrated, controlling the second movable marker to perform an action on the calibration image acquisition interface as the posture changes, including: In response to a left-right tilt change of the device to be calibrated, controlling the left-right tilt marker to move within a left-right tilt area of ​​the calibration image acquisition interface; In response to the front-back tilt change of the device to be calibrated, the front-back tilt mark is controlled to move within the front-back tilt area of ​​the calibration image acquisition interface.

4. The method according to claim 3, characterized in that The method further comprises: In response to a plane rotation change of the device to be calibrated, the first movable marker is controlled to rotate on the calibration image acquisition interface along with the plane rotation change.

5. The method according to claim 1, wherein After collecting the calibration image of the current posture by the device to be calibrated, the method further includes: When the calibration image is not the last calibration image, updating the target pose identifier; Returning to the step of controlling, in response to the change in the posture of the device to be calibrated, the first movable marker to perform an action on the calibration image acquisition interface as the position changes, and controlling the second movable marker to perform an action on the calibration image acquisition interface as the posture changes; When the calibration image is the last calibration image, it is displayed that image acquisition is completed.

6. The method according to claim 5, characterized in that The method further comprises: Display the serial number of the calibration image; When the calibration image is the last calibration image, displaying that image acquisition is complete includes: When the sequence number of the calibration image is the target sequence number, it is displayed that the image acquisition is completed.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Collecting a reference image of the calibration point by the device to be calibrated; Extracting feature points from the reference image to obtain the two-dimensional image coordinates of the calibration points; Determine the three-dimensional coordinates of the calibration point according to the two-dimensional image coordinates of the calibration point and the projection matrix of the device to be calibrated; Determining the current posture of the device to be calibrated according to the correspondence between the three-dimensional coordinates and the two-dimensional image coordinates; When both the first movable marker and the second movable marker match the target posture marker, collecting a calibration image of the current posture by the device to be calibrated includes: When the current posture matches the target posture indicated by the target posture identifier, both the first movable identifier and the second movable identifier match the target posture identifier, and a calibration image of the current posture is collected by the device to be calibrated.

8. The method according to claim 7, characterized in that The current posture is represented by a rotation matrix and a translation matrix; The controlling the first movable marker to perform an action on the calibration image acquisition interface as the position changes includes: determining a display position and a display size of the first movable marker on the calibration image acquisition interface according to the translation matrix; The controlling the second movable marker to perform an action on the calibration image acquisition interface as the posture changes includes: An execution action of the second movable marker on the calibration image acquisition interface is determined according to the rotation matrix.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images