Visual tracking tool quality detection method and controller

Abstract

The invention relates to the technical field of visual tracking tools, and discloses a visual tracking tool quality detection method and a controller, a visual tracking tool comprises a plurality of positioning identifiers, and the method comprises the following steps: determining a test position of the visual tracking tool in a camera working space; the visual tracking tool is rotated at the test position, tool information of the visual tracking tool is calculated according to an imaging image of the visual tracking tool collected by the camera in the rotation process, and tool information of the visual tracking tool at multiple angles is obtained; whether the visual tracking tool is qualified or not is judged according to the tool information at all the angles, and when it is judged that the visual tracking tool is unqualified, the problem positioning mark of the unqualified visual tracking tool is determined. The method can quickly, accurately and efficiently evaluate whether the precision of the visual tracking tool is qualified or not, and quickly and efficiently find out the problem positioning identifier of the unqualified visual tracking tool.

Description

Technical Field

[0001] The present invention relates to the technical field of visual tracking tools, and in particular to a quality detection method and a controller for visual tracking tools. Background Art

[0002] Tracer tools may experience a decrease in accuracy after installation and disinfection, or after repeated use. Evaluating the quality of tracer tools and identifying the problems with poor-quality tracer tools are issues that need to be addressed by those skilled in the art. Summary of the Invention

[0003] The present invention aims to at least partially address one of the technical problems in the related art. To this end, one object of the present invention is to provide a method for inspecting the quality of visual tracking tools that can quickly, accurately, and efficiently evaluate the accuracy of visual tracking tools and quickly and efficiently identify problems in unqualified visual tracking tools.

[0004] The second object of the present invention is to provide a controller.

[0005] To achieve the above-mentioned objectives, a first aspect of an embodiment of the present invention proposes a method for quality inspection of a visual tracking tool, wherein the visual tracking tool includes multiple positioning identifiers, and the method includes: determining a test position of the visual tracking tool in a camera working space; rotating the visual tracking tool at the test position, and calculating tool information of the visual tracking tool based on the imaging image of the visual tracking tool captured by the camera during the rotation process, to obtain tool information of the visual tracking tool at multiple angles; determining whether the visual tracking tool is qualified based on the tool information at each of the angles, and determining the problem positioning identifier of the unqualified visual tracking tool when the visual tracking tool is determined to be unqualified.

[0006] According to an embodiment of the present invention, a visual tracking tool quality inspection method places the visual tracking tool at a test position in a camera workspace and rotates the tool. Tool information for the visual tracking tool at multiple angles is calculated based on the images captured by the camera during the rotation. The tool accuracy of the visual tracking tool is then evaluated based on this tool information at each angle. If a visual tracking tool is deemed unqualified, the problem location of the unqualified tool can be quickly and efficiently identified.

[0007] In addition, the visual tracking tool quality detection method proposed in the above embodiment of the present invention may also have the following additional technical features: According to one embodiment of the present invention, the test position includes a horizontal field of view limit position, a vertical field of view limit position and a middle position, and determining the test position of the visual tracking tool in the camera workspace includes: placing the visual tracking tool at the target test position, collecting the current position information of the visual tracking tool, and generating a current position identifier based on the current position information; detecting whether the current position identifier is consistent with a pre-established target test position mark, and generating a prompt message if they are consistent.

[0008] According to one embodiment of the present invention, rotating the visual tracking tool at the test position includes: controlling the yaw rotation of the visual tracking tool within a preset yaw angle range at the test position; and controlling the pitch rotation of the visual tracking tool within a preset pitch angle range at the test position.

[0009] According to one embodiment of the present invention, the number of the positioning marks is at least 3, the tool information includes a tool error value and the point coordinates of each positioning mark, and the tool information of the visual tracking tool is calculated based on the imaging image of the visual tracking tool captured by the camera during the rotation process, including: identifying the point coordinates of each positioning mark of the visual tracking tool in the imaging image; calculating a plane normal vector based on the point coordinates of at least 3 positioning marks, and converting the plane normal vector in the opposite direction when the z-direction component of the plane normal vector is negative to obtain a tool normal vector; calculating the yaw angle or pitch angle of the visual tracking tool in the imaging image based on the tool normal vector; determining a target image based on a preset angle interval and the yaw angle or pitch angle, and calculating the tool error value of the visual tracking tool at the corresponding yaw angle or the tool error value at the corresponding pitch angle based on the target image to obtain tool error values ​​at multiple yaw angles or tool error values ​​at multiple pitch angles.

[0010] According to one embodiment of the present invention, determining whether the visual tracking tool is qualified based on the tool information at each of the angles includes: detecting whether the tool information at each of the angles is not missing, and whether the tool error value at each of the angles is less than a preset error threshold; if the tool information at each of the angles is not missing, and the tool error value at each of the angles is less than the preset error threshold, determining that the visual tracking tool is qualified; otherwise, determining that the visual tracking tool is unqualified.

[0011] According to one embodiment of the present invention, the method further includes: translating the visual tracking tool along the z-axis direction of the camera within the depth range of the camera working space, and keeping the tool normal vector consistent with the z-axis direction of the camera; collecting a preset number of imaging images during the translation process, identifying the point coordinates of each positioning mark of the visual tracking tool in each of the imaging images, and calculating the tool error value of the visual tracking tool at multiple depths in each of the imaging images; determining whether the visual tracking tool is qualified based on the tool information at each of the depths, and determining the problem positioning mark of the unqualified visual tracking tool when the visual tracking tool is determined to be unqualified.

[0012] According to one embodiment of the present invention, determining whether the visual tracking tool is qualified based on the tool information at each of the depths includes: detecting whether the tool information at each of the depths is not missing, and whether the tool error values ​​at each of the depths are less than a preset error threshold; if the tool information at each of the depths is not missing, and the tool error values ​​at each of the depths are less than the preset error threshold, determining that the visual tracking tool is qualified; otherwise, determining that the visual tracking tool is unqualified.

[0013] According to one embodiment of the present invention, the camera is a binocular camera, the imaged images include a first image and a second image, and determining the problem positioning marker of the unqualified visual tracking tool includes: calculating the actual point coordinates of each positioning marker of the visual tracking tool based on the first image and the second image at the angle or depth corresponding to the tool error value greater than a preset error threshold; calculating the actual distance value between each positioning marker and other positioning markers based on the actual point coordinates of each positioning marker; calculating the calibrated distance value between each positioning marker and other positioning markers based on the calibration point coordinates of each positioning marker; calculating the number of times the absolute value of the difference between the actual distance value and the calibrated distance value between each positioning marker and other positioning markers is greater than a preset error threshold, to obtain the number of distance deviations of each positioning marker; sorting the positioning markers in descending order according to the size of the distance deviation number; and judging, based on the sorted positioning markers, whether the distance deviation number corresponding to the positioning marker is greater than or equal to Cnt+1. If it is greater, Cnt is incremented by 1, and the positioning marker is recorded as the problem positioning marker, wherein the initial value of Cnt is 0.

[0014] According to one embodiment of the present invention, the method of determining the problem location mark of an unqualified visual tracking tool includes: obtaining a first image and a second image at an angle or depth corresponding to the missing tool information; respectively obtaining the center of each location mark of the visual tracking tool in the first image and the second image; for the center of each location mark in the first image and the second image, respectively calculating the number of pixels of multiple straight lines passing through the center of the location mark that meet the grayscale threshold, and obtaining the number of pixels corresponding to each straight line passing through the center of the location mark; if the pixel difference between the number of pixels corresponding to any two straight lines is greater than a preset pixel number difference threshold, the location mark is identified as the problem location mark.

[0015] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes a controller, including a memory and a processor, wherein a computer program is stored on the memory. When the computer program is executed by the processor, the visual tracking tool quality detection method proposed in the first embodiment of the present invention is implemented.

[0016] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flow chart of a method for quality inspection of a visual tracking tool according to an embodiment of the present invention; Figure 2 is a schematic diagram of the relative positions of a camera and a visual tracking tool according to a specific embodiment of the present invention; FIG3( a ) is a three-dimensional diagram of a camera working space according to a specific embodiment of the present invention; FIG3( b ) is a top view of a camera working space according to a specific embodiment of the present invention; FIG3( c ) is a left side view of the camera working space according to a specific embodiment of the present invention; FIG4( a ) is a schematic diagram of the placement of a visual tracking tool according to a specific embodiment of the present invention; FIG4( b ) is a schematic diagram of the placement of a visual tracking tool according to another specific embodiment of the present invention; Figure 5 is a flow chart of tool information of a computational vision tracking tool according to an embodiment of the present invention; Figure 6 is a scatter plot of pitch angle-tool error according to a specific embodiment of the present invention; Figure 7 is a flow chart for determining whether a visual tracking tool is qualified according to an embodiment of the present invention; Figure 8 is a flowchart for determining whether a visual tracking tool is qualified according to another embodiment of the present invention; Figure 9 is a scatter plot of depth-tool error according to one embodiment of the present invention; Figure 10 is a flow chart for determining a problem location identification of a non-conforming visual tracking tool according to an embodiment of the present invention; Figure 11 is a flow chart for determining a problem location identification of an unqualified visual tracking tool according to another embodiment of the present invention; Figure 12 Schematic diagram of four straight lines passing through the center of a positioning mark according to a specific embodiment of the present invention; Figure 13 4 is a structural block diagram of a controller according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0019] The visual tracking tool quality detection method and controller according to the embodiment of the present invention will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0020] The visual tracking tool in the embodiment of the present invention includes a plurality of positioning marks.

[0021] The visual tracking tool in the embodiment of the present invention may be a tracer tool, wherein the positioning mark on the visual tracking tool may be a sphere, a circular plane, etc.

[0022] For example, when the positioning marker is a sphere, the surface of the sphere is covered with a high-reflectivity coating (such as aluminum or silver) to reflect light.

[0023] It's important to note that repeated sterilization and use of the sphere can damage the coating on its surface. Damage or loss of the coating can affect the accuracy of the visual tracking tool. Furthermore, improper installation, negligence, or other errors when mounting the sphere on the tracking tool's bracket can also affect the tool's accuracy.

[0024] Visual tracking tools are one of the core tools in medical imaging technology. Therefore, it is crucial to evaluate the accuracy of visual tracking tools before using them.

[0025] Figure 1 FIG. 1 is a flow chart of a method for detecting quality of a visual tracking tool according to an embodiment of the present invention. Figure 1 As shown, the visual tracking tool quality inspection method may include: S101, determining a test position of a visual tracking tool in a camera workspace; S102, rotating the visual tracking tool at a test position, and calculating tool information of the visual tracking tool based on imaging images of the visual tracking tool captured by a camera during the rotation process, to obtain tool information of the visual tracking tool at multiple angles; S103 , determining whether the visual tracking tool is qualified according to the tool information at each angle, and determining a problem location mark of the unqualified visual tracking tool when the visual tracking tool is determined to be unqualified.

[0026] The embodiment of the present invention can be used to evaluate the accuracy of a visual tracking tool after repeated use, determine whether the visual tracking tool is qualified, and identify problem locations of unqualified visual tracking tools.

[0027] Specifically, a determination is made as to whether the visual tracking tool is in a test position within the camera workspace. Upon determining that the visual tracking tool is in the test position within the camera workspace, the visual tracking tool is rotated. During the rotation of the visual tracking tool, the camera captures images of the visual tracking tool in real time. Tool information of the visual tracking tool is calculated based on the images captured by the camera, thereby obtaining tool information of the visual tracking tool at multiple angles. In embodiments of the present invention, the tool information includes tool error values ​​of positioning markers and the point coordinates of each positioning marker.

[0028] The accuracy of the visual tracking tool is determined to be acceptable based on whether the tool information at each angle is missing and whether the tool error value at each angle is less than a preset error threshold. If the accuracy of the visual tracking tool is determined to be unacceptable, a problem location marker of the unacceptable visual tracking tool is determined based on the image at the angle corresponding to the missing tool information and / or the tool error value being greater than or equal to the preset error threshold, and the problem location marker is reinstalled or replaced.

[0029] It should be noted that after reinstalling or replacing the problem positioning mark, it is necessary to re-test whether the visual tracking tool is qualified.

[0030] The visual tracking tool quality inspection method of an embodiment of the present invention places the visual tracking tool at a test position in a camera workspace and rotates the tool. Tool information for the visual tracking tool at multiple angles is calculated based on the images captured by the camera during the rotation. The tool accuracy of the visual tracking tool is then evaluated based on this information at each angle. If the visual tracking tool is deemed unqualified, the corresponding images can be used to quickly and efficiently identify the problem location of the unqualified visual tracking tool.

[0031] In one embodiment of the present invention, the test position may include a horizontal field of view limit position, a vertical field of view limit position, and a center position. Determining the test position of the visual tracking tool in the camera workspace may include: Placing the visual tracking tool at the target test position, collecting current position information of the visual tracking tool, and generating a current position identifier based on the current position information; Check whether the current position identifier is consistent with the pre-established target test position mark, and generate a prompt message if they are consistent.

[0032] The test positions in embodiments of the present invention may include three extreme positions of the camera workspace, namely, the horizontal field of view extreme position, the vertical field of view extreme position, and the center position of the camera workspace. The three test positions may be determined by the camera's computing system. After determining the three test positions, markers corresponding to the three test positions may be established. The established test position markers may be displayed within the camera workspace on a display screen.

[0033] When judging whether the visual tracking tool is at the horizontal field of view limit position of the camera working space, the horizontal field of view limit position of the camera working space is recorded as the target test position. The staff places the visual tracking tool according to the target test position. The computing system collects the current position information of the visual tracking tool, generates a current position identifier based on the current position information of the visual tracking tool, and detects whether the current position identifier is consistent with the pre-established target test position mark. If the current position identifier is inconsistent with the pre-established target test position mark, the visual tracking tool is moved and the above detection process is performed until the current position identifier is consistent with the pre-established target test position mark. When the current position identifier is consistent with the pre-established target test position mark, a sound prompt or a visual prompt can be used to prompt that the visual tracking tool has reached the preset test position.

[0034] Based on the above process, it can be determined whether the visual tracking tool is at the vertical field of view limit position and the middle position of the camera workspace.

[0035] Figure 2 A schematic diagram of the relative positions of the visual tracking tool and the camera is shown in FIG. Figure 2 The coordinate system shown in satisfies the right-hand coordinate system. Figure 2 The camera locator is placed horizontally, with the horizontal direction being the Y direction, the vertical direction being the X direction, and the working depth direction being the Z direction. Figure 2 The visual tracking tool in is configured with four spheres.

[0036] In the embodiments of the present invention, the camera workspace is determined based on the camera's capture space, which is determined by the camera's own parameters. Software can be used to display the captured image on a display screen in real time, as well as to create a three-dimensional workspace based on the camera's position. The three-dimensional, top, and left views of the camera workspace are shown in Figures 3(a), 3(b), and 3(c), respectively.

[0037] The following example illustrates how to determine the horizontal field of view limit of the camera working space: Based on the top view of the camera workspace shown in Figure 4(a), we tested whether the slope of the camera's horizontal field of view changes as the working distance between the visual tracking tool and the camera increases. The slope line is determined by the camera center point position and the position of the visual tracking tool. It should be noted that the slope of the camera's horizontal field of view in Figure 4(a) changes as the working distance between the visual tracking tool and the camera increases.

[0038] If the slope changes, it is necessary to follow the angle limit position (Figure 4 (a) The visual tracking tool should be placed in the black area in Figure 4 (a). The black area is the horizontal limit position area. In the horizontal limit position area, select a position at the same height as the camera in the vertical direction (in the horizontal limit position area). Figure 2 The position where the X direction value is 0 in the coordinate system defined in is the limit position of the horizontal field of view.

[0039] If the slope does not change, that is, if the 、 、 Collinearity: In the horizontal limit direction, you can select any position with the same depth as the camera at the same height as the vertical direction.

[0040] The following example illustrates how to determine the vertical field of view limit of a camera's working space: Based on the left view of the camera workspace shown in Figure 4(b), the detection camera's vertical field of view has a slope change as the working distance between the visual tracking tool and the camera increases. It should be noted that the camera's horizontal field of view in Figure 4(b) does not have a slope change as the working distance between the visual tracking tool and the camera increases.

[0041] If the slope changes, the angle limit position area is recorded as the vertical limit position area according to the above method, and the horizontal direction and the camera facing position (in the vertical limit position area) are selected. Figure 2 The position where the Y direction value is 0 in the defined coordinate system is the vertical field of view limit position.

[0042] If the slope does not change, that is, 、 、 If they are collinear, you can choose any position with a depth corresponding to the horizontal direction and the camera facing position in the vertical limit direction of 4 (b).

[0043] The following example illustrates how to determine the exact center position of the camera working space: The center of the camera working space is Figure 2 In the defined coordinate system, any position corresponding to the depth can be selected where the X and Y direction values ​​are both 0.

[0044] In the embodiment of the present invention, a visual tracking tool is placed at the horizontal visual field limit position, the vertical visual field limit position and the middle position of the above-mentioned camera working space respectively, and the visual tracking tool can be rotated manually or based on an automatic platform in the following manner.

[0045] In one embodiment of the present invention, rotating the vision tracking tool in a test position may include: Control the visual tracking tool's yaw rotation within a preset yaw angle range at the test position; Control the pitch rotation of the visual tracking tool at the test position to preset the pitch angle range.

[0046] It should be noted that the preset yaw angle range and the preset pitch angle range can be determined according to the application scenario of the visual tracking tool.

[0047] Specifically, when using an automatic platform to rotate the visual tracking tool, the visual tracking tool can be fixedly mounted on the automatic platform, and the automatic platform can be controlled to drive the visual tracking tool to yaw and rotate within a preset yaw angle range, and to pitch and rotate within a preset pitch angle range.

[0048] For example, if the preset yaw angle range is ±50 degrees, then at the test location, the automated platform must be controlled to rotate the visual tracking tool in yaw from -50 degrees to +50 degrees. This rotation must be continuous and slow to allow the camera to capture images of the visual tracking tool at a wider range of yaw angles. The same principle applies to pitch rotation, which will not be further explained here.

[0049] In one embodiment of the present invention, the number of positioning marks is at least 3, and the tool information includes the tool error value and the point coordinates of each positioning mark, such as Figure 5 As shown, during the rotation process, tool information of the visual tracking tool is calculated based on the imaging image of the visual tracking tool captured by the camera, which may include: S201, identifying the point coordinates of each positioning mark of the visual tracking tool in the imaging image; S202, calculating a plane normal vector based on the point coordinates of at least three positioning marks, and converting the plane normal vector in the opposite direction when the z-direction component of the plane normal vector is negative, to obtain a tool normal vector; S203, calculating the yaw angle or pitch angle of the visual tracking tool in the imaging image according to the tool normal vector; S204, determining a target image according to a preset angle interval and a yaw angle or a pitch angle, and calculating a tool error value of the visual tracking tool at a corresponding yaw angle or a tool error value at a corresponding pitch angle according to the target image, to obtain tool error values ​​at multiple yaw angles or tool error values ​​at multiple pitch angles.

[0050] An embodiment of the present invention improves the accuracy of determining whether a visual tracking tool is qualified and obtains relatively uniform tool error values ​​at different yaw and pitch angles. Specifically, the yaw angle or pitch angle of the visual tracking tool is identified in an image captured by a camera, a target yaw angle or target pitch angle to be selected is determined from the image according to a preset angle interval, the captured image corresponding to the target yaw angle or target pitch angle is used as the target image, and the tool error values ​​of the visual tracking tool at different yaw and pitch angles are calculated based on the target image.

[0051] Specifically, for each collected imaging image, the point coordinates of each positioning mark of the visual tracking tool in the imaging image are identified to obtain the point coordinates of each positioning mark 、 、……、 , where n represents the number of positioning identifiers, n≥3.

[0052] If the visual tracking tool has 3 spheres, the plane normal vector is calculated based on the point coordinates of the 3 spheres. If the visual tracking tool has 4 spheres, the plane normal vector is calculated based on the point coordinates of the 4 spheres. If the visual tracking tool has 5 or more spheres, any 4 spheres are selected as a group to calculate the plane normal vector.

[0053] As a specific example, the visual tracking tool has three spheres, and the point coordinates of the three spheres are 、 、 . Plane normal vector ,in, , . That is, the plane normal vector for and The cross product vector of .

[0054] As another specific example, if the visual tracking tool has 4 or more spheres, and the point coordinates of the selected 4 spheres are 、 、 、 . Plane normal vector ,in, , .

[0055] It should be noted that if the plane normal vector The Z direction component is positive, then the calculated plane normal vector Denoted as tool normal vector If the plane normal vector If the Z direction component of is negative, the calculated plane normal vector Convert the opposite direction and transform the plane normal vector Denoted as tool normal vector . Set the plane normal vector with a positive Z component Denoted as tool normal vector , which can be made according to the tool normal vector The calculated yaw angle is within a preset yaw angle range, and the calculated pitch angle is within a preset pitch angle range, so as to prevent the calculated yaw angle or pitch angle from differing by 180 degrees from the actual yaw angle or the actual pitch angle.

[0056] According to the determined tool normal vector , calculate the yaw angle or pitch angle of the visual tracking tool in the image according to the following yaw angle calculation formula or pitch angle calculation formula.

[0057] The yaw angle calculation formula is:

[0058] in, is the yaw angle, is the normal vector The component in the Y direction, is the normal vector Component in the Z direction.

[0059] The pitch angle calculation formula is:

[0060] in, is the yaw angle, is the normal vector The component in the X direction, is the normal vector Component in the Z direction.

[0061] Specifically, when obtaining tool error values ​​for a visual tracking tool at multiple yaw angles, a target image is selected based on the calculated yaw angle and a preset angle interval. A computing system then calculates the tool error values ​​for the visual tracking tool at the corresponding yaw angles based on the target image. Based on the calculated tool error values ​​at each yaw angle, a coordinate graph is plotted with the yaw angle value on the horizontal axis and the tool error value on the vertical axis.

[0062] Specifically, when obtaining the tool error values ​​of the visual tracking tool at multiple pitch angles, a target image is selected based on the calculated pitch angle and the preset angle interval. The tool error values ​​of the visual tracking tool at the corresponding pitch angle are calculated based on the target image using the computing system. Based on the calculated tool error values ​​at each pitch angle, a scatter plot can be drawn with the pitch angle value as the horizontal axis and the tool error as the vertical axis, see Figure 6 .

[0063] It should be noted that the preset angle interval can be 1 degree, 3 degrees, 5 degrees, etc. The smaller the preset angle interval, the denser the calculated tool error values ​​at each yaw angle. For better results, the preset angle interval is less than or equal to 5 degrees.

[0064] It should be noted that the tool error values ​​of the visual tracking tool at different yaw and pitch angles can also be obtained by fixing the visual tracking tool and rotating the camera. The embodiment of the present invention does not limit the specific rotation method.

[0065] In one embodiment of the present invention, Figure 7 As shown, the visual tracking tool is qualified based on the tool information at each angle, including: S301, checking whether the tool information at each angle is complete and whether the tool error value at each angle is less than a preset error threshold; S302: If the tool information at each angle is not missing and the tool error value at each angle is less than the preset error threshold, the visual tracking tool is determined to be qualified; otherwise, the visual tracking tool is determined to be unqualified.

[0066] It should be noted that when the error of the visual tracking tool is large, that is, the error of the point coordinates of the positioning mark recognized by the computing system is large, or the error of the calculated tool error value is large, the computing system will discard the corresponding point coordinates or tool error value.

[0067] Specifically, it is detected whether the tool information at each yaw and pitch angle is not missing, that is, whether the tool error value at each yaw and pitch angle and the point coordinates of each positioning marker are missing. If the tool error value at each yaw and pitch angle and the point coordinates of each positioning marker are not missing, it is detected whether the tool error value at each yaw and pitch angle is less than the preset error threshold. If the tool information at each angle is not missing and the tool error value at each angle is less than the preset error threshold, the visual tracking tool is determined to be qualified. Otherwise, the visual tracking tool is determined to be unqualified.

[0068] In one embodiment, when calculating the tool error value for the target image, if the number of identified positioning marks of the visual tracking tool is equal to or greater than 3, that is, the point coordinates of at least 3 positioning marks are identified, it is considered that the tool information at the corresponding angle is not missing.

[0069] In another embodiment, when calculating the tool error value for the target image, if the positioning marks of the visual tracking tool are not all recognized, that is, the coordinates of all the positioning marks of the visual tracking tool are not recognized, it is considered that the tool information at the corresponding angle is not missing.

[0070] In one embodiment of the present invention, Figure 8 As shown, the visual tracking tool quality inspection method may further include: S401, translating the visual tracking tool along the camera z-axis within the depth range of the camera workspace, and keeping the tool normal vector consistent with the camera z-axis direction; S402, collecting a preset number of imaging images during the translation process, identifying the point coordinates of each positioning mark of the visual tracking tool in each imaging image, and calculating the tool error value of the visual tracking tool at multiple depths in each imaging image; S403 , determining whether the visual tracking tool is qualified according to the tool information at each depth, and determining a problem location identifier of the unqualified visual tracking tool if the visual tracking tool is determined to be unqualified.

[0071] Since the visual tracking tool is located at different depths in the camera working space, the camera has different exposure values ​​when capturing images, and different exposure values ​​will also affect the accuracy of the visual tracking tool.

[0072] To further test whether different exposure values ​​at different depths affect the accuracy of the visual tracking tool, translate the visual tracking tool along the camera's z-axis within the depth range of the camera's workspace. During the translation process, the tool's normal vector must remain aligned with the camera's z-axis.

[0073] During the translation process, a preset number of images, such as at least 100, are collected within the depth range of the camera workspace. A camera-based computing system is used to identify the coordinates of the positioning marks of the visual tracking tool in each image to form 100 sets of points. The computing system is used to calculate the tool error values ​​of the visual tracking tool at multiple depths in each image. The obtained point sets and tool error values ​​cover the nearest to the farthest depth of the camera workspace. Based on the calculated tool error values ​​at each depth, a coordinate graph can be drawn with the horizontal axis being the depth and the vertical axis being the tool error, see Figure 9 .

[0074] Check whether the tool information at each depth is not missing, and whether the tool error value at each depth is less than the preset error threshold, to determine whether the visual tracking tool is qualified. When the visual tracking tool is determined to be unqualified, determine the problem location mark of the unqualified visual tracking tool based on the corresponding imaging image.

[0075] In one embodiment of the present invention, determining whether the visual tracking tool is qualified based on the tool information at each depth may include: Check whether the tool information at each depth is complete and whether the tool error value at each depth is less than the preset error threshold; If the tool information at each depth is not missing and the tool error value at each depth is less than the preset error threshold, the visual tracking tool is determined to be qualified; otherwise, the visual tracking tool is determined to be unqualified.

[0076] Specifically, the tool information at each depth is checked to see if it is present. Specifically, the tool error values ​​and the coordinates of each positioning marker are checked for missing values ​​at each depth. If the tool error values ​​and the coordinates of each positioning marker are present at each depth, the tool error values ​​at each depth are checked to see if they are all below a preset error threshold. If the tool information at each angle is present and the tool error values ​​at each angle are all below the preset error threshold, the visual tracking tool is determined to be qualified. Otherwise, the visual tracking tool is determined to be unqualified.

[0077] The visual tracking tool quality detection method of an embodiment of the present invention controls the visual tracking tool to perform yaw rotation and pitch rotation at the three test positions mentioned above, respectively, to detect whether the tool information at each yaw angle and pitch angle at the three test positions is not missing, and whether the corresponding tool error values ​​are all less than a preset error threshold. The visual tracking tool is also controlled to translate within the depth range of the camera workspace to detect whether the tool information at each depth is not missing, and whether the corresponding tool error values ​​are all less than a preset error threshold. The embodiment of the present invention combines accuracy-angle and accuracy-depth to determine whether the tool accuracy is qualified.

[0078] If the tool information calculated during the yaw rotation and pitch rotation at the above three positions is not missing, and the tool error values ​​are all less than the preset error threshold, and the tool information calculated during the translation within the depth range of the camera working space is not missing, and the tool error values ​​are all less than the preset error threshold, then it is determined that the accuracy of the visual tracking tool meets the requirements, that is, the visual tracking tool is qualified and can be used.

[0079] If there is missing tool information and / or the tool error value is greater than or equal to the preset error threshold, the imaging image corresponding to the missing tool information and / or the imaging image corresponding to the tool error value is greater than or equal to the preset error threshold is saved to check the problem positioning mark of the unqualified visual tracking tool based on the saved imaging image.

[0080] In one embodiment of the present invention, the camera is a binocular camera, and the imaging image includes a first image and a second image. Figure 10 As shown, the problem location identification of unqualified visual tracking tools may include: S501, calculating the actual point coordinates of each positioning mark of the visual tracking tool based on the first image and the second image at the angle or depth corresponding to the tool error value being greater than the preset error threshold; S502, calculating the actual distance between each positioning marker and other positioning markers according to the actual point coordinates of each positioning marker; S503, calculating the calibration distance value between each positioning marker and other positioning markers according to the calibration point coordinates of each positioning marker; S504, calculating the number of times that the absolute value of the difference between the actual distance value and the calibration distance value between each positioning marker and other positioning markers is greater than a preset error threshold, and obtaining the number of distance deviations of each positioning marker; S505, sorting the positioning identifiers in descending order according to the number of distance deviations; S506: Based on the sorted positioning identifiers, determine in turn whether the number of distance deviations corresponding to the positioning identifiers is greater than or equal to Cnt+1. If so, Cnt is incremented by 1, and the positioning identifier is recorded as a problem positioning identifier. The initial value of Cnt is 0.

[0081] The camera in the embodiment of the present invention is a binocular camera, and the imaging images collected by the binocular camera include a first image and the second image .

[0082] Specifically, according to the first image at the angle or depth corresponding to the tool error value being greater than the preset error threshold and the second image , calculate the actual point coordinates of each positioning mark of the visual tracking tool 、 、……、 Calculate the actual distance between each positioning mark and other positioning marks based on the actual point coordinates of each positioning mark ,in, Indicates the i-th positioning marker ( ) and the j-th positioning marker ( ), 1≤i≤n, 1≤j≤n. In the case where the visual tracking tool has n spheres, There are n-1 actual distance values The actual distance in the embodiment of the present invention .

[0083] Using the above distance calculation formula, according to the calibration point coordinates of each positioning mark 、 、……、 , calculate the calibration distance value between each positioning mark and other positioning marks The coordinates of the calibration points are determined based on the design values ​​or manufacturing values ​​of the relative positions before the positioning marks in the visual tracking tool.

[0084] Calculate the actual distance between each positioning mark and other positioning marks and calibration distance value The absolute value of the difference , count the n-1 locations of each location marker Greater than the preset error threshold The number of each positioning mark The number of distance deviations .

[0085] It is feasible to initialize each positioning identifier The number of distance deviations , in calculating the positioning identifier The number of distance deviations When the positioning mark The number of distance deviations Is it satisfied When the number of distance deviations is satisfied, Add 1.

[0086] According to the number of distance deviations The size of the positioning mark is arranged in descending order. Define the counter Cnt and initialize Cnt to 0. According to the sorting order of the positioning markers, detect the positioning markers in turn. The number of distance deviations Is it satisfied ≥Cnt+1. If satisfied, Cnt will be incremented by 1 and the location marker will be marked Mark as the problem location marker. If the above sequence judgment process does not meet the ≥Cnt+1, the value of Cnt remains unchanged and the next positioning mark is determined.

[0087] If there are only one or two problem location markers, you can directly replace the problem location markers. If there are many problem location markers, you can directly replace the visual tracking tool.

[0088] In one embodiment of the present invention, Figure 11 As shown, the problem location identification of unqualified visual tracking tools may include: S601, acquiring a first image and a second image at an angle or depth corresponding to missing tool information; S602, respectively obtaining the center of each positioning mark of the visual tracking tool in the first image and the second image; S603, for the center of each positioning mark in the first image and the second image, respectively calculate the number of pixels of multiple straight lines passing through the center of the positioning mark that meet the grayscale threshold, and obtain the number of pixels corresponding to each straight line passing through the center of the positioning mark. If the pixel difference between the number of pixels corresponding to any two straight lines is greater than the preset pixel number difference threshold, then the positioning mark is regarded as a problem positioning mark.

[0089] Specifically, obtain the first image at the angle or depth corresponding to the missing tool information and the second image .

[0090] Since the positioning markers based on medical treatment are generally spherical or circular, the shape of the image formed by the projection of the theoretically intact marker point on the camera acquisition plane is circular. Therefore, the center of each positioning marker of the visual tracking tool in each image can be obtained. . Calculate the number of pixels that meet the grayscale threshold of multiple straight lines passing through the center of the positioning mark circle.

[0091] In a specific example, on the first image and the second image, based on the centers of the identified positioning marks, straight lines are created in multiple preset directions, such as Figure 12 As shown, there are four straight lines passing through the center of the positioning mark circle, and they are based on the center of the circle. When four straight lines are drawn along the X direction, 45-225 degree direction, Y direction, and 135-45 degree direction, each straight line is detected to meet the grayscale threshold. The number of pixels, i and j start to increase from 1, and stop counting when the following formula is no longer satisfied. The calculation method for the X direction is:

[0092] The calculation method for the 45-degree and 225-degree directions is:

[0093] The calculation method for the Y direction is:

[0094] The calculation method for the 135-45 degree direction is:

[0095] The calculated 4 straight lines meet the gray threshold The number of pixels is recorded as 、 、 、 .

[0096] Calculate the number of pixels corresponding to each line The number of pixels corresponding to other lines If the pixel difference between , then the positioning mark is recorded as the problem positioning mark. Among them, the preset pixel number difference threshold Usually Set to 1, which can be determined based on the actual positioning accuracy requirements of the camera system.

[0097] The visual tracking tool quality inspection method of an embodiment of the present invention determines the accuracy of visual tracking tools to address the problem of decreased tool accuracy that may occur due to repeated use and disinfection of visual tracking tools in actual clinical practice. The visual tracking tool is rotated and moved automatically or manually to image and track the tool at multiple positions and angles in the camera workspace. The accuracy of the tool is then determined by combining accuracy-angle and accuracy-depth views. If the tool fails to meet the requirements, the actual point coordinates and image information of the visual tracking tool are combined with the tool calibration information or design information to determine the location of the problem.

[0098] The invention provides a controller.

[0099] In this embodiment, the controller may include a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the above-mentioned visual tracking tool quality detection method is implemented.

[0100] Figure 13 4 is a structural block diagram of a controller according to an embodiment of the present invention.

[0101] like Figure 13 As shown, controller 500 includes a processor 501 and a memory 503. Processor 501 and memory 503 are connected, for example, via a bus 502. Optionally, controller 500 may further include a transceiver 504. It should be noted that in practical applications, the number of transceivers 504 is not limited to one, and the structure of controller 500 does not constitute a limitation on the embodiments of the present invention.

[0102] Processor 501 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. Processor 501 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0103] Bus 502 may include a path for transmitting information between the above components. Bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 502 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 13 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0104] The memory 503 is used to store a computer program corresponding to the visual tracking tool quality detection method of the above embodiment of the present invention, and the computer program is controlled and executed by the processor 501. The processor 501 is used to execute the computer program stored in the memory 503 to implement the content shown in the above method embodiment. Figure 13 The controller 500 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0105] The controller in the embodiment of the present invention, based on the above-mentioned visual tracking tool quality detection method, can quickly, accurately and efficiently evaluate whether the accuracy of the visual tracking tool is qualified, and quickly and efficiently find the problem location marks of unqualified visual tracking tools.

[0106] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0107] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0108] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0109] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0110] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0111] In the present invention, unless otherwise specified or limited, the terms "installed," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediate medium; internal communication between two components, or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0112] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediary. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0113] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A visual tracking tool quality detection method, characterized in that: The visual tracking tool includes a plurality of positioning marks, and the method includes: Determine the test position of the visual tracking tool in the camera workspace; Rotating the visual tracking tool at the test position, and calculating tool information of the visual tracking tool based on an imaging image of the visual tracking tool captured by the camera during the rotation process, to obtain tool information of the visual tracking tool at multiple angles; Whether the visual tracking tool is qualified is determined according to the tool information at each of the angles, and when the visual tracking tool is determined to be unqualified, a problem location identifier of the unqualified visual tracking tool is determined.

2. The visual tracking tool quality inspection method according to claim 1, characterized in that: The test positions include a horizontal visual field limit position, a vertical visual field limit position, and a center position. Determining the test position of the visual tracking tool in the camera workspace includes: Placing the visual tracking tool at a target test position, collecting current position information of the visual tracking tool, and generating a current position identifier based on the current position information; Detect whether the current position identifier is consistent with a pre-established target test position marker, and generate a prompt message if they are consistent.

3. The visual tracking tool quality inspection method according to claim 1, characterized in that: The rotating the vision tracking tool at the test position comprises: Controlling the visual tracking tool to yaw and rotate within a preset yaw angle range at the test position; At the test position, the visual tracking tool is controlled to pitch and rotate within a preset pitch angle range.

4. The visual tracking tool quality inspection method according to claim 1, characterized in that: The number of the positioning marks is at least 3, the tool information includes a tool error value and the point coordinates of each positioning mark, and the tool information of the visual tracking tool is calculated based on the imaging image of the visual tracking tool captured by the camera during the rotation process, including: Identifying the point coordinates of each positioning mark of the visual tracking tool in the imaging image; Calculating a plane normal vector based on the point coordinates of at least three of the positioning marks, and converting the plane normal vector in the opposite direction when the z-direction component of the plane normal vector is negative, to obtain a tool normal vector; Calculating the yaw angle or pitch angle of the visual tracking tool in the imaging image according to the tool normal vector; A target image is determined according to a preset angle interval and the yaw angle or pitch angle, and a tool error value of the visual tracking tool at the corresponding yaw angle or the tool error value at the corresponding pitch angle is calculated according to the target image to obtain the tool error values ​​at multiple yaw angles or the tool error values ​​at multiple pitch angles.

5. The visual tracking tool quality inspection method according to claim 4, characterized in that: The determining whether the visual tracking tool is qualified according to the tool information at each angle includes: Check whether the tool information at each angle is not missing and whether the tool error value at each angle is less than a preset error threshold; If the tool information at each of the angles is not missing and the tool error value at each of the angles is less than the preset error threshold, the visual tracking tool is determined to be qualified; otherwise, the visual tracking tool is determined to be unqualified.

6. The visual tracking tool quality inspection method according to claim 1, characterized in that: The method further comprises: Within the depth range of the camera working space, translate the visual tracking tool along the camera z-axis direction and keep the tool normal vector consistent with the camera z-axis direction; Collecting a preset number of imaging images during the translation process, identifying the point coordinates of each positioning mark of the visual tracking tool in each imaging image, and calculating the tool error value of the visual tracking tool at multiple depths in each imaging image; Whether the vision tracking tool is qualified is determined according to the tool information at each depth, and when the vision tracking tool is determined to be unqualified, a problem location identifier of the unqualified vision tracking tool is determined.

7. The visual tracking tool quality inspection method according to claim 6, characterized in that: The determining whether the visual tracking tool is qualified according to the tool information at each depth includes: Checking whether the tool information at each depth is not missing and whether the tool error value at each depth is less than a preset error threshold; If the tool information at each of the depths is not missing and the tool error values ​​at each of the depths are less than a preset error threshold, the visual tracking tool is determined to be qualified; otherwise, the visual tracking tool is determined to be unqualified.

8. The visual tracking tool quality inspection method according to claim 1 or 6, characterized in that: The camera is a binocular camera, the imaging image includes a first image and a second image, and the method of determining the problem location mark of the unqualified visual tracking tool includes: Calculating the actual point coordinates of each positioning mark of the visual tracking tool based on the first image and the second image at the angle or depth corresponding to the tool error value being greater than the preset error threshold; Calculate the actual distance between each positioning marker and other positioning markers according to the actual point coordinates of each positioning marker; Calculating the calibration distance between each positioning marker and other positioning markers according to the calibration point coordinates of each positioning marker; Calculating the number of times that the absolute value of the difference between the actual distance value and the calibrated distance value between each positioning marker and other positioning markers is greater than a preset error threshold, to obtain the number of distance deviations of each positioning marker; Sorting the positioning identifiers in descending order according to the number of the distance deviations; According to the sorted positioning identifiers, determine in turn whether the number of distance deviations corresponding to the positioning identifier is greater than or equal to Cnt+1. If it is greater, Cnt is incremented by 1, and the positioning identifier is recorded as the problem positioning identifier. The initial value of Cnt is 0.

9. The visual tracking tool quality inspection method according to claim 1 or 6, characterized in that: The problem location identification of the unqualified visual tracking tool includes: Acquire a first image and a second image at an angle or depth corresponding to the missing tool information; respectively obtaining the center of each positioning mark of the visual tracking tool in the first image and the second image; For the center of each positioning mark in the first image and the second image, the number of pixels of multiple straight lines passing through the center of the positioning mark that meet the grayscale threshold is calculated respectively, and the number of pixels corresponding to each straight line passing through the center of the positioning mark is obtained. If the pixel difference between the number of pixels corresponding to any two straight lines is greater than the preset pixel number difference threshold, the positioning mark is identified as the problem positioning mark.

10. A controller comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the computer program is executed by the processor, the visual tracking tool quality detection method according to any one of claims 1 to 9 is implemented.

Images