Visual tracking tool quality detection method and controller

Abstract

The application relates to the technical field of visual tracking tools, and discloses a visual tracking tool quality detection method and a controller, wherein the visual tracking tool comprises multiple positioning marks, the method comprises the following steps: determining a test position of the visual tracking tool in a camera working space; rotating the visual tracking tool at the test position, and calculating tool information of the visual tracking tool according to imaging images of the visual tracking tool collected by a camera in the rotating process to obtain the tool information of the visual tracking tool at multiple angles; determining whether the visual tracking tool is qualified according to the tool information at the multiple angles, and determining a problem positioning mark of the unqualified visual tracking tool when it is determined that the visual tracking tool is unqualified. The method can quickly, accurately and efficiently evaluate whether the precision of the visual tracking tool is qualified, and can quickly and efficiently find out the problem positioning mark of the unqualified visual tracking tool.

Description

Technical Field

[0001] This invention relates to the field of visual tracking tool technology, and in particular to a method and controller for quality inspection of visual tracking tools. Background Technology

[0002] Tracker tools may experience a decrease in accuracy after installation, disinfection, or repeated use. Evaluating the quality of tracker tools and identifying the problems in inferior tracker tools are issues that need to be addressed by those skilled in the art. Summary of the Invention

[0003] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, one objective of this invention is to provide a method for quality inspection of visual tracking tools, which can quickly, accurately, and efficiently evaluate whether the accuracy of visual tracking tools is up to standard, and quickly and efficiently identify the problem location markers of substandard visual tracking tools.

[0004] The second objective of this invention is to provide a controller.

[0005] To achieve the above objectives, a first aspect of the present invention provides a method for quality inspection of a visual tracking tool. The visual tracking tool includes multiple positioning markers. The method includes: determining a test position of the visual tracking tool in a camera workspace; rotating the visual tracking tool at the test position and calculating tool information of the visual tracking tool based on the imaging images of the visual tracking tool captured by the camera during the rotation, thereby obtaining tool information of the visual tracking tool at multiple angles; determining whether the visual tracking tool is qualified based on the tool information at each angle, and identifying a problem positioning marker of the unqualified visual tracking tool when the visual tracking tool is determined to be unqualified.

[0006] According to the visual tracking tool quality inspection method of this invention, the visual tracking tool is placed at a test position in the camera's workspace and rotated. Based on the imaging images of the visual tracking tool captured by the camera during rotation, tool information at multiple angles is calculated. The accuracy of the visual tracking tool is then evaluated based on this tool information at each angle. When a visual tracking tool is determined to be unqualified, the problem location markers of the unqualified visual tracking tool can be quickly and efficiently identified.

[0007] In addition, the visual tracking tool quality detection method proposed in the above embodiments of the present invention may also have the following additional technical features:

[0008] According to one embodiment of the present invention, the test position includes a horizontal field of view limit position, a vertical field of view limit position, and a center position. Determining the test position of the visual tracking tool in the camera workspace includes: placing the visual tracking tool at the target test position, collecting the current position information of the visual tracking tool, and generating a current position identifier based on the current position information; detecting whether the current position identifier is consistent with a pre-established target test position marker, and generating a prompt message when they are consistent.

[0009] According to one embodiment of the present invention, rotating the visual tracking tool at the test position includes: controlling the visual tracking tool to yaw within a preset yaw angle range at the test position; and controlling the visual tracking tool to pitch within a preset pitch angle range at the test position.

[0010] According to one embodiment of the present invention, the number of positioning markers is at least three, and the tool information includes tool error values ​​and point coordinates of each positioning marker. The step of calculating the tool information of the visual tracking tool based on the imaging image of the visual tracking tool acquired by the camera during rotation includes: identifying the point coordinates of each positioning marker of the visual tracking tool in the imaging image; calculating a plane normal vector based on the point coordinates of at least three positioning markers, and reversing the plane normal vector in the opposite direction when the z-direction component of the plane normal vector is negative to obtain a tool normal vector; calculating the yaw angle or pitch angle of the visual tracking tool in the imaging image based on the tool normal vector; determining a target image based on a preset angle interval and the yaw angle or pitch angle, and calculating the tool error value of the visual tracking tool at the corresponding yaw angle or the corresponding pitch angle based on the target image to obtain multiple tool error values ​​at multiple yaw angles or multiple tool error values ​​at multiple pitch angles.

[0011] According to an embodiment of the present invention, determining whether the visual tracking tool is qualified based on the tool information at each angle includes: detecting whether the tool information at each angle is not missing and whether the tool error value at each angle is less than a preset error threshold; if the tool information at each angle is not missing and the tool error value at each angle is less than the preset error threshold, then the visual tracking tool is determined to be qualified; otherwise, the visual tracking tool is determined to be unqualified.

[0012] According to an embodiment of the present invention, the method further includes: translating the visual tracking tool along the z-axis direction of the camera within the depth range of the camera workspace, while keeping the tool normal vector consistent with the z-axis direction of the camera; acquiring a preset number of imaging images during the translation process, identifying the point coordinates of each positioning marker of the visual tracking tool in each imaging image, and calculating the tool error value of the visual tracking tool at multiple depths in each imaging image; determining whether the visual tracking tool is qualified based on the tool information at each depth, and identifying the problem positioning marker of the unqualified visual tracking tool when the visual tracking tool is determined to be unqualified.

[0013] According to an embodiment of the present invention, determining whether the visual tracking tool is qualified based on the tool information at each depth includes: detecting whether the tool information at each depth is not missing and whether the tool error value at each depth is less than a preset error threshold; if the tool information at each depth is not missing and the tool error value at each depth is less than the preset error threshold, then the visual tracking tool is determined to be qualified; otherwise, the visual tracking tool is determined to be unqualified.

[0014] According to an embodiment of the present invention, the camera is a binocular camera, and the imaging image includes a first image and a second image. Determining the problematic positioning identifier of the unqualified visual tracking tool includes: calculating the actual point coordinates of each positioning identifier of the visual tracking tool based on the first image and the second image at the angle or depth corresponding to the tool error value being greater than a preset error threshold; calculating the actual distance value between each positioning identifier and other positioning identifiers based on the actual point coordinates of each positioning identifier; calculating the calibration distance value between each positioning identifier and other positioning identifiers based on the calibration point coordinates of each positioning identifier; calculating the number of positions where the absolute value of the difference between the actual distance value and the calibration distance value between each positioning identifier and other positioning identifiers is greater than a preset error threshold, thereby obtaining the number of distance deviations for each positioning identifier; sorting the positioning identifiers in descending order based on the number of distance deviations; and sequentially determining whether the number of distance deviations corresponding to the sorted positioning identifiers is greater than or equal to Cnt+1. If it is greater, Cnt is incremented by 1, and the positioning identifier is recorded as the problematic positioning identifier, wherein the initial value of Cnt is 0.

[0015] According to an embodiment of the present invention, determining the problem location marker of a defective visual tracking tool includes: acquiring a first image and a second image at the angle or depth corresponding to the missing tool information; determining the center of each location marker of the visual tracking tool in the first image and the second image respectively; for each location marker in the first image and the second image, calculating the number of pixels that satisfy a grayscale threshold for multiple straight lines passing through the center of the location marker, obtaining the number of pixels corresponding to each straight line passing through the center of the location marker; if there is a pixel difference between the number of pixels corresponding to any two straight lines that is greater than a preset pixel difference threshold, then the location marker is the problem location marker.

[0016] To achieve the above objectives, a second aspect of the present invention provides a controller, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it implements the visual tracking tool quality detection method as proposed in the first aspect of the present invention.

[0017] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0018] Figure 1 This is a flowchart of a visual tracking tool quality inspection method according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram showing the relative positions of the camera and the visual tracking tool according to a specific embodiment of the present invention;

[0020] Figure 3(a) is a three-dimensional view of the camera workspace according to a specific embodiment of the present invention;

[0021] Figure 3(b) is a top view of the camera workspace according to a specific embodiment of the present invention;

[0022] Figure 3(c) is a left view of the camera workspace according to a specific embodiment of the present invention;

[0023] Figure 4(a) is a schematic diagram of the placement of the visual tracking tool according to a specific embodiment of the present invention;

[0024] Figure 4(b) is a schematic diagram of the placement of the visual tracking tool according to another specific embodiment of the present invention;

[0025] Figure 5 This is a flowchart illustrating tool information of a computational vision tracking tool according to an embodiment of the present invention;

[0026] Figure 6 This is a scatter plot of pitch angle-tool error according to a specific embodiment of the present invention;

[0027] Figure 7 This is a flowchart illustrating the determination of whether a visual tracking tool is qualified according to an embodiment of the present invention;

[0028] Figure 8 This is a flowchart illustrating the determination of whether a visual tracking tool is qualified according to another embodiment of the present invention;

[0029] Figure 9 This is a scatter plot of depth-tool error according to an embodiment of the present invention;

[0030] Figure 10 This is a flowchart illustrating the process of identifying the problem location markers of a non-compliant visual tracking tool according to an embodiment of the present invention;

[0031] Figure 11 This is a flowchart illustrating the process of identifying problem location markers for a non-compliant visual tracking tool according to another embodiment of the present invention;

[0032] Figure 12 This is a schematic diagram of four straight lines passing through the center of the positioning mark according to a specific embodiment of the present invention;

[0033] Figure 13 This is a structural block diagram of the controller according to an embodiment of the present invention. Detailed Implementation

[0034] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0035] The visual tracking tool quality detection method and controller of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] The visual tracking tool in this embodiment of the invention includes multiple positioning markers.

[0037] The visual tracking tool in this embodiment of the invention can be a tracer tool. The positioning markers on the visual tracking tool can be spheres, circular planes, etc.

[0038] For example, when the positioning identifier is a sphere, the surface of the sphere is covered with a high-reflectivity coating (such as aluminum or silver) to reflect light.

[0039] It should be noted that repeated disinfection and use of the sphere can damage the coating covering its surface. Damage or peeling of the coating will affect the accuracy of the visual tracking tool. Additionally, improper installation or negligence during the mounting of the sphere onto the support of the visual tracking tool can also affect the tool's accuracy.

[0040] Visual tracking tools are one of the core tools in medical imaging technology. Therefore, it is crucial to evaluate the accuracy of visual tracking tools before using them.

[0041] Figure 1 This is a flowchart of a quality inspection method for a visual tracking tool according to an embodiment of the present invention. Figure 1 As shown, the quality inspection methods for visual tracking tools may include:

[0042] S101, Determine the test location of the visual tracking tool in the camera workspace;

[0043] S102, rotate the visual tracking tool at the test position, and calculate the tool information of the visual tracking tool based on the imaging image of the visual tracking tool captured by the camera during the rotation, so as to obtain the tool information of the visual tracking tool at multiple angles;

[0044] S103, determine whether the visual tracking tool is qualified based on the tool information from each angle, and determine the problem location mark of the unqualified visual tracking tool when the visual tracking tool is determined to be unqualified.

[0045] The embodiments of the present invention can be used to evaluate the accuracy of a visual tracking tool after multiple repeated uses, determine whether the visual tracking tool is qualified, and identify the problem location markers of unqualified visual tracking tools.

[0046] Specifically, it is determined whether the visual tracking tool is at the test position in the camera workspace, and when the test position is determined, the visual tracking tool is rotated. During the rotation of the visual tracking tool, the camera acquires images of the visual tracking tool in real time. Based on the images acquired by the camera, tool information of the visual tracking tool is calculated to obtain tool information of the visual tracking tool at multiple angles. In this embodiment of the invention, the tool information includes the tool error value of the positioning markers and the point coordinates of each positioning marker.

[0047] The accuracy of the visual tracking tool is determined based on whether tool information is missing at various angles and whether the tool error value at each angle is less than a preset error threshold. Specifically, if the visual tracking tool is deemed unacceptable in terms of accuracy, a problem location marker for the unacceptable visual tracking tool is identified based on the image at the angle corresponding to the missing tool information and / or the tool error value being greater than or equal to the preset error threshold. This problem location marker can then be reinstalled or replaced.

[0048] It should be noted that after reinstalling or replacing the problem location marker, the visual tracking tool needs to be tested again to ensure it is up to standard.

[0049] The visual tracking tool quality inspection method of this invention places the visual tracking tool at a test position in the camera's workspace and rotates the tool. Based on the imaging images of the visual tracking tool captured by the camera during rotation, tool information at multiple angles is calculated. The accuracy of the visual tracking tool is then evaluated based on this information. When a visual tracking tool is determined to be unqualified, the problem location markers of the unqualified visual tracking tool can be quickly and efficiently identified based on the corresponding imaging images.

[0050] In one embodiment of the present invention, the test position may include a horizontal field-of-view limit position, a vertical field-of-view limit position, and a center position. Determining the test position of the visual tracking tool in the camera workspace may include:

[0051] Place the visual tracking tool at the target test location, collect the current location information of the visual tracking tool, and generate a current location identifier based on the current location information;

[0052] It checks whether the current location identifier matches the pre-established target test location marker, and generates a prompt message when they match.

[0053] The test positions in this embodiment of the invention may include three extreme positions of the camera workspace: the horizontal field-of-view extreme position, the vertical field-of-view extreme position, and the center position. These three test positions can be determined based on the camera's computing system, and corresponding markers can be established after determining them. The established test position markers can be displayed on the display screen within the camera workspace.

[0054] When determining whether the visual tracking tool is at the horizontal field of view limit of the camera's workspace, this horizontal field of view limit is recorded as the target test position. The operator places the visual tracking tool according to this target test position. The computing system collects the current position information of the visual tracking tool, generates a current position marker based on this information, and checks whether the current position marker matches the pre-established target test position marker. If the current position marker does not match the pre-established target test position marker, the visual tracking tool is moved, and the above detection process is repeated until the current position marker matches the pre-established target test position marker. When the current position marker matches the pre-established target test position marker, an audio or visual prompt can be used to indicate that the visual tracking tool has reached the preset test position.

[0055] Based on the above process, it can be determined whether the visual tracking tool is at the vertical field of view limit and the exact center position in the camera's workspace.

[0056] Figure 2 The diagram shows the relative positions of the visual tracking tool and the camera. Figure 2 The coordinate system shown satisfies the right-handed coordinate system. This invention embodiment defines... Figure 2 The camera positioning device is placed horizontally, with the horizontal direction as the Y-axis, the vertical direction as the X-axis, and the working depth direction as the Z-axis. Among these, Figure 2 The visual tracking tool in the system is configured with four spheres.

[0057] In this embodiment of the invention, the camera workspace is determined based on the camera's shooting space, which is determined by the camera's own parameters. The images captured by the camera and the three-dimensional workspace created based on the camera's position can be displayed in real time on a screen using software. The three-dimensional view, top view, and left view of the camera workspace are shown in Figures 3(a), 3(b), and 3(c), respectively.

[0058] A specific example illustrates the determination of the horizontal field-of-view limit position in the camera's workspace:

[0059] Based on the top view of the camera workspace shown in Figure 4(a), we detect whether the slope of the camera's horizontal field of view changes as the working distance between the visual tracking tool and the camera increases. The slope line is determined by the position of the camera's center point and the location of the visual tracking tool. It should be noted that the slope of the camera's horizontal field of view in Figure 4(a) changes as the working distance between the visual tracking tool and the camera increases.

[0060] If the slope changes, it needs to be adjusted according to the extreme position of the angle (Figure 4(a)). Place the visual tracking tool within the slope line, specifically in the black area of ​​Figure 4(a), which represents the horizontal limit position area. Within this horizontal limit position area, select a position at the same vertical height as the camera (in...). Figure 2 The position where the X-direction value is 0 in the coordinate system defined in the figure is the limit position of the horizontal field of view.

[0061] If the slope remains unchanged, that is, if in Figure 4(a) , , Collinearity means that any position at a depth corresponding to the same vertical height as the camera can be chosen in the horizontal limit direction.

[0062] A specific example illustrates the determination of the vertical field-of-view limit position in the camera's workspace:

[0063] Based on the left view of the camera workspace shown in Figure 4(b), it is detected that the vertical field of view of the camera changes slope as the working distance between the visual tracking tool and the camera increases. It should be noted that the horizontal field of view of the camera in Figure 4(b) does not change slope as the working distance between the visual tracking tool and the camera increases.

[0064] If the slope changes, then following the above method, the angular limit position region is recorded as the vertical limit position region. Within this vertical limit position region, a horizontal position and a position directly opposite the camera are selected (in...). Figure 2 The position where the Y-direction value is 0 in the defined coordinate system is the vertical field of view limit position.

[0065] If the slope remains unchanged, i.e., in 4(b) , , If they are collinear, then in the vertical limit direction of 4(b), any position corresponding to the depth can be chosen between the horizontal direction and the position directly opposite the camera.

[0066] To illustrate how to determine the exact center of the camera workspace, a specific example is provided:

[0067] The exact center of the camera workspace is in Figure 2 In the defined coordinate system, any position corresponding to any depth can be selected from the locations where the X and Y values ​​are both 0.

[0068] In this embodiment of the invention, a visual tracking tool is placed at the horizontal field of view limit position, the vertical field of view limit position, and the center position of the camera workspace, respectively. The visual tracking tool can be manually or based on an automatic platform according to the following rotation method.

[0069] In one embodiment of the present invention, rotating the visual tracking tool at the test position may include:

[0070] At the test position, control the yaw rotation of the vision tracking tool within a preset yaw angle range;

[0071] At the test position, control the pitch and rotation of the vision tracking tool within a preset pitch angle range.

[0072] It should be noted that the preset yaw angle range and preset pitch angle range can be determined according to the application scenario of the visual tracking tool.

[0073] Specifically, when using an automatic platform to rotate a vision tracking tool, the vision tracking tool can be fixedly mounted on the automatic platform. The automatic platform can be controlled to drive the vision tracking tool to yaw and rotate within a preset yaw angle range, and to pitch and rotate within a preset pitch angle range.

[0074] For example, if the preset yaw angle range is ±50 degrees, then at the test position, the automatic platform controlling the visual tracking tool to yaw and rotate needs to rotate from -50 degrees to +50 degrees. This rotation must be continuous and slow to allow the camera to capture images of the visual tracking tool at more yaw angles. The same principle applies to pitch rotation, and will not be elaborated further here.

[0075] In one embodiment of the present invention, the number of positioning markers is at least three, and the tool information includes the tool error value and the point coordinates of each positioning marker, such as... Figure 5 As shown, during the rotation process, the tool information of the visual tracking tool is calculated based on the imaging image of the visual tracking tool captured by the camera, which may include:

[0076] S201, Identify the point coordinates of each positioning marker of the visual tracking tool in the image;

[0077] S202, calculate the plane normal vector based on the point coordinates of at least 3 positioning markers, and when the z-direction component of the plane normal vector is negative, reverse the direction of the plane normal vector to obtain the tool normal vector;

[0078] S203, calculate the yaw angle or pitch angle of the visual tracking tool in the image based on the tool normal vector;

[0079] S204, determine the target image according to the preset angle interval and yaw angle or pitch angle, and calculate the tool error value of the visual tracking tool at the corresponding yaw angle or the corresponding pitch angle according to the target image, so as to obtain the tool error value at multiple yaw angles or multiple pitch angles.

[0080] This invention aims to improve the accuracy of determining whether a visual tracking tool is qualified by obtaining relatively uniform tool error values ​​under different yaw and pitch angles. Specifically, it identifies the yaw or pitch angle of the visual tracking tool in the imaging image captured by the camera, determines the target yaw or pitch angle to be selected from the imaging image according to a preset angular interval, uses the image captured at the target yaw or pitch angle as the target image, and calculates the tool error values ​​of the visual tracking tool under different yaw and pitch angles based on the target image.

[0081] Specifically, for each acquired image, the coordinates of each positioning marker of the visual tracking tool in the image are identified, and the coordinates of each positioning marker are obtained. , ... , where n represents the number of location markers, n≥3.

[0082] If the visual tracking tool has 3 spheres, calculate the plane normal vector based on the coordinates of the points on the 3 spheres. If the visual tracking tool has 4 spheres, calculate the plane normal vector based on the coordinates of the points on the 4 spheres. If the visual tracking tool has 5 or more spheres, randomly select 4 spheres as a group and calculate the plane normal vector.

[0083] As a specific example, the visual tracking tool has three spheres, and the coordinates of the points on the three spheres are as follows: , , Plane normal vector ,in, , That is, the plane normal vector. for and The cross product vector.

[0084] As another specific example, if the visual tracking tool has four or more spheres, and the coordinates of the points of the four selected spheres are... , , , Plane normal vector ,in, , .

[0085] It should be noted that if the plane normal vector If the Z-direction component is positive, then the calculated plane normal vector will be... Let it be denoted as the tool normal vector. If the plane normal vector If the Z-direction component is negative, then the calculated plane normal vector will be... Reverse the direction to get the transformed plane normal vector Let it be denoted as the tool normal vector. The plane normal vector with positive Z-direction component. Let it be denoted as the tool normal vector. This allows us to use the tool normal vector The calculated yaw angle is within the preset yaw angle range, and the calculated pitch angle is within the preset pitch angle range, to prevent the calculated yaw angle or pitch angle from differing from the actual yaw angle or pitch angle by 180 degrees.

[0086] Based on the determined tool normal vector Calculate the yaw angle or pitch angle of the visual tracking tool in the image according to the following formula for yaw angle or pitch angle calculation.

[0087] The formula for calculating the yaw angle is as follows:

[0088]

[0089] in, Yaw angle Normal vector The component in the Y direction, Normal vector The component in the Z direction.

[0090] The formula for calculating the pitch angle is:

[0091]

[0092] in, Yaw angle Normal vector The component in the X direction, Normal vector The component in the Z direction.

[0093] Specifically, when acquiring the tool error values ​​of the visual tracking tool at multiple yaw angles, a target image is selected based on the calculated yaw angle and a preset angle interval. Using a calculation system, the tool error values ​​of the visual tracking tool at the corresponding yaw angles are calculated based on the target image. Based on the calculated tool error values ​​at each yaw angle, a coordinate graph can be plotted with the yaw angle value on the horizontal axis and the tool error on the vertical axis.

[0094] Specifically, when obtaining the tool error values ​​of the visual tracking tool at multiple pitch angles, a target image is selected based on the calculated pitch angle and a preset angle interval. Using the calculation system, the tool error values ​​of the visual tracking tool at the corresponding pitch angles are calculated based on the target image. Based on the calculated tool error values ​​at each pitch angle, a scatter plot can be generated with the pitch angle value on the horizontal axis and the tool error on the vertical axis. (See [link to relevant documentation]). Figure 6.

[0095] It should be noted that the preset angular interval can be 1 degree, 3 degrees, 5 degrees, etc. The smaller the preset angular interval, the denser the tool error values at each yaw angle calculated. For better results, the preset angular interval is less than or equal to 5 degrees.

[0096] It should be noted that to obtain the tool error values of the visual tracking tool at different yaw angles and pitch angles, it can also be obtained by fixing the visual tracking tool and rotating the camera. The embodiments of the present invention do not limit the specific rotation method.

[0097] In an embodiment of the present invention, as Figure 7 shown, determining whether the visual tracking tool is qualified according to the tool information at each angle includes:

[0098] S301, detecting whether the tool information at each angle is not missing, and whether the tool error values at each angle are less than the preset error threshold;

[0099] S302, if the tool information at each angle is not missing and the tool error values at each angle are less than the preset error threshold, then determine that the visual tracking tool is qualified; otherwise, determine that the visual tracking tool is unqualified.

[0100] It should be noted that when the error of the visual tracking tool is large, that is, the point coordinate error of the positioning mark recognized by the calculation system is large, or the error of the calculated tool error value is large, the calculation system will discard the corresponding point coordinates or tool error values.

[0101] Specifically, detecting whether the tool information at each yaw angle and pitch angle is not missing means detecting whether the tool error values and the point coordinates of each positioning mark at each yaw angle and pitch angle are missing. When the tool error values and the point coordinates of each positioning mark at each yaw angle and pitch angle are not missing, detect whether the tool error values at each yaw angle and pitch angle are less than the preset error threshold. If the tool information at each angle is not missing and the tool error values at each angle are less than the preset error threshold, then determine that the visual tracking tool is qualified. Otherwise, determine that the visual tracking tool is unqualified.

[0102] In one implementation, when calculating the tool error value for the target image, if the number of recognized positioning marks of the visual tracking tool is equal to or more than 3, that is, at least 3 point coordinates of the positioning marks are recognized, it is considered that the tool information at the corresponding angle is not missing.

[0103] In another implementation, when calculating the tool error value for the target image, if the positioning markers of the visual tracking tool are not all identified, that is, the coordinates of all the positioning markers of the visual tracking tool are not identified, then it is considered that the tool information is not missing at the corresponding angle.

[0104] In one embodiment of the present invention, such as Figure 8 As shown, the quality inspection method for visual tracking tools may also include:

[0105] S401, within the depth range of the camera workspace, translate the visual tracking tool along the camera's z-axis direction, while keeping the tool's normal vector consistent with the camera's z-axis direction;

[0106] S402: During the translation process, a preset number of imaging images are acquired, the coordinates of each positioning mark of the visual tracking tool in each imaging image are identified, and the tool error value of the visual tracking tool at multiple depths in each imaging image is calculated.

[0107] S403, determine whether the visual tracking tool is qualified based on the tool information at each depth, and determine the problem location mark of the unqualified visual tracking tool when it is determined to be unqualified.

[0108] Because the depth of the camera's working space varies depending on the location of the visual tracking tool, the exposure value when the camera captures images will also vary, and different exposure values ​​will affect the accuracy of the visual tracking tool.

[0109] To further investigate whether different depths and exposure values ​​affect the accuracy of the visual tracking tool, the tool is translated along the camera's z-axis within the depth range of the camera's workspace. During this translation, the tool's normal vector must remain aligned with the camera's z-axis.

[0110] During the translation process, a preset number of images, such as at least 100, are acquired within the depth range of the camera's workspace. A camera-based computing system is used to identify the coordinates of each positioning marker of the visual tracking tool in each image, forming 100 point sets. The computing system then calculates the tool error values ​​of the visual tracking tool at multiple depths in each image. The resulting point sets and tool error values ​​cover the closest to farthest depths within the camera's workspace. Based on the calculated tool error values ​​at each depth, a coordinate graph with depth on the horizontal axis and tool error on the vertical axis can be plotted. (See [reference]). Figure 9 .

[0111] The system checks whether the tool information at each depth is complete and whether the tool error value at each depth is less than the preset error threshold to determine whether the visual tracking tool is qualified. If the visual tracking tool is determined to be unqualified, the system determines the problem location marker of the unqualified visual tracking tool based on the corresponding imaging image.

[0112] In one embodiment of the present invention, determining whether a visual tracking tool is qualified according to the tool information at each depth may include:

[0113] Detecting whether the tool information at each depth is not missing, and whether the tool error values at each depth are all less than a preset error threshold;

[0114] If the tool information at each depth is not missing, and the tool error values at each depth are all less than the preset error threshold, it is determined that the visual tracking tool is qualified; otherwise, it is determined that the visual tracking tool is unqualified.

[0115] Specifically, detecting whether the tool information at each depth is not missing means detecting whether the tool error values and the point coordinates of each positioning identifier at each depth are missing. When the tool error values and the point coordinates of each positioning identifier at each depth are not missing, detecting whether the tool error values at each depth are all less than the preset error threshold. If the tool information at each angle is not missing, and the tool error values at each angle are all less than the preset error threshold, it is determined that the visual tracking tool is qualified. Otherwise, it is determined that the visual tracking tool is unqualified.

[0116] The visual tracking tool quality detection method according to the embodiment of the present invention controls the visual tracking tool to perform yaw rotation and pitch rotation at the above three test positions respectively, and detects whether the tool information at each yaw angle and pitch angle at the above three test positions is not missing, and whether the corresponding tool error values are all less than the preset error threshold. It also controls the visual tracking tool to translate within the depth range of the camera working space, and detects whether the tool information at each depth is not missing, and whether the corresponding tool error values are all less than the preset error threshold. The embodiment of the present invention combines accuracy-angle and accuracy-depth to determine whether the tool accuracy is qualified.

[0117] If the tool information calculated during the yaw rotation and pitch rotation at the above three positions is not missing, and the tool error values are all less than the preset error threshold, and the tool information calculated during the translation within the depth range of the camera working space is not missing, and the tool error values are all less than the preset error threshold, it is determined that the accuracy of the visual tracking tool meets the requirements, that is, the visual tracking tool is qualified and can be used.

[0118] If there is a situation where the tool information is missing and / or the tool error value is greater than or equal to the preset error threshold, save the imaging image corresponding to the missing tool information and / or the imaging image corresponding to the tool error value greater than or equal to the preset error threshold, so as to check the problem positioning identifier of the unqualified visual tracking tool according to the saved imaging image.

[0119] In one embodiment of the present invention, the camera is a binocular camera, and the imaging image includes a first image and a second image, as Figure 10As shown, identifying the problem location markers for non-compliant visual tracking tools may include:

[0120] S501, Calculate the actual point coordinates of each positioning mark of the visual tracking tool based on the first image and the second image at the angle or depth corresponding to the tool error value being greater than the preset error threshold;

[0121] S502, calculate the actual distance between each positioning marker and other positioning markers based on the actual point coordinates of each positioning marker;

[0122] S503, calculate the calibration distance between each positioning marker and other positioning markers based on the calibration point coordinates of each positioning marker;

[0123] S504, calculate the number of locations where the absolute value of the difference between the actual distance value and the calibrated distance value between each location marker and other location markers is greater than a preset error threshold, and obtain the number of distance deviations for each location marker;

[0124] S505, sort the positioning markers in descending order according to the number of distance deviations;

[0125] S506. Based on the sorted positioning identifiers, determine whether the number of distance deviations corresponding to each positioning identifier is greater than or equal to Cnt+1. If it is greater, increment Cnt by 1 and record the positioning identifier as the problem positioning identifier. The initial value of Cnt is 0.

[0126] The camera in this embodiment of the invention is a binocular camera, and the imaging images acquired by the binocular camera include a first image. Second image .

[0127] Specifically, based on the first image at the angle or depth corresponding to the tool error value exceeding a preset error threshold. Second image The actual coordinates of each positioning marker in the computer vision tracking tool , ... Calculate the actual distance between each positioning marker and other positioning markers based on their actual point coordinates. ,in, Indicates the i-th location identifier ( ) and the j-th location identifier ( The distance between spheres, 1≤i≤n, 1≤j≤n. In the case of a visual tracking tool with n spheres, There are n-1 actual distance values Actual distance in this embodiment of the invention .

[0128] Using the distance calculation formula described above, based on the coordinates of the calibration points of each positioning marker... , ... Calculate the calibration distance values ​​between each positioning marker and other positioning markers. The calibration point coordinates are determined based on the design or manufacturing values ​​of the relative positions of the positioning markers in the visual tracking tool.

[0129] Calculate the actual distance between each location marker and other location markers. With calibration distance value absolute value of the difference Count the n-1 locations of each location marker. Medium greater than the preset error threshold The number of each location identifier is obtained. Number of distance deviations .

[0130] Implementably, initialize each positioning identifier. Number of distance deviations In calculating the location identifier Number of distance deviations At that time, determine the location identifier. Number of distance deviations Does it meet the requirements? When the condition is met, the number of distance deviations is... Add 1.

[0131] Based on the number of distance deviations The sizes of the positioning markers are arranged in descending order. Sort the location identifiers. Define a counter Cnt and initialize Cnt to 0. Detect the location identifiers sequentially according to their sorting order. Number of distance deviations Does it meet the requirements? ≥Cnt+1. If satisfied, Cnt is incremented by 1 and the location identifier is updated. Marked as a problem location identifier. If the above sequential judgment process does not meet the following conditions... If the value is greater than or equal to Cnt+1, then the value of Cnt remains unchanged, and the next positioning identifier is determined.

[0132] If there are only one or two problem location markers, you can directly replace the problem location markers. If there are many problem location markers, you can directly replace the visual tracking tool.

[0133] In one embodiment of the present invention, such as Figure 11 As shown, identifying the problem location markers for non-compliant visual tracking tools may include:

[0134] S601, acquire the first and second images at the angle or depth corresponding to the missing tool information;

[0135] S602, respectively find the center of each positioning mark of the visual tracking tool in the first image and the second image;

[0136] S603, for the center of each positioning mark in the first image and the second image, calculate the number of pixels that satisfy the grayscale threshold for multiple straight lines passing through the center of the positioning mark, and obtain the number of pixels corresponding to each straight line passing through the center of the positioning mark. If the pixel difference between the number of pixels corresponding to any two straight lines is greater than the preset pixel difference threshold, then the positioning mark is a problem positioning mark.

[0137] Specifically, the first image is obtained at the angle or depth corresponding to the missing tool information. Second image .

[0138] Since medical positioning markers are generally spherical or circular, the theoretically intact marker points projected onto the camera's acquisition plane will form a circular image. Therefore, the center of each positioning marker of the visual tracking tool in each image can be determined. Calculate the number of pixels that satisfy the grayscale threshold for each of the multiple straight lines passing through the center of the positioning marker.

[0139] In a specific example, on the first and second images, straight lines are created in multiple preset directions based on the centers of the circles of the identified positioning markers, such as... Figure 12 As shown, there are four straight lines passing through the center of the positioning marker circle, and each line is based on the center of the circle. When traversing four straight lines along the X direction, 45° to 225°, and the Y direction, 135° to 45°, detect that each line satisfies the grayscale threshold. The number of pixels, i and j, starts from 1 and increases until the following formula is no longer satisfied, at which point the counting stops.

[0140] The X-direction is calculated as follows:

[0141] The calculation method for the 45-degree and 225-degree directions is as follows:

[0142] The calculation method for the Y direction is as follows:

[0143] The calculation method for the 135-degree to 45-degree direction is as follows:

[0144] The four calculated lines satisfy the grayscale threshold. The number of pixels is denoted as follows: , , , .

[0145] Calculate the number of pixels corresponding to each line. Number of pixels corresponding to other lines The pixel difference between them, if it satisfies If so, this location identifier is recorded as the problem location identifier. The preset pixel difference threshold is used as an example. Usually Set to 1, but the specific value can be determined based on the actual positioning accuracy requirements of the camera system.

[0146] This invention provides a method for quality inspection of visual tracking tools to address the potential decrease in tool accuracy caused by repeated use and disinfection of clinical visual tracking tools. The method involves automatically or manually rotating and moving the visual tracking tool to image and track it from multiple positions and angles within the camera's workspace. The accuracy is then assessed using a combination of accuracy-angle and accuracy-depth views to determine if the tool's accuracy is acceptable. If the accuracy is unacceptable, the actual point coordinates of the visual tracking tool and image information, combined with tool calibration or design information, are used to pinpoint the problem location.

[0147] This invention provides a controller.

[0148] In this embodiment, the controller may include a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, it implements the visual tracking tool quality detection method as described above.

[0149] Figure 13 This is a structural block diagram of the controller according to an embodiment of the present invention.

[0150] like Figure 13 As shown, the controller 500 includes a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, for example, via a bus 502. Optionally, the controller 500 may also include a transceiver 504. It should be noted that in practical applications, the transceiver 504 is not limited to one, and the structure of the controller 500 does not constitute a limitation on the embodiments of the present invention.

[0151] Processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 501 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0152] Bus 502 may include a pathway for transmitting information between the aforementioned components. Bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 502 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 13 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0153] The memory 503 stores a computer program corresponding to the visual tracking tool quality detection method of the above embodiments of the present invention. This computer program is controlled and executed by the processor 501. The processor 501 executes the computer program stored in the memory 503 to implement the content shown in the foregoing method embodiments. Figure 13 The controller 500 shown is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.

[0154] The controller in this embodiment of the invention, based on the above-described visual tracking tool quality detection method, can quickly, accurately, and efficiently evaluate whether the accuracy of the visual tracking tool is up to standard, and quickly and efficiently identify the problem location markers of unqualified visual tracking tools.

[0155] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0156] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0157] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0158] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0159] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0160] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0161] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0162] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for quality inspection of a visual tracking tool, characterized in that, The visual tracking tool includes multiple positioning markers, and the method includes: Determine the test location for the visual tracking tool in the camera workspace; The visual tracking tool is rotated at the test position, and the tool information of the visual tracking tool is calculated based on the imaging image of the visual tracking tool captured by the camera during the rotation. The tool information of the visual tracking tool at multiple angles is obtained, wherein the tool information includes the tool error value and the point coordinates of each positioning mark. The point coordinates of each positioning mark are determined by the imaging image of the visual tracking tool at the corresponding angle, and the tool error value is determined by the point coordinates of each positioning mark of the visual tracking tool at the corresponding angle. The visual tracking tool is deemed qualified based on whether tool information is missing at each angle and whether the tool error value at each angle is less than a preset error threshold. If the visual tracking tool is deemed unqualified, a problem location identifier for the unqualified visual tracking tool is determined.

2. The method for quality inspection of visual tracking tools according to claim 1, characterized in that, The test positions include the horizontal field-of-view limit position, the vertical field-of-view limit position, and the center position. Determining the test position of the visual tracking tool in the camera workspace includes: The visual tracking tool is placed at the target test location, the current location information of the visual tracking tool is collected, and a current location identifier is generated based on the current location information; The system detects whether the current location identifier matches the pre-established target test location marker, and generates a prompt message if they match.

3. The method for quality inspection of visual tracking tools according to claim 1, characterized in that, The step of rotating the visual tracking tool at the test position includes: At the test position, control the visual tracking tool to yaw and rotate within a preset yaw angle range; At the test position, control the pitch and rotation of the visual tracking tool within a preset pitch angle range.

4. The method for quality inspection of visual tracking tools according to claim 1, characterized in that, The number of positioning markers is at least three, and the tool information includes the tool error value and the point coordinates of each positioning marker. The calculation of the tool information of the visual tracking tool based on the imaging images of the visual tracking tool captured by the camera during rotation includes: Identify the point coordinates of each positioning marker of the visual tracking tool in the imaging image; The plane normal vector is calculated based on the point coordinates of at least three of the positioning markers, and when the z-direction component of the plane normal vector is negative, the plane normal vector is reversed to obtain the tool normal vector. Calculate the yaw angle or pitch angle of the visual tracking tool in the image based on the tool normal vector; The target image is determined based on the preset angle interval and the yaw angle or pitch angle, and the tool error value of the visual tracking tool at the corresponding yaw angle or the corresponding pitch angle is calculated based on the target image to obtain the tool error value at multiple yaw angles or multiple pitch angles.

5. The method for quality inspection of visual tracking tools according to claim 4, characterized in that, The step of determining whether the visual tracking tool is qualified based on whether tool information is missing at each angle and whether the tool error value at each angle is less than a preset error threshold includes: Check whether the tool information at each angle is complete and whether the tool error value at each angle is less than a preset error threshold. If the tool information at each of the stated angles is complete and the tool error value at each of the stated angles is less than the preset error threshold, then the visual tracking tool is determined to be qualified; otherwise, the visual tracking tool is determined to be unqualified.

6. The method for quality inspection of visual tracking tools according to claim 1, characterized in that, The method further includes: Within the depth range of the camera workspace, the visual tracking tool is translated along the camera's z-axis direction, while keeping the tool's normal vector aligned with the camera's z-axis direction; During the translation process, a preset number of imaging images are acquired, the coordinates of each positioning mark of the visual tracking tool in each imaging image are identified, and the tool error value of the visual tracking tool at multiple depths in each imaging image is calculated. The visual tracking tool is determined to be qualified based on the tool information at each depth, and a problem location identifier for the unqualified visual tracking tool is determined when the visual tracking tool is determined to be unqualified.

7. The method for quality inspection of a visual tracking tool according to claim 6, characterized in that, The step of determining whether the visual tracking tool is qualified based on the tool information at each depth includes: Check whether the tool information at each depth is complete and whether the tool error value at each depth is less than a preset error threshold. If the tool information at each depth is complete and the tool error value at each depth is less than the preset error threshold, then the visual tracking tool is determined to be qualified; otherwise, the visual tracking tool is determined to be unqualified.

8. The method for quality inspection of a visual tracking tool according to claim 1 or 6, characterized in that, The camera is a binocular camera, and the image includes a first image and a second image. The problem location marker for determining the substandard visual tracking tool includes: Based on the first and second images at the angles or depths corresponding to tool error values ​​greater than preset error thresholds, calculate the actual point coordinates of each positioning marker of the visual tracking tool; Calculate the actual distance between each positioning marker and other positioning markers based on the actual point coordinates of each positioning marker. Calculate the calibration distance value between each positioning marker and other positioning markers based on the calibration point coordinates of each positioning marker; The number of distance deviations of each positioning identifier is obtained by calculating the absolute value of the difference between the actual distance value and the calibrated distance value between each positioning identifier and other positioning identifiers that is greater than a preset error threshold. The positioning identifiers are sorted in descending order based on the number of distance deviations. Based on the sorted location identifiers, determine whether the number of distance deviations corresponding to each location identifier is greater than or equal to Cnt+1. If it is greater, increment Cnt by 1 and record the location identifier as the problem location identifier. The initial value of Cnt is 0.

9. The method for quality inspection of a visual tracking tool according to claim 1 or 6, characterized in that, The problem location markers for identifying substandard visual tracking tools include: Acquire the first and second images at the angles or depths where tool information is missing; Find the center of each positioning marker of the visual tracking tool in the first image and the second image respectively; For the center of each location marker in the first image and the second image, calculate the number of pixels that satisfy the grayscale threshold for multiple straight lines passing through the center of the location marker. Obtain the number of pixels corresponding to each straight line passing through the center of the location marker. If the pixel difference between the number of pixels corresponding to any two straight lines is greater than the preset pixel difference threshold, then the location marker is designated as the problem location marker.

10. A controller, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, it implements the visual tracking tool quality inspection method as described in any one of claims 1-9.

Images