Infrared target-based light spot connected component detection method and apparatus, and detection device

Abstract

The embodiments of the present application are applicable to the technical fields of computer-aided medical treatment and image processing. Provided are an infrared target-based light spot connected component detection method and apparatus, and a detection device. The method comprising: acquiring a plurality of infrared light spots in an image of a surgical positioning and navigation system; extracting light spot connected components formed by the plurality of infrared light spots; determining coordinates of each pixel point in each light spot connected component, and, on the basis of the coordinates of each pixel point, calculating first center point coordinates of each light spot connected component; allocating a weight coefficient to each pixel point in each light spot connected component; and, on the basis of the weight coefficient, correcting the first center point coordinates, so as to obtain second center point coordinates of each light spot connected component. Using the method can accurately localize the center of each light spot connected component, thereby improving the accuracy of detecting light spot connected components, and improving the stability and accuracy of positioning and navigation during surgery.

Description

Method, apparatus, and detection equipment for detecting connected regions of infrared targets based on light spots

[0001] This application claims priority to Chinese Patent Application No. 202410941680.0, filed on July 12, 2024, entitled "Method, Apparatus and Detection Equipment for Spot Connectivity Detection Based on Infrared Target", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application belongs to the fields of computer-aided medical technology and image processing technology, and in particular relates to a method, apparatus and detection equipment for detecting the connected domain of light spots based on infrared targets. Background Technology

[0003] Surgical navigation is an important application of computer-aided surgery. It helps surgeons select the optimal surgical path, reducing surgical trauma and improving the accuracy, speed, and success rate of the procedure. Currently, the most commonly used surgical navigation system is the infrared optical positioning and tracking system. This system uses two cameras to capture real-time images of the surgical instruments and the infrared target on the lesion. Then, it uses tracking and positioning algorithms to calculate the working position, direction, and movement path of the surgical instruments, and completes the surgery according to the pre-designed and planned route and steps.

[0004] An infrared optical positioning and tracking system consists of an infrared light source and an infrared camera. The infrared light source is composed of an array of infrared LEDs (light-emitting diodes) arranged around the infrared camera. The circular target on the infrared target is made of a film with high infrared reflectivity. During operation, the infrared light source illuminates the circular infrared reflective target on the surgical instrument or lesion, forming a specific target reflective spot pattern in the image captured by the camera. The infrared camera consists of an image sensor, a lens, and an infrared transmission filter, enabling the imaging of the reflective spots on the target. Due to the high infrared reflectivity of the target and the low infrared reflectivity of surrounding objects, a special image can be formed, resulting in a target spot pattern with high brightness and contrast. Multiple target spot patterns can constitute a connected region. By discovering these connected regions and determining their distance relationships, the position and attitude parameters of the surgical instrument can be calculated, achieving positioning and tracking. Generally, during system operation, adjusting the intensity of the infrared light source and the camera parameters can result in a larger grayscale value of the reflected spots in the image captured by the camera. Because the image formed by the reflection from the target has unique characteristics, in practical applications, camera parameters can be adjusted so that the pixel grayscale value in the central area of ​​the reflected spot reaches its highest value, or its brightness is highly saturated. Specifically, when the image grayscale range is 0–255, the grayscale value of the central area of ​​the spot is 255. Meanwhile, the grayscale value or brightness of surrounding objects will be relatively lower. This allows for easy extraction of the connected components of the spot and its center based on the grayscale values, which can then be used for subsequent positioning calculations.

[0005] Traditional methods for calculating the center coordinates of a bright spot's connected domain involve first setting a pixel grayscale threshold and then binarizing the image. Specifically, a reasonable pixel grayscale threshold (e.g., T) is set. Pixels with a grayscale value greater than T are classified as valid bright spots and included in the connected domain center calculation; pixels with a grayscale value less than T are discarded and not included in the calculation. This method has low accuracy and cannot provide more precise center coordinates for the connected domain of a bright spot for surgical navigation. Technical issues

[0006] In view of this, embodiments of this application provide a method, apparatus, and detection device for detecting light spot connected regions based on infrared targets, which can accurately locate the center of the light spot connected region, improve the accuracy of light spot connected region detection, and improve the stability and accuracy of positioning and navigation during surgery. Technical solutions

[0007] The first aspect of this application provides a method for detecting connected components of an infrared target spot, including:

[0008] Multiple infrared spots are acquired in the image of the surgical positioning and navigation system. The multiple infrared spots are formed in the image by illuminating multiple infrared targets in the surgical scene with an infrared light source and reflecting infrared light from the multiple infrared targets.

[0009] Extract the connected regions of each spot formed by the multiple infrared spots;

[0010] Determine the coordinates of each pixel in each of the light spot connected regions, and calculate the coordinates of the first center point of each of the light spot connected regions based on the coordinates of each pixel;

[0011] Assign weight coefficients to each pixel in each connected region of the light spot;

[0012] The coordinates of the first center point are corrected based on the weighting coefficients to obtain the coordinates of the second center point of each of the light spot connected domains.

[0013] A second aspect of this application provides a spot connectivity detection device based on an infrared target, comprising:

[0014] The acquisition module is used to acquire multiple infrared light spots in the image of the surgical positioning and navigation system. The multiple infrared light spots are formed in the image by illuminating multiple infrared targets in the surgical scene with an infrared light source and reflecting infrared light from the multiple infrared targets.

[0015] The extraction module is used to extract the connected domains of each spot formed by the multiple infrared spots;

[0016] The determination module is used to determine the coordinates of each pixel in each of the light spot connected regions;

[0017] The calculation module is used to calculate the coordinates of the first center point of each of the light spot connected regions based on the coordinates of each pixel.

[0018] The allocation module is used to assign weight coefficients to each pixel in each connected domain of the light spot.

[0019] The correction module is used to correct the coordinates of the first center point based on the weighting coefficient to obtain the coordinates of the second center point of each of the light spot connected domains.

[0020] A third aspect of this application provides a detection device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the infrared target-based spot connectivity detection method as described in the first aspect above.

[0021] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the infrared target-based spot connectivity detection method as described in the first aspect above.

[0022] The fifth aspect of this application provides a computer program product that, when run on a computer, causes the computer to execute the infrared target-based spot connectivity detection method described in the first aspect. Beneficial effects

[0023] Compared with the prior art, the embodiments of this application have the following beneficial effects:

[0024] This application embodiment acquires multiple infrared spots in an image. After extracting the connected components of the spots, the coordinates of the first center point of each connected component can be calculated based on the coordinates of each pixel within the connected component, i.e., the initial center point coordinates. Based on this, by assigning weight coefficients (such as grayscale weight coefficients and distance weight coefficients) to each pixel within each connected component, the initially obtained first center point coordinates can be corrected to obtain the second center point coordinates of each connected component, i.e., more accurate center point coordinates. This application embodiment addresses the issue that the edges of infrared reflected spots always have areas of gradual grayscale value changes and exhibit grayscale value jitter. By increasing the weight of pixels with less grayscale value jitter in the localization algorithm, the calculation accuracy of the center point coordinates is improved. This contributes to more accurate, faster, and more reliable connected component recognition and detection, improving the stability and accuracy of positioning and navigation during surgery. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 is a schematic diagram of the gray value distribution of an infrared spot provided in an embodiment of this application;

[0027] Figure 2 is a schematic diagram of a local pixel of a reflected light spot provided in an embodiment of this application;

[0028] Figure 3 is a schematic diagram of a spot connectivity detection method based on an infrared target provided in an embodiment of this application;

[0029] Figure 4 is a schematic diagram of an infrared target provided in an embodiment of this application;

[0030] Figure 5 is a schematic diagram of the operation of an infrared optical positioning and tracking system provided in an embodiment of this application;

[0031] Figure 6 is a schematic diagram of a target image captured by an infrared camera according to an embodiment of this application;

[0032] Figure 7 is a schematic diagram of an infrared target reflected light spot provided in an embodiment of this application;

[0033] Figure 8 is a schematic diagram of correcting the center point coordinates based on the gray-scale weighting coefficient provided in an embodiment of this application;

[0034] Figure 9 is a schematic diagram of correcting the center point coordinates based on a distance weighting coefficient according to an embodiment of this application;

[0035] Figure 10 is a schematic diagram of a spot connectivity detection device based on an infrared target according to an embodiment of this application;

[0036] Figure 11 is a schematic diagram of a detection device provided in an embodiment of this application. Embodiments of the present invention

[0037] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0038] Orthopedic surgical robots assist surgeons in performing operations, and their navigation and control are key technologies. Before surgery, surgeons plan the operation and design the surgical route based on CT, MRI, and other imaging and diagnostic results. During surgery, a target is fixed to the patient's bone (near the lesion) and the robot's manipulator arm. The system then uses a positioning and tracking device to acquire the robot's position and trajectory relative to the lesion, completing the surgical procedure (such as cutting or grinding) according to the pre-planned route. Therefore, the accuracy and stability of the target's positioning and tracking are crucial to the success of the surgery; thus, it is essential to ensure the accuracy and stability of the positioning and tracking device's detection of the center of the reflected light spot at the target's marker point.

[0039] During the experiment, the applicant discovered that the edges of light spots captured by the infrared optical positioning and tracking system exhibit gradual changes in grayscale. This is caused by the incomplete photosensitive area of ​​the image sensor pixels in the edge region. Furthermore, noise generated by mechanical vibrations of the equipment and ambient light (e.g., sunlight, illumination sources, and reflections from surrounding objects) also causes variations in the grayscale of the light spot edges. The aforementioned method of binarizing the image by setting a pixel grayscale threshold T to distinguish whether a light spot belongs to a valid bright spot eligible for calculation does not consider the grayscale changes of pixels at the spot edges, which affects the accuracy of the light spot center calculation. On the other hand, the applicant found that when calculating the coordinates of the light spot center, pixels closer to the center contribute more to the calculation accuracy than pixels farther away. Therefore, this application proposes a light spot connected component detection method. This method introduces weight coefficients for each pixel in the light spot during the calculation process, allowing for correction of the light spot center coordinates calculated using traditional methods based on the different weights of pixels at different locations, thereby improving the accuracy of the light spot connected component center calculation.

[0040] One objective of this application is to optimize the positioning accuracy and stability of a surgical positioning and navigation system. The surgical positioning and navigation system can use an infrared camera to acquire images and detect the center coordinates of a light spot formed by the reflection of a circular target under infrared light illumination. In the implementation of this method, adjusting the sensor gain, camera aperture, and exposure time of the infrared camera can increase the gray-scale gradient at the edge of the reflected light spot, resulting in higher contrast with the background area, thus distinguishing the target light spot from the background object. However, the edge of the infrared reflected light spot always has a gray-scale gradient region, and there is a phenomenon of gray-scale value jitter. Generally, pixels near the center of the light spot have less gray-scale value jitter; therefore, the weight of pixels with less gray-scale value jitter in the positioning algorithm can be increased, thereby improving the accuracy of the center point coordinate calculation.

[0041] Figure 1 shows a schematic diagram of the grayscale value distribution of an infrared spot provided in an embodiment of this application. The infrared reflected spot in the diagram consists of 380 pixels. In Figure 1, the horizontal axis represents the grayscale value, and the vertical axis represents the number of pixels with the current grayscale value. In surgical positioning and navigation applications, the infrared reflected spot collected by the surgical positioning and navigation system has obvious characteristics. That is, as can be seen from Figure 1, more than half of the pixels have a grayscale value of the full value 255.

[0042] Figure 2 shows a schematic diagram of local pixels of a reflected light spot according to an embodiment of this application. Figure 2 shows a partial schematic diagram of a row of pixels passing through the center of the reflection point, obtained by sampling an infrared image. In Figure 2, the horizontal axis represents the horizontal coordinate of the image, and the vertical axis represents the grayscale value of the image. It can be seen that the grayscale value of most areas in Figure 2 reaches the highest value of 255, and they can contribute most of the weight when calculating the center point coordinates. That is, one of the technical means adopted in this embodiment of the application is to reduce the weight of pixels with an edge grayscale value of less than 255 in the calculation of the center point coordinates, and to reduce the weight of pixels far from the center of the light spot in the calculation process.

[0043] Based on the above principles, this application proposes a method for calculating the coordinates of the center point of the light spot connected domain by weighting two main parameters, namely pixel gray level and distance from the center point, in conjunction with the parameter adjustment of the camera, thereby improving the accuracy and stability of calculating the center of the light spot connected domain.

[0044] Specifically, this application embodiment, through the design of a reasonable infrared target and infrared camera, can acquire high-quality infrared images of the target markers (spots). After filtering the background noise, the white connected regions are detected using the region growing method. The required circular and elliptical spots are extracted by combining the quantity and shape characteristics of pixels in the connected regions, and the preliminary coordinates of the center of the connected regions are calculated. Based on these center coordinates, the distance from each pixel in the connected regions to the center of the regions is calculated. This calculation can be performed simultaneously using both distance and pixel grayscale parameters to obtain new center coordinates, thus correcting the center coordinates and achieving more accurate, faster, and more reliable connected region identification and detection.

[0045] The technical solution of this application will be described below through specific embodiments.

[0046] Referring to Figure 3, a schematic diagram of a spot connectivity detection method based on an infrared target provided in an embodiment of this application is shown, which may specifically include the following steps:

[0047] S301. Obtain multiple infrared light spots in the image of the surgical positioning and navigation system. The multiple infrared light spots are formed in the image by illuminating multiple infrared targets in the surgical scene with an infrared light source and reflecting infrared light from the multiple infrared targets.

[0048] It should be noted that this method can be applied to surgical positioning and navigation systems. Specifically, the detection device in the surgical positioning and navigation system, or other electronic devices with corresponding functions, can use this method to perform image processing on the infrared spot captured by the surgical positioning and navigation system during surgery, and accurately calculate the center of the connected region of the spot. The aforementioned detection device or other electronic devices with corresponding functions can be devices with data processing capabilities in the surgical positioning and navigation system, such as processors, processing units, or computing units, etc.

[0049] The surgical positioning and navigation system in this embodiment can be an infrared optical positioning and tracking system, which can consist of an infrared light source and an infrared camera. When the system is working, the infrared light source can illuminate multiple infrared targets in the surgical scene. These targets will reflect infrared light, thereby forming infrared light spots in the image.

[0050] Specifically, the infrared light source can be a circular array of near-infrared LEDs, positioned around the infrared camera. The camera uses an infrared filter, and the transmitted wavelength should strictly match the dominant wavelength of the infrared light source, with a narrow transmission wavelength range that matches the light source's wavelength range as closely as possible. The corresponding infrared target points can be made using a high-infrared-reflectance film. The light intensity of the infrared light source should be sufficiently high to ensure the brightness of the reflected light spot on the target. Simultaneously, by adjusting the image sensor gain, camera aperture, and exposure time, the system can acquire images that ensure both high brightness of the target light spot and limit the grayscale values ​​of background noise pixels to a low range. Generally, the grayscale value of the central region of the reflected light spot in the acquired image is the saturation value; that is, if the image pixel grayscale value range is 0–255, the grayscale value of the central region of the reflected light spot should be saturated, at 255.

[0051] Figure 4 shows a schematic diagram of an infrared target provided in an embodiment of this application. The infrared target shown in Figure 4 includes four circular infrared reflective markers, namely infrared reflective markers 201-204 shown in Figure 4. These four infrared reflective markers are connected by a bracket 205. Typically, the positions of each infrared reflective marker on each infrared target can be specially designed. In this way, multiple spacings can be formed between the markers, allowing each infrared target to form a specific and unique target pattern, thereby enabling the detection, identification, and differentiation of different targets during operation.

[0052] Figure 5 shows a schematic diagram of an infrared optical positioning and tracking system provided in an embodiment of this application. The infrared optical positioning and tracking system shown in Figure 5 includes an infrared light source 501 and an infrared camera 502. The infrared camera 502 can be an infrared binocular camera, and the infrared light source 501 consists of an infrared LED array arranged around the periphery of the infrared binocular camera. During the procedure, by controlling the infrared light source 501 in Figure 5 to illuminate multiple infrared targets 503, multiple infrared light spots can be formed on the image of the infrared camera 502 by the infrared reflective markers on each infrared target 503. For example, for the target in Figure 4 with four infrared reflective markers, four infrared light spots can be formed on the image.

[0053] S302. Extract the connected regions of each spot formed by the multiple infrared spots.

[0054] The purpose of this step is to discover the planar coordinates and number of white pixels within the target spot area. Based on the range of variation in the target spot area, the range of the number of pixels occupied by each connected region is limited. Pixels that are above or below this limit are considered noise interference and are removed.

[0055] Furthermore, since the infrared spot is formed by reflecting infrared light from a circular infrared target, the spot formed in the camera image should be circular or elliptical. Therefore, the shape characteristics of the connected regions can be used to determine whether a region is circular or elliptical. Connected regions that are not circular or elliptical are removed. The result after removal is a circular or elliptical connected region formed by multiple infrared spots.

[0056] In one possible implementation of this application embodiment, before performing this step, each pixel in the image can be subjected to grayscale binarization processing according to a preset pixel threshold, so that the pixel value of each pixel in the image after grayscale binarization processing is a first value or a second value. The aforementioned pixel threshold can be a pixel grayscale threshold.

[0057] Specifically, by performing grayscale binarization on the image based on a pixel grayscale threshold T, pixels with grayscale values ​​higher than the threshold can be assigned a certain value, such as a first value; pixels with grayscale values ​​lower than or equal to the threshold can be assigned another value, such as a second value. The first value can be the same as the image grayscale value in a saturated state, and the second value can be the minimum grayscale value of the image. For example, if the image pixel grayscale value range is 0–255, then the grayscale value of the central region of the reflected light spot should be 255 when saturated. Therefore, the first value can be 255, and the second value can be 0. Thus, in the aforementioned example, by setting a reasonable pixel grayscale threshold T, such as T = 120, pixels with grayscale values ​​higher than the threshold can be assigned a value of 255, and other pixels can be assigned a value of 0. After this processing, under normal circumstances, the grayscale value of most background pixels will become 0, i.e., black; while the white area mainly represents the target's light spot range. Image binarization processing may reveal a small amount of fine particles and fine line-like white noise in the image, in addition to the white target spot. Image erosion and other operations can be used to remove this noise, so that only the white target spot is left in the image.

[0058] S303. Determine the coordinates of each pixel in each of the light spot connected regions, and calculate the coordinates of the first center point of each of the light spot connected regions based on the coordinates of each pixel.

[0059] In this embodiment, the coordinates of the first center point of the connected region of the light spot can be the calculated initial coordinates of the center of the connected region. The coordinates of the first center point of the connected region of the light spot can be calculated using the following formula:

[0060] Where, x i y i Let x and y represent the coordinates of pixel i within the connected region of a certain white light spot. i This represents the x-coordinate value of the pixel, y i This represents the ordinate value of the pixel. and Represents the coordinates of the center point of the connected domain. N is the total number of pixels in the connected component. In this calculation method, all pixels in the connected component have equal weight.

[0061] S304. Assign weight coefficients to each pixel in each of the light spot connected domains.

[0062] In this embodiment, the weighting coefficients assigned to each pixel in the connected component of the light spot may include grayscale weighting coefficients and / or distance weighting coefficients. The grayscale weighting coefficient reflects the magnitude of the grayscale value of each pixel in the calculation of the center point coordinates of the connected component, while the distance weighting coefficient reflects the degree of influence of the distance between each pixel and the initially determined center of the connected component (i.e., the pixel corresponding to the first center point coordinates in the aforementioned example) on the calculation of the center point coordinates.

[0063] In one possible implementation of this application, different weight coefficients can be assigned to each pixel based on its grayscale value or its distance from the center point. For example, the weight coefficient assigned to each pixel can be directly proportional to its grayscale value. For instance, a pixel with a larger grayscale value is assigned a larger weight coefficient. Alternatively, the weight coefficient assigned to each pixel can be inversely proportional to its distance from the center point. For instance, a pixel closer to the center point is assigned a larger weight coefficient.

[0064] In one possible implementation of this application, the grayscale weight coefficient of each pixel in each spot connected region can be determined as the pixel grayscale value of each pixel. That is, the grayscale value of each pixel in the original image can be used as the grayscale weight coefficient of that pixel.

[0065] In another possible implementation of this application, to further reduce the impact of unstable pixels, the grayscale weight coefficient can also be determined based on the grayscale value of each pixel and the pixel threshold used in the aforementioned grayscale binarization process. That is, the grayscale weight coefficient of each pixel in each spot connected region can be determined as the difference between the pixel grayscale value of each pixel and the preset pixel threshold.

[0066] For example, the pixel grayscale value of each pixel is G i The preset pixel threshold is T, and the grayscale weighting coefficient determined in the above manner can be G. i -T.

[0067] In another possible implementation of this application, the distance weight coefficient of each pixel can be determined based on the distance between each pixel and the center point. For example, the distance between each pixel in each connected region of the light spot and the first center point can be determined first, and then the distance weight coefficient of each pixel can be determined based on this distance. The aforementioned distance weight coefficient can be a function value of a function generated based on the calculated distance.

[0068] In this embodiment of the application, any pixel point (x) within the connected domain i ,y i ) and the calculated first center point The distance between them can be represented as D i ,Right now:

[0069] Assuming the pixel threshold for grayscale binarization of the image is T, the calculated connected component contains S pixels. Also assuming the connected component is a perfect circle, and approximating the area of ​​the circle to the number of pixels within the connected component, the radius of the circle can be calculated using the formula for the area of ​​a circle:

[0070] In practical applications, connected components are usually elliptical, which means that D i It may be greater than R or less than R, therefore a function W can be designed. d =f(D i As the distance weighting coefficient, that is, the distance weighting coefficient W d It is a D i Functions related to R.

[0071] In this embodiment of the application, the distance weighting coefficient W d =f(D i This can be represented as:

[0072] S305. Based on the weighting coefficient, the coordinates of the first center point are corrected to obtain the coordinates of the second center point of each of the light spot connected domains.

[0073] In this embodiment, the coordinates of the initially calculated first center point can be corrected based on the assigned grayscale weight coefficient. Alternatively, the coordinates of the first center point can be corrected based on the distance weight coefficient of each pixel. Or, both the grayscale weight coefficient and the distance weight coefficient can be used simultaneously to correct the coordinates of the first center point.

[0074] In one possible implementation of this application embodiment, when correcting the coordinates of the first center point based on the grayscale weighting coefficient, the sum of the grayscale weighting coefficients of each pixel in the connected domain of the light spot can be calculated first to obtain the first sum value. That is:

[0075] Where S1 is the first sum, N is the number of pixels in the connected domain of the light spot, and G i Let G be the grayscale weight coefficient for the i-th pixel. In this example, the grayscale weight coefficient G... i It has the same grayscale value as that pixel.

[0076] Then, the coordinates of each pixel can be weighted and summed using the grayscale weight coefficients of each pixel in the above-mentioned light spot connected domain as weights to obtain the second sum. Based on the first and second sums, the coordinates of the first center point can be corrected to obtain the coordinates of the second center point of each light spot connected domain.

[0077] In this embodiment of the application, the second sum may include a weighted sum of the second horizontal coordinate and a weighted sum of the second vertical coordinate. Therefore, when correcting the coordinates of the first center point based on the first and second sums, the ratios of the weighted sum of the second horizontal coordinate and the weighted sum of the second vertical coordinate to the first sum can be calculated to obtain the horizontal and vertical coordinate values ​​constituting the coordinates of the second center point.

[0078] Specifically, the corrected x-coordinate and y-coordinate values ​​of the second center point can be expressed as follows:

[0079] Among them, (x c ,y c () represents the corrected coordinates of the second center point.

[0080] In another possible implementation of this application, the grayscale weight coefficient of each pixel can be the difference between the grayscale value of the pixel and the pixel threshold used in the grayscale binarization process, that is, the grayscale weight coefficient of the i-th pixel can be expressed as G. i -T. Therefore, the process of correcting the coordinates of the first center point to obtain the coordinates of the second center point can be expressed as:

[0081] This further enhances the weighting of high pixel values ​​in the connected domain while reducing the impact of background noise on the center point coordinate correction.

[0082] In another possible implementation of this application, after determining the distance weight coefficient of each pixel in each spot connected domain, the coordinates of the first center point can be corrected simultaneously based on the grayscale weight coefficient and the distance weight coefficient to obtain the coordinates of the second center point of each spot connected domain.

[0083] In this embodiment of the application, the grayscale weighting coefficient G can be used as a reference. i or G i -T, and distance weighting coefficient W d =f(D i ), determine the weight coefficients of each pixel in the connected component of the light spot. Assume the weight coefficient of the i-th pixel is represented as (G... i -T)·f(D iBased on the weight coefficients of each pixel in the connected component of the light spot, the total weight S2 of each pixel in the connected component of the light spot can be calculated, that is:

[0084] Then, the coordinates of the first center point can be corrected based on the sum of the above weights to obtain the coordinates of the second center point of each spot connected domain.

[0085] In this embodiment, the coordinates of the second center point may include the abscissa and ordinate values ​​of the second center point. When correcting the coordinates of the first center point based on the aforementioned weighted sum, the coordinates of each pixel can be weighted and summed using their respective weight coefficients to obtain a third sum. This third sum may include a third weighted sum of abscissas and a third weighted sum of ordinates. Then, by calculating the ratios of the third weighted sum of abscissas and the third weighted sum of ordinates to the total weighted sum, the abscissa and ordinate values ​​constituting the coordinates of the second center point can be obtained.

[0086] Specifically, the process of correcting the coordinates of the first center point to obtain the coordinates of the second center point can be expressed as follows:

[0087] Among them, (x′ c ,y′ c The coordinates of the second center point are corrected based on the grayscale weight coefficient and the distance weight coefficient.

[0088] The position and orientation of the infrared target can be calculated based on the center point coordinates of each connected domain of the light spot, enabling positioning and navigation during surgery. In this embodiment, different weights are assigned to each pixel based on its grayscale and distance to correct the initially obtained center point coordinates. This significantly reduces the impact of pixel jitter at the edge of the reflection point on the calculation of the reflection target center, improving the stability and accuracy of positioning during surgery.

[0089] To facilitate understanding, a complete example is provided below to introduce the spot connected region detection method provided in this application embodiment, and to explain in detail the process of calculating the coordinates of the center point of the spot connected region using this method.

[0090] This method can be applied to the infrared optical positioning and tracking system shown in Figure 5, which includes an infrared light source 501 and an infrared camera 502. The infrared light source 501 consists of a circular array of infrared light-emitting diodes (LEDs) surrounding the infrared camera 502. The emitted light has a specific wavelength (e.g., 850 nm). After illuminating the target 503, the infrared reflective markers on the target 503 reflect the infrared light, forming an infrared reflective spot. A narrow-band filter with the same wavelength range as the LEDs of the infrared light source 501 is positioned in front of the lens of the infrared camera 502. This filter only responds to the specific LED infrared light, thus filtering out background light of other wavelengths. In this way, the system acquires the bright reflective spot of the target marker, while the background remains relatively dim.

[0091] The target markers used in this method can be represented as shown in Figure 4. The original image acquired by the system is an infrared grayscale image, containing multiple elliptical target marker spots reflecting light, as well as some noise. During processing, an appropriate threshold can be selected to binarize and preprocess the image, filtering out small-particle noise and obvious large-scale noise. Then, based on a region growing detection algorithm, adjacent white pixels are connected to form connected regions, and the position and number of pixels in each connected region are calculated. Finally, ellipse and geometric features are used to determine whether the reflected light spots meet the target marker conditions, and these are selected as candidate target marker spots.

[0092] Figure 6 is a schematic diagram of a target image captured by an infrared camera according to an embodiment of this application; Figure 7 is a schematic diagram of an infrared target reflected light spot according to an embodiment of this application. After obtaining the candidate light spot connected regions, the coordinates of the center point of each connected region can be calculated using the following formula:

[0093] Where, x i y i Let x and x represent the coordinates of the pixel with index i within the connected component. i This represents the x-coordinate value of the pixel, y i This represents the ordinate value of the pixel. and Represents the coordinates of the center point of the connected domain. N is the total number of pixels in the connected component. In this calculation method, all pixels in the connected component have equal weight.

[0094] As shown in Figure 7, there are grayscale variations at the edge of the light spot. This is due to incomplete light reception on the edge pixels and environmental noise. This situation affects the accuracy and stability of the center point coordinate calculation.

[0095] Figure 8 illustrates a schematic diagram of correcting the center point coordinates based on a grayscale weighting coefficient. The grayscale weighting coefficient can be set to the grayscale value G of each pixel. i The difference is obtained by subtracting the pixel threshold (denoted as T) from the grayscale binarization process. For example, if the pixel threshold is set to T = 120, and the grayscale value of the pixel with index 56 in the connected component is G... 56 =185, then the grayscale weight coefficient of this pixel is G. 56 -T = 185 - 120 = 65; the grayscale value of pixel number 98 is G. 98 =255, then the grayscale weight coefficient of this pixel is G. 98 -T = 255 - 120 = 135. Traverse all pixels in the connected component and recalculate the center point coordinates (x, y). c ,y c )for:

[0096] Figure 9 illustrates a schematic diagram of correcting the center point coordinates based on distance weighting coefficients. Note that pixels closer to the center in a connected component contribute more to the accuracy of the center than pixels in the edge regions; therefore, the distance of each pixel relative to the center is weighted. If any pixel (x...) within the connected component... i ,y i ) and center point (x c ,y c The distance between them is D. i Design function W d =f(D i ) is used as the distance weighting coefficient. Where:

[0097] Comprehensive grayscale weighting coefficient G i -T and distance weighting coefficient W d =f(D i The total weight is S2, and then the center point (x′) is calculated. c ,y′ c )for:

[0098] (x′ c ,y′ c The coordinates of the center point of the connected domain of the light spot after weighting by gray level and distance are given.

[0099] The spot connectivity detection method provided in this application allows for targeted selection of system parameters. For example, the high reflectivity of the infrared target film, the gain of the image sensor, the camera aperture, and the exposure time ensure that the acquired image maintains high brightness of the target spot while limiting the grayscale values ​​of background objects to a low range. This results in extremely high image contrast. By setting a reasonable pixel threshold T in the grayscale binarization process, the reflected spot of the target is highlighted, while background images unrelated to positioning are blurred, reducing interference from the surgical environment and surrounding light sources and reflective objects on target detection. Unlike other industrial and civilian applications, surgical navigation requires the highest possible update speed. The method provided in this application originates from the applicant's research and experiments on infrared optical positioning and tracking systems. The entire process of calculating the center point coordinates is accurate and simple, ensuring a high refresh rate for positioning. In the applicant's actual experiments, using this method, the system's refresh rate can reach over 170 frames per second. In the specific calculation process, the mathematical formulas and processing involved in this method are relatively simple, making it easy to implement on different processors (such as FPGAs). This also ensures the effective execution of the algorithm, the effective implementation of system functions, and the effective transmission and display of data. The specific algorithm proposed in this application considers the grayscale value changes of pixels at the edge of the white light spot reflected from the infrared target marker, thus improving the accuracy and stability of the light spot center point extraction. Based on the distance of each pixel in the connected domain of the light spot formed by the reflection from the infrared target marker relative to the center (centroid) of the connected domain, the weight of each pixel in the calculation process of the center of the connected domain is designed, improving the accuracy and stability of the center extraction. This reduces the interference of people and objects in the surrounding scene and their movement on the target detection, helping to improve the stability and accuracy of positioning and navigation during surgery.

[0100] It should be noted that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0101] Referring to Figure 10, a schematic diagram of a spot connectivity detection device based on an infrared target according to an embodiment of this application is shown. Specifically, it may include an acquisition module 1001, an extraction module 1002, a determination module 1003, a calculation module 1004, an allocation module 1005, and a correction module 1006, wherein:

[0102] The acquisition module 1001 is used to acquire multiple infrared light spots in the image of the surgical positioning and navigation system. The multiple infrared light spots are formed in the image by illuminating multiple infrared targets in the surgical scene with an infrared light source and reflecting infrared light from the multiple infrared targets.

[0103] Extraction module 1002 is used to extract each connected region of the infrared light spots formed by the plurality of infrared light spots;

[0104] The determining module 1003 is used to determine the coordinates of each pixel in each of the light spot connected regions;

[0105] Calculation module 1004 is used to calculate the coordinates of the first center point of each of the light spot connected regions based on the coordinates of each pixel.

[0106] The allocation module 1005 is used to allocate weight coefficients to each pixel in each connected domain of the light spot.

[0107] The correction module 1006 is used to correct the coordinates of the first center point based on the weight coefficient to obtain the coordinates of the second center point of each of the light spot connected domains.

[0108] In one possible implementation of this application embodiment, the weighting coefficient may include a grayscale weighting coefficient, and the allocation module 1005 may specifically be used for:

[0109] The grayscale weight coefficient of each pixel in each of the light spot connected domains is determined as the pixel grayscale value of each pixel.

[0110] In one possible implementation of this application embodiment, the correction module 1006 may specifically be used for:

[0111] Calculate the sum of the grayscale weight coefficients of each pixel in the connected domain of the light spot to obtain the first sum value;

[0112] Using the grayscale weight coefficient of each pixel in the connected domain of the light spot as the weight, the coordinates of each pixel are weighted and summed to obtain the second sum value;

[0113] Based on the first sum and the second sum, the coordinates of the first center point are corrected to obtain the coordinates of the second center point of each of the light spot connected domains.

[0114] In one possible implementation of this application embodiment, the second sum value may include a second weighted sum value of the horizontal axis and a second weighted sum value of the vertical axis, and the correction module 1006 may also be used for:

[0115] Calculate the weighted sum of the second horizontal coordinate and the ratio of the weighted sum of the second vertical coordinate to the first sum, respectively, to obtain the horizontal and vertical coordinate values ​​that constitute the coordinates of the second center point.

[0116] In one possible implementation of this application embodiment, the device may further include a processing module;

[0117] The processing module is used to perform grayscale binarization processing on each pixel in the image according to a preset pixel threshold; wherein, the pixel value of each pixel in the image after grayscale binarization processing can be a first value or a second value.

[0118] Accordingly, the allocation module 1005 can also be used for:

[0119] The grayscale weight coefficient of each pixel in each of the light spot connected regions is determined to be the difference between the pixel grayscale value of each pixel and the preset pixel threshold.

[0120] In another possible implementation of this application embodiment, the weighting coefficient may further include a distance weighting coefficient, and the correction module 1006 may further be used for:

[0121] Determine the distance weighting coefficients for each pixel in each of the light spot connected components;

[0122] The coordinates of the first center point are corrected based on the grayscale weight coefficient and the distance weight coefficient to obtain the coordinates of the second center point of each of the light spot connected domains.

[0123] In another possible implementation of this application embodiment, the correction module 1006 may also be used for:

[0124] Determine the distance between each pixel in each of the light spot connected regions and the first center point;

[0125] The distance weight coefficient for each pixel is determined based on the distance.

[0126] In another possible implementation of this application embodiment, the correction module 1006 may also be used for:

[0127] Based on the grayscale weight coefficient and the distance weight coefficient, determine the weight coefficient of each pixel in the connected domain of the light spot;

[0128] Calculate the total weight of each pixel in the connected region of the light spot based on the weight coefficient of each pixel in the connected region of the light spot.

[0129] The coordinates of the first center point are corrected based on the sum of the weights to obtain the coordinates of the second center point of each of the light spot connected domains.

[0130] In another possible implementation of this application embodiment, the coordinates of the second center point may include the abscissa and ordinate values ​​of the second center point, and the correction module 1006 may also be used for:

[0131] Using the weight coefficient of each pixel as the weight, the coordinates of each pixel are weighted and summed to obtain a third sum value, which includes the third horizontal coordinate weighted sum value and the third vertical coordinate weighted sum value.

[0132] Calculate the ratios of the weighted sum of the third horizontal coordinate and the weighted sum of the third vertical coordinate to the total weight, respectively, to obtain the horizontal and vertical coordinates that constitute the coordinates of the second center point.

[0133] This application provides a spot connectivity detection device based on an infrared target, which can be the detection device or one or more functional components of the detection device described in the foregoing embodiments. Using this device, the steps in the foregoing method embodiments can be implemented.

[0134] As the apparatus embodiments are basically similar to the method embodiments, they are described in a relatively simple manner. For relevant details, please refer to the description in the method embodiment section.

[0135] Referring to FIG11, a schematic diagram of a detection device provided in an embodiment of this application is shown. This detection device may be one of the components of the infrared optical positioning and tracking system in the foregoing embodiments. As shown in FIG11, the detection device 1100 in this embodiment includes: a processor 1110, a memory 1120, and a computer program 1121 stored in the memory 1120 and executable on the processor 1110. When the processor 1110 executes the computer program 1121, it implements the steps in the various embodiments of the above-described infrared target-based spot connectivity detection method, such as steps S301 to S305 shown in FIG3. Alternatively, when the processor 1110 executes the computer program 1121, it implements the functions of each module / unit in the above-described device embodiments, such as the functions of modules 1001 to 1006 shown in FIG10.

[0136] For example, the computer program 1121 can be divided into one or more modules / units, which are stored in the memory 1120 and executed by the processor 1110 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which can be used to describe the execution process of the computer program 1121 in the detection device 1100. For example, the computer program 1121 can be divided into an acquisition module, an extraction module, a determination module, a calculation module, an allocation module, and a correction module, with the specific functions of each module as follows:

[0137] The acquisition module is used to acquire multiple infrared light spots in the image of the surgical positioning and navigation system. The multiple infrared light spots are formed in the image by illuminating multiple infrared targets in the surgical scene with an infrared light source and reflecting infrared light from the multiple infrared targets.

[0138] The extraction module is used to extract the connected domains of each spot formed by the multiple infrared spots;

[0139] The determination module is used to determine the coordinates of each pixel in each of the light spot connected regions;

[0140] The calculation module is used to calculate the coordinates of the first center point of each of the light spot connected regions based on the coordinates of each pixel.

[0141] The allocation module is used to assign weight coefficients to each pixel in each connected domain of the light spot.

[0142] The correction module is used to correct the coordinates of the first center point based on the weighting coefficient to obtain the coordinates of the second center point of each of the light spot connected domains.

[0143] The detection device 1100 can be a device capable of implementing the various steps or having corresponding functions in the aforementioned method embodiments. The detection device 1100 can be an embedded system, a desktop computer, a cloud server, or other computing device. The detection device 1100 may include, but is not limited to, a processor 1110 and a memory 1120. Those skilled in the art will understand that Figure 11 is merely an example of the detection device 1100 and does not constitute a limitation on the detection device 1100. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the detection device 1100 may also include input / output devices, network access devices, buses, etc.

[0144] The processor 1110 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0145] The memory 1120 can be an internal storage unit of the detection device 1100, such as a hard disk or memory of the detection device 1100. The memory 1120 can also be an external storage device of the detection device 1100, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the detection device 1100. Furthermore, the memory 1120 can include both internal and external storage units of the detection device 1100. The memory 1120 is used to store the computer program 1121 and other programs and data required by the detection device 1100. The memory 1120 can also be used to temporarily store data that has been output or will be output.

[0146] This application also discloses a detection device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the infrared target-based spot connectivity detection method as described in the foregoing embodiments.

[0147] This application also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the infrared target-based spot connectivity detection method as described in the foregoing embodiments.

[0148] This application also discloses a computer program product that, when run on a computer, causes the computer to execute the infrared target-based spot connectivity detection method described in the foregoing embodiments.

[0149] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for detecting connected components of an infrared target spot, characterized in that, include: Multiple infrared spots are acquired in the image of the surgical positioning and navigation system. The multiple infrared spots are formed in the image by illuminating multiple infrared targets in the surgical scene with an infrared light source and reflecting infrared light from the multiple infrared targets. Extract the connected regions of each spot formed by the multiple infrared spots; Determine the coordinates of each pixel in each of the light spot connected regions, and calculate the coordinates of the first center point of each of the light spot connected regions based on the coordinates of each pixel; Assign weight coefficients to each pixel in each connected region of the light spot; The coordinates of the first center point are corrected based on the weighting coefficients to obtain the coordinates of the second center point of each of the light spot connected domains.

2. The method according to claim 1, characterized in that, The weighting coefficients include grayscale weighting coefficients, and the assignment of weighting coefficients to each pixel in each connected component of the light spot includes: The grayscale weight coefficient of each pixel in each of the light spot connected domains is determined as the pixel grayscale value of each pixel.

3. The method according to claim 2, characterized in that, The step of correcting the coordinates of the first center point based on the weighting coefficient to obtain the coordinates of the second center point of each of the connected domains of the light spot includes: Calculate the sum of the grayscale weight coefficients of each pixel in the connected domain of the light spot to obtain the first sum value; Using the grayscale weight coefficient of each pixel in the connected domain of the light spot as the weight, the coordinates of each pixel are weighted and summed to obtain the second sum value; Based on the first sum and the second sum, the coordinates of the first center point are corrected to obtain the coordinates of the second center point of each of the light spot connected domains.

4. The method according to claim 3, characterized in that, The second sum includes a weighted sum of the second horizontal coordinate and a weighted sum of the second vertical coordinate. The step of correcting the coordinates of the first center point based on the first and second sums to obtain the coordinates of the second center point of each of the connected domains of the light spot includes: Calculate the weighted sum of the second horizontal coordinate and the ratio of the weighted sum of the second vertical coordinate to the first sum, respectively, to obtain the horizontal and vertical coordinate values ​​that constitute the coordinates of the second center point.

5. The method according to claim 3, characterized in that, Before calculating the coordinates of the first center point of each connected region of the light spot based on the coordinates of each pixel, the method further includes: According to a preset pixel threshold, each pixel in the image is subjected to grayscale binarization; wherein, the pixel value of each pixel in the image after grayscale binarization is a first value or a second value; Accordingly, assigning weight coefficients to each pixel in each of the light spot connected regions further includes: The grayscale weight coefficient of each pixel in each of the light spot connected regions is determined to be the difference between the pixel grayscale value of each pixel and the preset pixel threshold.

6. The method according to any one of claims 2-5, characterized in that, The weighting coefficients also include distance weighting coefficients. The step of correcting the coordinates of the first center point based on the weighting coefficients to obtain the coordinates of the second center point of each of the connected domains of the light spot includes: Determine the distance weighting coefficients for each pixel in each of the light spot connected components; The coordinates of the first center point are corrected based on the grayscale weight coefficient and the distance weight coefficient to obtain the coordinates of the second center point of each of the light spot connected domains.

7. The method according to claim 6, characterized in that, Determining the distance weight coefficients for each pixel in each of the connected components of the light spot includes: Determine the distance between each pixel in each of the light spot connected regions and the first center point; The distance weight coefficient for each pixel is determined based on the distance.

8. The method according to claim 7, characterized in that, The step of correcting the coordinates of the first center point based on the grayscale weighting coefficient and the distance weighting coefficient to obtain the coordinates of the second center point of each of the connected domains of the light spot includes: Based on the grayscale weight coefficient and the distance weight coefficient, determine the weight coefficient of each pixel in the connected domain of the light spot; Calculate the total weight of each pixel in the connected region of the light spot based on the weight coefficient of each pixel in the connected region of the light spot. The coordinates of the first center point are corrected based on the sum of the weights to obtain the coordinates of the second center point of each of the light spot connected domains.

9. The method according to claim 8, characterized in that, The second center point coordinates include the x-coordinate and y-coordinate values ​​of the second center point. The step of correcting the first center point coordinates based on the weighted sum to obtain the second center point coordinates for each of the light spot connected regions includes: Using the weight coefficient of each pixel as the weight, the coordinates of each pixel are weighted and summed to obtain a third sum value, which includes the third horizontal coordinate weighted sum value and the third vertical coordinate weighted sum value. Calculate the ratios of the weighted sum of the third horizontal coordinate and the weighted sum of the third vertical coordinate to the total weight, respectively, to obtain the horizontal and vertical coordinates that constitute the coordinates of the second center point.

10. A spot connectivity detection device based on an infrared target, characterized in that, include: The acquisition module is used to acquire multiple infrared light spots in the image of the surgical positioning and navigation system. The multiple infrared light spots are formed in the image by illuminating multiple infrared targets in the surgical scene with an infrared light source and reflecting infrared light from the multiple infrared targets. The extraction module is used to extract the connected domains of each spot formed by the multiple infrared spots; The determination module is used to determine the coordinates of each pixel in each of the light spot connected regions; The calculation module is used to calculate the coordinates of the first center point of each of the light spot connected regions based on the coordinates of each pixel. The allocation module is used to assign weight coefficients to each pixel in each connected domain of the light spot. The correction module is used to correct the coordinates of the first center point based on the weighting coefficient to obtain the coordinates of the second center point of each of the light spot connected domains.

11. A detection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the infrared target-based spot connectivity detection method as described in any one of claims 1-9.

12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the infrared target-based spot connectivity detection method as described in any one of claims 1-9.

Images