Perspective image-based body surface positioning ball matching method and device

Abstract

The application relates to the technical field of medical image processing, and discloses a body surface positioning ball matching method and equipment based on perspective images, which comprises the following steps: acquiring a plurality of perspective images obtained by rotating scanning a target object; respectively detecting body surface positioning balls of the plurality of perspective images to obtain body surface positioning ball position data of the plurality of perspective images; dividing motion trajectories of the body surface positioning ball position data of the plurality of perspective images to obtain a plurality of classified motion trajectories; supplementing missing position data of each motion trajectory based on the plurality of classified motion trajectories to obtain target position data of the plurality of classified motion trajectories; and matching the target position data with perspective image frames based on the categories of the motion trajectories to obtain position data of the body surface positioning balls in different frames of perspective images. The application can accurately match each body surface positioning ball with different frames of perspective images and accurately output position data of each body surface positioning ball in different frames of perspective images.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and more specifically, to a method and device for matching small balls located on the body surface based on perspective images. Background Technology

[0002] The surface positioning strip is an auxiliary positioning component made of reflective or luminescent materials, used in the optical navigation module of surgical robots. The optical navigation module captures the reflected signals from the reflective marks on the surface positioning strip, and obtains the three-dimensional coordinates of the sphere in space through 3D reconstruction. Simultaneously, it obtains the sphere's coordinates from medical images, performs spatial registration between the two to obtain a transformation matrix, thereby converting the target lesion coordinates into three-dimensional space to guide the robotic arm in precise positioning. In recent years, CBCT (Cone Beam Computed Tomography) has become increasingly popular in intraoperative navigation due to its advantages such as low radiation dose and short scan time.

[0003] CBCT equipment acquires multi-angle fluoroscopic images through rotational scanning, from which the position of a localized sphere on the body surface can be reconstructed in three-dimensional space. However, because the projected position and shape of the same sphere continuously change in images from different angles, it is difficult to accurately match the localized sphere on the body surface in different images. Summary of the Invention

[0004] In view of this, the present invention provides a method and device for matching body surface positioning balls based on perspective images, so as to solve the problem of difficulty in accurately matching body surface positioning balls in different images.

[0005] In a first aspect, the present invention provides a method for matching surface positioning balls based on perspective images. The method includes: acquiring multiple frames of perspective images obtained by rotating and scanning a target object, wherein the target object is provided with multiple surface positioning balls; detecting surface positioning balls in the multiple frames of perspective images to obtain position data of surface positioning balls in the multiple frames of perspective images; dividing the position data of surface positioning balls in the multiple frames of perspective images into motion trajectories to obtain multiple categories of motion trajectories; supplementing the missing position data of each motion trajectory based on the multiple categories of motion trajectories to obtain target position data of multiple categories of motion trajectories; and matching the target position data with perspective image frames based on the category of motion trajectory to obtain position data of surface positioning balls in different frames of perspective images.

[0006] In one optional implementation, the motion trajectory of the surface positioning ball position data of multiple frames of perspective images is divided to obtain multiple categories of motion trajectories, including: dividing the surface positioning ball position data with similar vertical coordinates into a group based on the distribution characteristics of the surface positioning ball position data to obtain multiple groups of surface positioning ball position data; and performing linear fitting on the multiple groups of surface positioning ball position data to obtain multiple categories of motion trajectories.

[0007] In one optional implementation, the motion trajectory of the body surface positioning ball position data of the multi-frame perspective images is divided to obtain multiple categories of motion trajectories. The method further includes: performing cluster analysis on the body surface positioning ball position data of the multi-frame perspective images to obtain clustering results; and determining the position data in the same category as the position data of the same body surface positioning ball based on the clustering results to obtain multiple categories of motion trajectories.

[0008] In one optional implementation, cluster analysis is performed on the position data of surface positioning balls in multi-frame perspective images to obtain clustering results, including: determining the number of cluster categories based on the number of surface positioning balls; randomly selecting a corresponding number of position data points from the surface positioning ball position data as initial cluster centers based on the number of cluster categories, obtaining multiple initial cluster centers; calculating the distance from each position data point to each initial cluster center based on the position of the multiple initial cluster centers, and assigning each position data point to the nearest initial cluster center, obtaining multiple position data point clusters; calculating the mean of the position data of all position data points in each position data point cluster, obtaining multiple second cluster centers; determining whether a preset convergence condition is met based on the multiple second cluster centers; if the multiple second cluster centers meet the preset convergence condition, then the multiple position data point clusters are used as the clustering result; if the multiple second cluster centers do not meet the preset convergence condition, then the iteration continues based on the multiple second cluster centers until the preset convergence condition is met, obtaining the clustering result.

[0009] In one optional implementation, cluster analysis is performed on the position data of the body surface positioning balls in multi-frame perspective images to obtain clustering results. The method further includes: constructing a corresponding number of linear regression models based on the number of body surface positioning balls; using each linear regression model to perform initial line fitting on the position data of the body surface positioning balls in multi-frame perspective images to obtain multiple straight lines; calculating the distance from each position data point to the multiple straight lines based on the position data of the body surface positioning balls in multi-frame perspective images to obtain distance data for each position data point; assigning each position data point to the nearest straight line based on the distance data to obtain multiple combinations of straight line position data points; calculating the sum of squared distances for each combination based on the combinations of straight line position data points to obtain distance data for each combination; determining whether the distance data for each combination meets a preset distance condition; if the distance data for each combination meets the preset distance condition, then using the combination of multiple straight line position data points as the clustering result; if the distance data for each combination does not meet the preset distance condition, then updating the corresponding linear regression model based on the combination of multiple straight line position data points until a preset convergence condition is met to obtain the clustering result.

[0010] In one optional implementation, missing position data for each motion trajectory is supplemented based on multiple categories of motion trajectories to obtain target position data for multiple categories of motion trajectories. This includes: calculating missing position data in the corresponding category of motion trajectory using an interpolation algorithm based on the position data of each category of motion trajectory to obtain missing position data; and combining the position data of each category of motion trajectory with the corresponding missing position data to obtain target position data for multiple categories of motion trajectories.

[0011] In one optional implementation, the target position data is matched with perspective image frames based on the category of the motion trajectory to obtain the position data of the body surface positioning ball in different frames of perspective images. This includes: identifying frames with missing position data in multiple categories of motion trajectories based on the position data of the motion trajectory to obtain missing frame information for multiple categories of motion trajectories; and matching the target position data of each category of motion trajectory with the corresponding perspective image frames based on the missing frame information to obtain the position data of the body surface positioning ball in different frames of perspective images.

[0012] In one optional implementation, surface localization ball detection is performed on multiple frames of perspective images to obtain surface localization ball position data of the multiple frames of perspective images, including: extracting gradient information based on pixels in the multiple frames of perspective images to obtain multiple frames of gradient images; performing shape detection based on the multiple frames of gradient images to obtain shape data, the shape data including the center coordinates and radius information of the circle, and using the center coordinates as the surface localization ball position data of the multiple frames of perspective images.

[0013] In one optional implementation, before classifying the motion trajectory of the ball position data on the body surface of the multi-frame perspective images to obtain multiple categories of motion trajectories, the method further includes: obtaining the radius information of circles in the multi-frame perspective images; and deleting the center coordinates of circles whose radius information does not meet the radius threshold based on a radius threshold, thereby obtaining the filtered ball position data on the body surface of the multi-frame perspective images. Secondly, the present invention provides a body surface localization ball matching device based on perspective images, comprising: a processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to perform the above-described body surface localization ball matching method based on perspective images.

[0014] The present invention provides a method for matching small balls for body surface localization based on perspective images. First, multiple frames of perspective images of the target object are acquired. Then, independent ball detection is performed on each frame of perspective image, extracting the two-dimensional position data of all visible small balls, thus achieving automatic extraction of the positions of small balls in the perspective images. Next, based on the projection motion law followed by the small balls during rotational scanning, the discrete position data points in each frame are divided into motion trajectories according to the small ball to which they belong, obtaining the category motion trajectory of each small ball. Then, missing position data in each category motion trajectory due to occlusion or field-of-view limitations is supplemented, ensuring that each trajectory has complete position data under all scanning angles, effectively overcoming the interference of missing data on subsequent processing. Finally, the supplemented target position data is matched with the corresponding perspective image frames, so that each position data obtains the frame and scanning angle information, thereby accurately outputting the position data of each small ball in different frames of perspective images. Even if the small balls are not in the image field of view for several frames, continuous data can still be formed, ensuring the consistency of the ball's identity between frames. In summary, this embodiment, without relying on manual labeling, achieves automatic matching and cross-frame correspondence of surface-positioned balls in CBCT fluoroscopic image sequences through a processing flow of ball detection, trajectory division, missing data supplementation, and frame matching. It can effectively handle data loss problems caused by occlusion, field of view limitation, etc., and provide accurate input data for subsequent 3D reconstruction and surgical navigation positioning. Attached Figure Description

[0015] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating a method for locating and matching small balls on the body surface based on perspective images, according to an embodiment of the present invention. Figure 2 This is a data distribution diagram of the center position of the positioning ball on the body surface according to an embodiment of the present invention; Figure 3 This is a distribution diagram of the center positions of the screened body surface positioning balls according to an embodiment of the present invention; Figure 4 This is a motion trajectory diagram of the surface positioning ball according to an embodiment of the present invention; Figure 5 This is a motion trajectory diagram of the surface positioning ball after interpolation according to an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] According to an embodiment of the present invention, a method for locating small balls on the body surface based on perspective images is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0019] This embodiment provides a method for locating and matching small balls on the body surface based on perspective images, which can be used in the aforementioned mobile terminals, such as mobile phones and tablets. Figure 1 This is a flowchart of a body surface localization ball matching method based on perspective images according to an embodiment of the present invention, as follows: Figure 1 As shown, the process includes the following steps: Step S101: Obtain multiple perspective images of the target object obtained by rotating and scanning the target object. The target object has multiple surface positioning balls.

[0020] In this embodiment, the acquisition of multi-frame fluoroscopic images is as follows: A surface positioning band is fixed on the patient's (target object's) body surface. Multiple reflective spheres (non-collinearly distributed) are placed on this band. These spheres exhibit high contrast in fluoroscopic imaging, facilitating stable detection and positioning in subsequent image processing. During imaging, the CBCT device is activated, and its cone-beam X-ray source and two-dimensional flat panel detector are controlled to rotate and scan around the patient at a set angle. Two-dimensional fluoroscopic images are continuously acquired at a fixed frame rate during rotation, while the scanning angle corresponding to each frame is recorded in real time. This results in a multi-frame fluoroscopic image sequence covering multiple angles of the patient. The multi-frame fluoroscopic image sequence contains the projection information of the surface positioning spheres at different angles.

[0021] Step S102: Perform surface localization ball detection on multiple frames of perspective images to obtain surface localization ball position data for multiple frames of perspective images.

[0022] In this embodiment, the purpose of detecting surface-positioned spheres is to extract the two-dimensional center coordinates of all visible spheres from each frame of the perspective image. Before sphere detection, each frame of the perspective image can be preprocessed, including filtering and denoising, and grayscale adjustment, to suppress image noise and enhance the contrast between the spheres and the background. Then, an image target detection method can be used to extract the two-dimensional position information of all visible spheres from each frame of the perspective image, utilizing the circular features of the surface-positioned spheres in the perspective image, and output it in the form of center coordinates. The detection process for each frame is independent of each other, and the detection result contains the spatial position information of the surface-positioned spheres within a single frame.

[0023] Step S103: Divide the motion trajectory of the ball position data on the body surface of the multi-frame perspective image into multiple categories of motion trajectories.

[0024] In this embodiment, the purpose of dividing the motion trajectory is to group the independently detected position data in each frame of fluoroscopic images according to the surface positioning ball to which it belongs, that is, to determine which position data points come from the same surface positioning ball. Since the ball on the surface positioning band remains stationary with the patient during CBCT rotational scanning, while the X-ray source and detector rotate around the patient, the projected position of the same ball at different angles follows a specific motion pattern. Therefore, based on this distribution characteristic, this embodiment analyzes the position data points detected in all frames, grouping data points with similar motion patterns into the same category, forming multiple categories of motion trajectories, with each category uniquely corresponding to a surface positioning ball.

[0025] Step S104: Based on multiple categories of motion trajectories, supplement the missing location data of each motion trajectory to obtain target location data of multiple categories of motion trajectories.

[0026] In this embodiment, because some balls in the perspective images may not be detected due to occlusion or limited field of view, there may be gaps in the position data of each type of motion trajectory in the angular dimension. The purpose of data supplementation is to fill these gaps so that each trajectory has complete position data at all angles.

[0027] Step S105: Based on the category of motion trajectory, the target position data is matched with the perspective image frame to obtain the position data of the surface positioning ball in different frames of perspective images.

[0028] In this embodiment, after the location data is supplemented, the target location data of each trajectory is complete in spatial coordinates. However, these location data have not yet been correlated with their respective perspective image frames. That is, although the coordinate values ​​of all missing locations have been filled in for each trajectory, it is still unknown which perspective image frame each coordinate value corresponds to. This embodiment matches the target location data with the perspective image frames, so that each location data is associated with the corresponding scanning angle information, thereby accurately obtaining the location data of each body surface positioning ball in different perspective image frames. The above matching results can be used for the three-dimensional reconstruction of the body surface positioning ball. The three-dimensional spatial coordinates of the body surface positioning ball are calculated using the scanning angle information corresponding to each image frame, and then used for surgical navigation and positioning.

[0029] The method for matching spheres for body surface localization based on perspective images provided in this embodiment first acquires multiple frames of perspective images of the target object; then, it performs independent sphere detection on each frame of perspective images, extracting the two-dimensional position data of all visible spheres, thus achieving automatic extraction of the positions of spheres for body surface localization in the perspective images; next, based on the projection motion law followed by the spheres during rotational scanning, it divides the discrete position data points in each frame into motion trajectories according to the sphere to which they belong, obtaining the category motion trajectory of each sphere; then, it supplements the missing position data in each category motion trajectory due to occlusion or field of view limitation, so that each trajectory has complete position data under all scanning angles, effectively overcoming the interference of missing data on subsequent processing; finally, it matches the supplemented target position data with the corresponding perspective image frames, so that each position data obtains the frame and scanning angle information, thereby accurately outputting the position data of each sphere for body surface localization in different frames of perspective images, even if the spheres are not in the image field of view for several frames, continuous data can still be formed, ensuring the consistency of the sphere identity between frames. In summary, this embodiment, without relying on manual labeling, achieves automatic matching and cross-frame correspondence of surface-positioned balls in CBCT fluoroscopic image sequences through a processing flow of ball detection, trajectory division, missing data supplementation, and frame matching. It can effectively handle data loss problems caused by occlusion, field of view limitation, etc., and provide accurate input data for subsequent 3D reconstruction and surgical navigation positioning.

[0030] In some optional implementations, before dividing the motion trajectory of the surface positioning ball position data of the multi-frame perspective images into multiple categories of motion trajectories in step S103 above, the method further includes: Step 1: Obtain the radius information of the circle in the multi-frame perspective image.

[0031] In this embodiment, after step S102 completes the detection of the small ball on the body surface, each circular target detected in each frame of perspective image corresponds to the center coordinates and radius information. Based on the center coordinates and radius information of all circular targets detected in each frame, a system can be generated as follows: Figure 2 The diagram shows the data distribution of the center positions of the circles, with the horizontal and vertical axes representing pixel positions. From... Figure 2 As can be seen, there are randomly distributed points in the image. These points mainly originate from the circular projections formed by non-target structures such as bone edges and tissue interfaces in the perspective image, as well as false positive detection results caused by image noise and reconstruction artifacts. These false positive points are mixed with the projection points of the real ball, which seriously interferes with the recognition and trajectory division of the real ball.

[0032] Step 2: Based on the radius threshold, delete the center coordinates of circles whose radius information does not meet the radius threshold, and obtain the surface positioning ball position data of the filtered multi-frame perspective images.

[0033] This embodiment, based on the actual physical size of the surface positioning ball and its imaging size in the perspective image, presets a reasonable radius threshold range. It then individually judges the radius information of each circular target, deleting those whose radius information does not meet the preset threshold range as false positives. The center coordinates of the circular targets whose radius information meets the threshold range are retained as the filtered surface positioning ball position data. The resulting data is as follows: Figure 3 The data distribution map of the center position shown (the horizontal and vertical axes represent pixel positions) effectively eliminates false positive targets, and the movement trajectory of the ball with real body surface positioning is more clearly distinguishable, providing a clean data foundation for subsequent trajectory division.

[0034] This embodiment first extracts the radius information of each circular target from the detection results. Then, based on the actual size of the spheres located on the body surface, a preset radius threshold range is established. Targets whose radii do not meet the threshold range are discarded as false positives. Through the above processing, interference targets caused by non-spherical structures such as skeletal edges, image artifacts, and noise can be effectively filtered out, significantly reducing the data processing burden and misclassification risk during subsequent trajectory segmentation, improving the accuracy of trajectory segmentation, and providing clean and reliable position data for subsequent steps.

[0035] In some optional implementations, the process of dividing the motion trajectory of the ball position data on the body surface of multiple perspective images into multiple categories of motion trajectories in step S103 specifically includes: Step S103a1: Based on the distribution characteristics of the body surface positioning ball position data, the body surface positioning ball position data with similar vertical coordinates are divided into a group to obtain multiple groups of body surface positioning ball position data.

[0036] like Figure 3As shown, the scatter plot distribution formed by the position data of the surface positioning balls detected in each frame of fluoroscopic images reveals that each position data point exhibits several approximately horizontal band-like trajectories. This distribution pattern is determined by the geometric characteristics of CBCT rotational scanning: the balls on the surface positioning band remain stationary with the patient, while the X-ray source and detector rotate around the patient with the vertical axis as the axis. Therefore, in the fluoroscopic image, the longitudinal coordinate (i.e., head-to-toe direction) of the same surface positioning ball remains essentially unchanged during rotation, only changing continuously with the rotation angle in the horizontal coordinate direction; however, different balls exhibit significant differences in their longitudinal coordinates due to their different longitudinal positions on the body surface. Based on this pattern, it can be assumed that position data points with similar longitudinal coordinates originate from the same surface positioning ball. Therefore, in this embodiment, all position data points are categorized according to the distribution characteristics of their longitudinal coordinates, grouping position data points with similar longitudinal coordinates into the same group, thus obtaining multiple sets of position data, each group corresponding to the projection trajectory of a surface positioning ball.

[0037] Step S103a2: Perform linear fitting on multiple sets of surface positioning ball position data to obtain multiple categories of motion trajectories.

[0038] In this embodiment, since the projection points of the same ball positioned on the body surface are basically constant in the vertical direction and continuously change in the horizontal direction at different angles, its trajectory is approximately a straight line. Therefore, by performing straight line fitting (e.g., using the least squares method) on each group of position data obtained in step S103a1, multiple fitted straight lines can be obtained. Each straight line corresponds to the projection change trajectory of a ball in different frames, thereby achieving automatic trajectory differentiation.

[0039] It should be noted that, Figure 3 The illustration shows an example of three types of trajectories corresponding to three surface positioning balls. This diagram is only used to illustrate the trajectory division principle of this embodiment and does not represent the actual number of balls on the surface positioning band. In practical applications, the number of surface positioning balls can be flexibly set according to specific needs, and this embodiment does not impose any restrictions on this.

[0040] This embodiment, based on the geometric characteristics of CBCT rotational scanning, analyzes the motion law of the surface-positioned spheres maintaining a constant projected ordinate. Using this law, data points with similar ordinates are grouped together, achieving initial separation of data points from different spheres. Then, linear fitting is performed on each group of data points to obtain the motion trajectory corresponding to each sphere. This embodiment fully utilizes the physical laws of projection motion in rotational scanning, achieving automatic trajectory division without relying on any prior design values ​​or manual markings. Furthermore, the linear fitting method has low computational complexity, is easy to implement, and is suitable for scanning scenarios with clear motion trajectory patterns.

[0041] In some alternative embodiments, the process of dividing the motion trajectory of the surface positioning ball position data of multiple frames of perspective images into multiple categories of motion trajectories in step S103 above further includes: Step S103b1: Perform cluster analysis on the position data of the small balls on the body surface of the multi-frame perspective images to obtain the clustering results.

[0042] This embodiment uses clustering analysis to automatically group position data points from multi-frame perspective images, grouping data points with similar distribution characteristics into one category, while data points in different categories exhibit significant differences. By performing clustering analysis on position data points, projection points of the same surface positioning ball in different frames can be automatically grouped into the same category without any prior design values ​​or manual labeling. Clustering analysis has strong data adaptability, adaptively grouping data points based on their similarity. Its functionality covers various data distribution scenarios from simple to complex, flexibly addressing the need to distinguish different trajectory patterns.

[0043] Step S103b2: Based on the clustering results, the position data in the same category are determined as the position data of the same surface positioning ball, and multiple categories of motion trajectories are obtained.

[0044] In this embodiment, after cluster analysis, each cluster contains all positional data points determined to originate from the same surface positioning ball. Based on this clustering result, all positional data points within the same cluster are identified as the projected positions of the same surface positioning ball in perspective images from different angles, thereby forming the complete motion trajectory of the ball (e.g., ...). Figure 4 As shown, the horizontal and vertical axes represent pixel positions. Each cluster category corresponds to a surface positioning ball, and multiple cluster categories correspond to the motion trajectories of multiple surface positioning balls. This achieves automatic classification and trajectory division of position data points in all frames according to the ball's identity.

[0045] This embodiment first employs clustering analysis to automatically group all location data points in an unsupervised manner, aggregating data points belonging to the same ball without relying on any prior design values ​​or manual labeling. Then, location data of the same category are identified as the projected positions of the same ball in different frames, forming the motion trajectory corresponding to each ball. This embodiment achieves automatic trajectory division through a data-driven approach, without strict linear assumptions regarding data distribution, exhibiting good data adaptability, and is particularly suitable for situations where the differences in the ordinates of different balls are not significant or the trajectory patterns are relatively complex.

[0046] In some optional implementations, the process of performing cluster analysis on the position data of the small balls on the body surface of multiple perspective images to obtain the clustering results in step S103b1 specifically includes: Step S103b11c: Determine the number of cluster categories based on the number of small balls located on the body surface.

[0047] In this embodiment, since the number of small balls set on the body surface positioning band is known, and each small ball corresponds to an independent motion trajectory category, the number of small balls on the body surface positioning band is directly determined as the number of clustering categories. This ensures that the target number of clustering algorithms is consistent with the actual number of small balls, and that each small ball has one and only one corresponding clustering category, thus avoiding over-clustering or under-clustering problems caused by improper setting of the number of categories.

[0048] Step S103b12c: Based on the number of cluster categories, a corresponding number of location data points are randomly selected from the location data of the small balls on the body surface as initial cluster centers, resulting in multiple initial cluster centers.

[0049] In this embodiment, based on the determined number K of cluster categories, K data points are randomly selected from all location data points. Each selected data point is used as an initial cluster center, resulting in K initial cluster centers. Each initial cluster center serves as the initial reference position for each category, used for the initial allocation of all data points in subsequent iterative calculations. The method of randomly selecting initial cluster centers is simple to implement and requires no prior knowledge to initialize the cluster centers.

[0050] Step S103b13c: Based on the locations of multiple initial cluster centers, calculate the distance from each location data point to each initial cluster center, and assign each location data point to the nearest initial cluster center to obtain multiple location data point clusters.

[0051] In this embodiment, for each data point in the total number of location data points, the distance from each initial cluster center is calculated, and the data point is assigned to the category corresponding to the nearest cluster center, thereby forming multiple location data point clusters. In this way, each data point is initially assigned to the category represented by the cluster center closest to its spatial location, completing the initial division of all data points, so that the data points within each cluster are initially clustered together in spatial location.

[0052] Step S103b14c: Calculate the mean value of all location data points in each location data point cluster to obtain multiple second cluster centers.

[0053] In this embodiment, for each cluster of location data points formed in step S103b13c, the mean position of all location data points within that cluster is calculated, and this mean is used as the new cluster center (i.e., the second cluster center) for that cluster. Through mean updating, the cluster center is moved to the optimal position under the current clustering result, making the cluster center closer to the true data distribution center of the cluster. The mean of each of the K clusters is calculated independently, resulting in K updated cluster centers, which are used for the next data point allocation.

[0054] Step S103b15c: Determine whether the preset convergence condition is met based on multiple second cluster centers. If the multiple second cluster centers meet the preset convergence condition, then execute step S103b16c; if the multiple second cluster centers do not meet the preset convergence condition, then execute step S103b17c.

[0055] The preset convergence conditions in this embodiment include, but are not limited to, the following: the positional change of the second cluster center relative to the previous cluster center is less than a preset threshold, indicating that the cluster center has stabilized; the cluster affiliation of each data point no longer changes, indicating that the clustering result has converged; or the number of iterations reaches a preset maximum value. This embodiment determines whether the second cluster center meets the convergence conditions and selects to execute step S103b16c or step S103b17c based on the determination result. The determination of the convergence conditions ensures that the termination time of the clustering process is reasonable and controllable, preventing inaccurate clustering results due to insufficient iterations and avoiding waste of computational resources due to excessive iterations.

[0056] Step S103b16c: Cluster multiple location data points as clustering results.

[0057] In this embodiment, when the preset convergence condition is met, it indicates that the clustering process has stabilized, and the current clustering result can effectively reflect the true grouping of data points. At this time, the final clusters of multiple location data points are output as the clustering result. Each cluster represents the set of projection positions of a body surface positioning ball in different frames, and the clustering result is directly used for the generation of subsequent motion trajectories.

[0058] Step S103b17c involves continuing the iteration based on multiple second cluster centers until the preset convergence condition is met, thus obtaining the clustering result.

[0059] In this embodiment, when the preset convergence condition is not met, it indicates that the current clustering result has not yet reached the optimal state and further iterative optimization is needed. At this time, the multiple second cluster centers obtained in step S103b14c are used as new cluster centers, and the process returns to step S103b13c. The distance from each data point to each new cluster center is recalculated and allocated. Then, the mean is calculated again to update the cluster centers. This process is repeated iteratively until the preset convergence condition is met, and the final clustering result is output. Through the above iterative process, the cluster centers gradually approach the true data distribution center, and the allocation of data points gradually becomes more stable, ultimately achieving accurate grouping results.

[0060] This embodiment first determines the number of cluster categories based on the number of surface positioning balls, setting the correct grouping target for the clustering algorithm. Next, it randomly selects a corresponding number of initial cluster centers from the location data. Then, it iteratively updates the process of assigning location data points to cluster centers: each iteration assigns each location data point to the nearest cluster center, forming multiple data point clusters, and then calculates the mean of each cluster as the new cluster center. When the cluster centers no longer change significantly or the preset number of iterations is reached, the iteration terminates, and the final clustering result is output. This embodiment is based on the K-means clustering algorithm, which does not rely on specific assumptions about the data distribution pattern. It can automatically complete grouping based solely on the spatial distance between data points, and has high computational efficiency. It can automatically distinguish and classify different surface positioning ball location data without prior design values ​​or manual labeling.

[0061] In some alternative embodiments, the step S103b1 above, which involves clustering the position data of the small balls on the body surface of multiple perspective images to obtain clustering results, further includes: Step S103b11d: Construct a linear regression model corresponding to the number of small balls located on the body surface.

[0062] In this embodiment, an equal number of linear regression models are constructed based on the number of surface-positioning balls. Each linear regression model corresponds to the trajectory of one surface-positioning ball and is used for subsequent fitting and allocation of position data points. The purpose of constructing multiple models is to ensure that each ball has an independent model for trajectory representation, thereby achieving parallel processing and differentiation of multiple targets. The initial linear parameters of each model can be obtained in various ways, such as randomly selecting two points from all data points to determine a straight line, or performing a global least-squares fit on all data points as a unified initial value. The specific method of selecting the initial parameters does not affect the final result.

[0063] In step S103b12d, each linear regression model is used to perform initial line fitting on the position data of the small ball on the body surface of multiple perspective images to obtain multiple straight lines.

[0064] In this embodiment, each constructed linear regression model is used to perform initial line fitting on all location data points. Each model outputs an initial line, resulting in multiple lines, each representing the initial trajectory of a surface-positioning ball. Since the initial fitting is based on all data points, the lines are not yet associated with a specific ball and are in an initial state, providing an initial reference for subsequent iterative allocation.

[0065] Step S103b13d: Based on the position data of the ball on the body surface from the multi-frame perspective images, calculate the distance from each position data point to multiple straight lines to obtain the distance data of each position data point.

[0066] In this embodiment, for each data point among all location data points, the distance from it to multiple straight lines is calculated, obtaining the distance data of each data point relative to each straight line. This distance reflects the degree of deviation of the data point from each trajectory and serves as the basis for subsequent assignment.

[0067] Step S103b14d: Based on the distance data, each location data point is assigned to the nearest straight line to obtain a combination of multiple straight line location data points.

[0068] This embodiment calculates the distances from each position data point to multiple straight lines based on steps S103b13d. Each data point is assigned to the nearest straight line, forming multiple combinations of straight line position data points. Each combination contains all position data points assigned to the same straight line, representing the set of projection points corresponding to the ball's trajectory under the current straight line parameters. Through this assignment, all position data points are initially divided into trajectory categories represented by different straight lines, achieving initial grouping of the data points.

[0069] Step S103b15d: Calculate the sum of squared distances for each combination based on the combination of straight-line position data points to obtain the distance data for each combination.

[0070] In this embodiment, for each combination of data points at the line position formed in step S103b14d, the sum of squared distances from each data point within that combination to its corresponding line is calculated, resulting in the sum of squared distances for each combination. The sum of squared distances reflects the degree of fit of the currently fitted line to the data points within that combination; the smaller the value, the better the fit, meaning the line better represents the distribution trend of the data points in that group. This indicator serves as a quantitative basis for judging whether the current classification result is reasonable and whether further iteration is needed.

[0071] Step S103b16d: Determine whether the preset distance conditions are met based on each combination of distance data. If each combination of distance data meets the preset distance conditions, then proceed to step S103b17d; if each combination of distance data does not meet the preset distance conditions, then proceed to step S103b18d.

[0072] In this embodiment, the preset distance conditions include, but are not limited to, the following: the sum of squared distances of each combination is less than a preset threshold, indicating that the current line has well fitted the data points of the corresponding group; or the change in the sum of squared distances of this iteration relative to the previous iteration is less than a preset threshold, indicating that the fitting effect has tended to stabilize; or the number of iterations has reached a preset maximum value. This step determines whether the clustering process should be terminated by judging whether any of the above conditions are met. When the condition is met, it means that the current fitting result is stable and reliable enough, and step S103b17d is executed to output the clustering result; when the condition is not met, it means that the fitting has not yet reached the optimal level, and step S103b18d is executed to continue the optimization iteration.

[0073] Step S103b17d combines multiple straight-line position data points as the clustering result.

[0074] In this embodiment, when the preset distance condition is met, it indicates that each straight line has well fitted its corresponding data point group, and the current classification result has stabilized. At this time, the final combination of multiple straight line position data points is output as the clustering result. Each combination represents the set of projection positions of a body surface positioning ball in different frames, and the clustering result is directly used for the generation of subsequent motion trajectories.

[0075] Step S103b18d involves updating the corresponding linear regression model based on the combination of multiple straight-line position data points until the preset convergence condition is met, thus obtaining the clustering result.

[0076] In this embodiment, when the preset distance condition is not met, it indicates that the current fit has not yet reached the optimal state, and the model needs to be updated to improve the fitting effect. At this time, based on the data point combinations of each straight line position obtained in step S103b14d, the data points in each combination are refitted with straight lines, and the straight line parameters of the corresponding linear regression model are updated so that the updated straight line more accurately represents the distribution trend of the data points in that group. The above allocation and update process is repeated, that is, each iteration first reallocates the data points according to the current straight line parameters, and then updates the straight line parameters according to the new allocation relationship, until the preset distance condition is met, and the final clustering result is output. Through the above iterative process, the straight line gradually approximates the real trajectory of the ball, and the allocation of the data points gradually tends to stabilize, ultimately achieving an accurate grouping effect.

[0077] This embodiment first constructs a corresponding number of linear regression models based on the number of surface positioning balls, establishing an independent trajectory fitting function for each ball. Next, each model is used to initialize straight line fitting for all positional data, resulting in multiple initial straight lines. Then, the assignment and fitting process is repeatedly executed iteratively: in each iteration, each positional data point is assigned to the nearest straight line, forming multiple data point combinations. Then, the corresponding straight line is refitted based on the data points in each combination, updating the model parameters. When the sum of squared distances from a data point to its assigned straight line reaches its minimum or a preset convergence condition is met, the iteration terminates, and the final clustering result is output. Because the linear regression model directly utilizes the physical law that the trajectory of the surface positioning balls approximates a straight line, this implementation has high fitting accuracy in scenarios with clear trajectory patterns, and the model parameter update process converges quickly, obtaining accurate trajectory division results within a relatively small number of iterations.

[0078] In some optional implementations, the process of supplementing the missing position data of each motion trajectory based on multiple categories of motion trajectories in step S104 above to obtain the target position data of multiple categories of motion trajectories mainly includes: Step S1041: Based on the position data of each category of motion trajectory, an interpolation algorithm is used to calculate the missing position data in the corresponding category of motion trajectory to obtain the missing position data.

[0079] In this embodiment, for each identified missing location, the extracted position data from the trajectory is used as known sample points. Utilizing the characteristic that the projected position of the positioning ball continuously changes with angle during rotational scanning, an interpolation algorithm is employed to calculate the position data at the missing location. The interpolation algorithm uses known position data at adjacent angles before and after the missing location to estimate the optimal value of the missing location through fitting or interpolation. The interpolation processes for different types of motion trajectories are independent, with each calculating its own missing location data based on its own trajectory's known data, avoiding errors caused by mixing cross-trajectory data.

[0080] Besides interpolation algorithms, other data completion methods can also be used. For example, based on the assumption of motion continuity of the trajectory, Kalman filtering or particle filtering can be used to optimally estimate the missing positions; deep learning-based methods can also be used, such as using a long short-term memory network to learn the variation pattern of the ball's projected position on the frame sequence to generate position data for missing frames.

[0081] like Figure 4 As shown in the figure (the horizontal and vertical axes represent pixel positions), Category 1, Category 2, and Category 3 represent three surface positioning balls. It can be seen from the figure that there are missing position points in the trajectory of each category of ball. In this embodiment, the coordinates of these missing position data points are calculated using an interpolation algorithm.

[0082] Step S1042: Combine the location data of each category of motion trajectory with the corresponding missing location data to obtain target location data for multiple categories of motion trajectories.

[0083] This embodiment combines the previously extracted position data from each type of motion trajectory with the missing position data calculated in step S1041 to form a complete position data sequence for each type of motion trajectory at all angles, which serves as the target position data for that trajectory. Multiple types of motion trajectories independently complete the above combination, obtaining their respective target position data, providing complete data input for subsequent frame matching. For example... Figure 5 As shown (the horizontal and vertical axes represent pixel positions), the data points that were originally missing due to occlusion or limited field of view have been supplemented by interpolation, and each trajectory has continuous and complete position data at all angles.

[0084] This embodiment first uses interpolation algorithms to calculate missing position data based on existing position data within the same trajectory, relying solely on known data from the same trajectory to ensure the accuracy of the interpolation results. Then, the original position data is combined with supplementary position data to obtain a complete position data sequence for each trajectory at all angles. This embodiment effectively compensates for data loss caused by occlusion or limited field of view, ensuring that each surface positioning ball has complete position data at all angles, providing a continuous and complete data foundation for subsequent frame matching and 3D reconstruction.

[0085] In some optional implementations, the process of matching the target position data with perspective image frames based on the category of motion trajectory in step S105 above to obtain the position data of the surface positioning ball in different frames of perspective images specifically includes: Step S1051: Identify frames with missing position data in multiple categories of motion trajectories based on the position data of the motion trajectory, and obtain the missing frame information of multiple categories of motion trajectories.

[0086] In this embodiment, after data supplementation is completed in step S104, the position data of each trajectory is complete in the angular dimension. However, it contains two types of data: one is the originally detected position data (known source frame), and the other is the interpolated and completed position data (only coordinate values, not yet associated with a frame). This embodiment analyzes the frame information corresponding to the extracted position data in each category of motion trajectory to identify which frames in the trajectory originally did not have detected position data (i.e., missing frames), thus obtaining missing frame information. The missing frame information includes the number and quantity of missing frames, used to clarify which frames need to be matched and associated with the completed position data.

[0087] Step S1052: Based on the missing frame information, the target position data of each category of motion trajectory is matched with the corresponding perspective image frame to obtain the position data of the body surface positioning ball in different frame perspective images.

[0088] This embodiment, based on missing frame information, matches the completed position data in each category of motion trajectory with the corresponding missing frames one by one. Simultaneously, it associates the previously detected position data with their original frames, ensuring that each target position data in the trajectory is linked to the corresponding frame information. Ultimately, each category of motion trajectory has complete position data in all frames. Therefore, different categories of motion trajectories correspond to different surface positioning balls, thus obtaining the position data of each surface positioning ball in different frame perspective images, thereby clarifying the position coordinates of each surface positioning ball in each frame.

[0089] This embodiment first identifies frames with missing position data in each category of motion trajectory, determining which frames need to be matched with the completed position data. Then, it associates the completed position data with the corresponding perspective image frames, while preserving the correspondence between the original detection data and the original frames, ensuring that all target position data obtain frame information. Through this method, the interpolated and completed data can accurately correspond to the missing frames, ensuring that the position data of the same ball in different frames are correctly associated, providing complete position data with frame correspondence for subsequent 3D reconstruction.

[0090] In some optional embodiments, the process of detecting the body surface localization ball in step S102 of the above-mentioned multi-frame perspective images to obtain the position data of the body surface localization ball in the multi-frame perspective images specifically includes: Step S1021: Extract gradient information from the pixels in the multi-frame perspective images to obtain multi-frame gradient images.

[0091] In this embodiment, for each frame of perspective image, a gradient extraction algorithm is used to calculate the gradient magnitude and gradient direction at each pixel location in the image, resulting in multiple frames of gradient images. Gradient information reflects the degree of drastic change in pixel values ​​in the image. The grayscale difference between the sphere and the background is manifested as a clear edge response in the gradient image. Therefore, gradient extraction can highlight the contour information of the sphere's edge, enhance the edge features of the circular target, and provide clear edge response data for subsequent shape detection.

[0092] Step S1022: Perform shape detection based on multiple frames of gradient images to obtain shape data. The shape data includes the center coordinates and radius information of the circle, and use the center coordinates as the position data of the small ball on the body surface of the multiple frames of perspective images.

[0093] This embodiment uses the gradient images obtained in step S1021 to detect circular targets in the gradient images using a shape detection algorithm (such as Hough transform). It identifies all candidate targets that satisfy the circular feature and extracts their center coordinates and radius information. Because the gradient images highlight the edge response of the target, the shape detection algorithm can more quickly locate the circular target, improving both detection efficiency and accuracy. Finally, the center coordinates of the detected circular targets in each frame are used as the surface positioning data for the small ball in that frame.

[0094] This embodiment first extracts gradient information from each frame of perspective image, using the grayscale difference between the sphere's edge and the background to highlight the edge contour of the circular target. Then, it performs circular shape detection on the gradient image, using the gradient information to enhance detection accuracy and processing efficiency, and outputs the center coordinates and radius information of the circular target in each frame. This embodiment can accurately extract the two-dimensional projection position and radius parameters of the sphere located on the body surface in each frame of perspective image, providing a reliable data foundation for subsequent trajectory division.

[0095] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0096] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for locating and matching small balls on the body surface based on perspective images, characterized in that, The method includes: Acquire multiple perspective images of a target object obtained by rotating and scanning the target object, wherein the target object is provided with multiple surface positioning balls; The body surface localization ball is detected in the multi-frame perspective images respectively to obtain the body surface localization ball position data of the multi-frame perspective images; The motion trajectory of the positioning ball on the body surface in the multi-frame perspective images is divided to obtain multiple categories of motion trajectories; Based on the multiple categories of motion trajectories, the missing location data of each motion trajectory is supplemented to obtain the target location data of the multiple categories of motion trajectories; Based on the category of the motion trajectory, the target position data is matched with perspective image frames to obtain the position data of the body surface positioning ball in different frames of perspective images.

2. The method according to claim 1, characterized in that, The motion trajectory is divided into multiple categories based on the position data of the small ball on the body surface in the multi-frame perspective images, including: Based on the distribution characteristics of the body surface positioning ball position data, the body surface positioning ball position data with similar vertical coordinates are divided into a group to obtain multiple groups of body surface positioning ball position data; Linear fitting is performed on the position data of the multiple sets of surface positioning balls to obtain the multiple types of motion trajectories.

3. The method according to claim 1, characterized in that, The step of dividing the motion trajectory of the surface positioning ball position data of the multi-frame perspective images into multiple categories of motion trajectories also includes: Cluster analysis is performed on the position data of the small balls on the body surface of the multi-frame perspective images to obtain the clustering results; Based on the clustering results, the location data in the same category are identified as the location data of the same surface positioning ball, resulting in multiple categories of motion trajectories.

4. The method according to claim 3, characterized in that, The clustering analysis of the body surface positioning ball position data of the multi-frame perspective images to obtain the clustering results includes: The number of cluster categories is determined based on the number of the small balls located on the body surface. Based on the number of cluster categories, a corresponding number of location data points are randomly selected from the location data of the body surface positioning balls as initial cluster centers to obtain multiple initial cluster centers; Based on the locations of the multiple initial cluster centers, the distance from each location data point to each initial cluster center is calculated, and each location data point is assigned to the nearest initial cluster center to obtain multiple location data point clusters; Multiple second cluster centers are obtained by calculating the mean of the location data of all location data points in each location data point cluster. Based on the multiple second cluster centers, determine whether the preset convergence condition is met; If the plurality of second cluster centers satisfy the preset convergence condition, then the plurality of location data point clusters are taken as the clustering result; if the plurality of second cluster centers do not satisfy the preset convergence condition, then the iteration is continued based on the plurality of second cluster centers until the preset convergence condition is satisfied, and the clustering result is obtained.

5. The method according to claim 3, characterized in that, The step of performing cluster analysis on the position data of the small balls on the body surface of the multi-frame perspective images to obtain clustering results also includes: A linear regression model is constructed based on the number of the surface positioning balls; Each of the linear regression models is used to perform initial line fitting on the position data of the small ball on the body surface of the multi-frame perspective images to obtain multiple straight lines; Based on the position data of the small ball on the body surface of the multi-frame perspective image, the distance from each position data point to multiple straight lines is calculated to obtain the distance data of each position data point; Based on the distance data, each location data point is assigned to the nearest straight line, resulting in a combination of multiple straight line location data points; Based on the combination of the line position data points, the sum of squared distances for each combination is calculated to obtain the distance data for each combination; Based on the various combined distance data, it is determined whether a preset distance condition is met. If the various combined distance data meet the preset distance condition, the multiple straight-line position data points are combined as the clustering result. If the various combined distance data do not meet the preset distance condition, the corresponding linear regression model is updated based on the multiple straight-line position data point combinations until the preset convergence condition is met, and the clustering result is obtained.

6. The method according to claim 1, characterized in that, The process of supplementing the missing location data of each motion trajectory based on the multiple categories of motion trajectories to obtain the target location data of the multiple categories of motion trajectories includes: Based on the position data of each category of motion trajectory, an interpolation algorithm is used to calculate the missing position data in the corresponding category of motion trajectory to obtain the missing position data; The target location data of the multiple categories of motion trajectories are obtained by combining the location data of each category of motion trajectory with the corresponding missing location data.

7. The method according to claim 1, characterized in that, The method of matching the target position data with perspective image frames based on the category of the motion trajectory to obtain the position data of the body surface positioning ball in different frames of perspective images includes: Based on the position data of the motion trajectory, frames with missing position data in the multiple categories of motion trajectories are identified to obtain missing frame information for multiple categories of motion trajectories; Based on the missing frame information, the target position data of each category of motion trajectory is matched with the corresponding perspective image frame to obtain the position data of the body surface positioning ball in different frame perspective images.

8. The method according to claim 1, characterized in that, The step of detecting the surface localization spheres in the multiple perspective images to obtain the surface localization sphere position data in the multiple perspective images includes: Gradient information is extracted from the pixels in the multi-frame perspective images to obtain multi-frame gradient images; Shape detection is performed on the multi-frame gradient images to obtain shape data, which includes the center coordinates and radius information of the circle. The center coordinates are used as the position data of the small ball on the body surface of the multi-frame perspective images.

9. The method according to claim 1, characterized in that, Before dividing the motion trajectory of the surface positioning ball position data of the multi-frame perspective images into multiple categories of motion trajectories, the method further includes: Obtain the radius information of the circles in the multi-frame perspective images; Based on the radius threshold, the center coordinates of the circles that do not conform to the radius threshold are deleted to obtain the surface positioning ball position data of the filtered multi-frame perspective images.

10. A body surface positioning ball matching device based on perspective images, characterized in that, include: A processor and a memory connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to cause the processor to perform the body surface localization ball matching method based on perspective images as described in any one of claims 1-9.

Images