Three-dimensional target detection method and device, surgical robot, and electronic device

Abstract

This application discloses a three-dimensional target detection method and apparatus, a surgical robot, and an electronic device. The method includes: acquiring a first three-dimensional image; performing semantic segmentation on the first three-dimensional image to obtain n first candidate regions, where the voxels in each first candidate region represent a first object, and n is a positive integer; obtaining n first volumes from the n first candidate regions based on the number of voxels in each of the n first candidate regions and the volume of the voxels in the first three-dimensional image, with each first volume corresponding one-to-one with a first candidate region; and obtaining m target regions from the n first volumes and n first candidate regions, where each target region is a region in the first three-dimensional image corresponding to a first object, and m is a positive integer less than or equal to n. Using this three-dimensional target detection method to perform target detection on a three-dimensional image can improve the accuracy of the target detection results.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a three-dimensional target detection method and apparatus, a surgical robot, and an electronic device. Background Technology

[0002] By performing target detection on a 3D image, the image region corresponding to the object to be detected can be determined. Then, relevant processing can be performed based on the image region. For example, based on the position of the image region in the 3D image and the transformation relationship between the position in the 3D image and the position in the world coordinate system, the position of the detected object in the world coordinate system can be obtained.

[0003] However, target detection based on traditional methods is prone to having a large number of false positive regions in the detection results, which in turn leads to low accuracy. False positive regions refer to image regions that are misidentified as corresponding to the object to be detected. Summary of the Invention

[0004] This application provides a three-dimensional target detection method and apparatus, a surgical robot, and an electronic device to improve the accuracy of target detection results in three-dimensional images by reducing the number of false positive regions in the target detection results.

[0005] Firstly, a three-dimensional target detection method is provided, the method comprising:

[0006] Obtain the first 3D image;

[0007] By performing semantic segmentation on the first three-dimensional image, n first candidate regions are obtained, and the semantics of the voxels in the first candidate regions are first objects, where n is a positive integer;

[0008] Based on the number of voxels in each of the n first candidate regions and the volume of the voxels in the first three-dimensional image, n first volumes of the n first candidate regions are obtained, and each first volume corresponds one-to-one with a first candidate region.

[0009] Based on the n first volumes and the n first candidate regions, m target regions are obtained. The target regions are the regions in the first three-dimensional image that correspond to the first object, and m is a positive integer less than or equal to n.

[0010] In any embodiment of this application, obtaining m target regions based on the n first volumes and the n first candidate regions includes:

[0011] A third volume is determined from the n first volumes, wherein the difference between the second volume and the second volume is less than or equal to a first threshold, and the second volume is obtained based on the actual volume of the first object;

[0012] Based on the regions corresponding to the third volume in the n first candidate regions, the m target regions are obtained.

[0013] In any embodiment of this application, where the first object is a sphere, the method further includes:

[0014] Determine the minimum distance between the voxels in the m target regions and the background region in the first 3D image to obtain m distance sets. The semantics of the voxels in the background region are not the first object. The distance sets correspond one-to-one with the target regions.

[0015] Based on the m distance sets, m distance subsets are obtained, each distance subset corresponding to one of the distance sets, and the distances in each distance subset are the k largest distances in the distance sets.

[0016] Based on the m distance subsets, m sub-regions are determined from the m target regions, and the sub-regions correspond one-to-one with the distance subsets, and the voxels in the sub-regions correspond to the distances in the distance subsets;

[0017] Based on the positions of voxels in the m sub-regions in the first 3D image, m sphere center positions are obtained, each corresponding to one of the sub-regions, and each sphere center position is the position of the sphere center of the first object in the first 3D image.

[0018] In conjunction with any embodiment of this application, the step of obtaining n first candidate regions by semantic segmentation of the first three-dimensional image includes:

[0019] Semantic segmentation is performed on the first three-dimensional image to obtain x second candidate regions. The semantics of the voxels in the second candidate regions are first objects, and x is a positive integer greater than or equal to n.

[0020] From the x second candidate regions, regions whose shapes match the shape of the first object are determined to obtain the n first candidate regions.

[0021] In any embodiment of this application, the number of the first objects is i, where i and n are both integers greater than or equal to 2;

[0022] The step of determining regions whose shapes match the shape of the first object from the x second candidate regions to obtain the n first candidate regions includes:

[0023] From the x second candidate regions, regions whose shapes match the shape of the first object are determined to obtain y third candidate regions, where y is a positive integer, y is less than or equal to x, and y is greater than or equal to n;

[0024] Based on the relative positional relationship of the i first objects at the target time, n first candidate regions are determined from the y third candidate regions, where the target time is the time when the first three-dimensional image is acquired, and the relative positional relationship of the n first candidate regions matches the relative positional relationship of the target.

[0025] In conjunction with any embodiment of this application, before determining the n first candidate regions from the y third candidate regions based on the target relative positional relationship of the i first objects at the target time, the method further includes:

[0026] Based on the matching degree between the shape of the y third candidate regions and the shape of the first object, y confidence scores of the y third candidate regions are obtained. The confidence scores correspond one-to-one with the third candidate regions, and the confidence scores are positively correlated with the matching degree.

[0027] Select the region with the highest confidence level from the y third candidate regions as the initial region;

[0028] The step of determining the n first candidate regions from the y third candidate regions based on the relative positional relationships of the i first objects at the target time includes:

[0029] Based on the relative positional relationship between the initial region and the target, j fourth candidate regions are determined from the y third candidate regions, wherein the relative positional relationship between the initial region and the j fourth candidate regions matches the relative positional relationship of the target, and j is a positive integer less than n;

[0030] Based on the initial region and the j fourth candidate regions, the n first candidate regions are obtained.

[0031] In any embodiment of this application, the first object is a ball, and the first object is attached to the skin of the second object;

[0032] Before determining regions whose shapes match the shape of the first object from the x second candidate regions to obtain y third candidate regions, the method further includes:

[0033] Determine the skin region of the second object from the first three-dimensional image;

[0034] The step of determining regions whose shapes match the shape of the first object from the x second candidate regions to obtain y third candidate regions includes:

[0035] From the x second candidate regions, regions whose shapes match the shape of the first object are determined to obtain r fifth candidate regions, where r is a positive integer, r is less than or equal to x, and r is greater than or equal to y;

[0036] The centers of the r fifth candidate regions are determined respectively, resulting in r centers;

[0037] Based on the r centers, r sets of first rays are obtained, each set of first rays corresponding to one of the centers, and each set of first rays is a set of rays with the center as its endpoint.

[0038] The proportion of rays intersecting the skin region in each of the first ray sets is determined to obtain r proportions;

[0039] The first ray set corresponding to the proportion of the r proportions that is greater than the second threshold is determined as the second ray set;

[0040] Based on the second ray set, the y third candidate regions are determined from the r fifth candidate regions, and the center corresponding to the second ray set is the center of the third candidate region.

[0041] Secondly, a three-dimensional target detection device is provided, the device comprising:

[0042] The acquisition unit is used to acquire the first three-dimensional image;

[0043] The processing unit is configured to obtain at least one first candidate region by performing semantic segmentation on the first three-dimensional image, wherein the semantics of the voxels in the first candidate region are first objects;

[0044] The processing unit is further configured to obtain at least one first volume of the at least one first candidate region based on the number of voxels in each of the at least one first candidate regions and the volume of the voxels in the first three-dimensional image, wherein the first volume corresponds one-to-one with the first candidate region.

[0045] The processing unit is further configured to obtain at least one target region based on the at least one first volume and the at least one first candidate region, wherein the target region is the region in the first three-dimensional image corresponding to the first object.

[0046] In conjunction with any embodiment of this application, the processing unit is specifically used for:

[0047] A third volume is determined from the n first volumes, wherein the difference between the second volume and the second volume is less than or equal to a first threshold, and the second volume is obtained based on the actual volume of the first object;

[0048] Based on the regions corresponding to the third volume in the n first candidate regions, the m target regions are obtained.

[0049] In any embodiment of this application, the first object is a sphere, and the processing unit is further configured to:

[0050] Determine the minimum distance between the voxels in the m target regions and the background region in the first 3D image to obtain m distance sets. The semantics of the voxels in the background region are not the first object. The distance sets correspond one-to-one with the target regions.

[0051] Based on the m distance sets, m distance subsets are obtained, each distance subset corresponding to one of the distance sets, and the distances in each distance subset are the k largest distances in the distance sets.

[0052] Based on the m distance subsets, m sub-regions are determined from the m target regions, and the sub-regions correspond one-to-one with the distance subsets, and the voxels in the sub-regions correspond to the distances in the distance subsets;

[0053] Based on the positions of voxels in the m sub-regions in the first 3D image, m sphere center positions are obtained, each corresponding to one of the sub-regions, and each sphere center position is the position of the sphere center of the first object in the first 3D image.

[0054] In conjunction with any embodiment of this application, the processing unit is specifically used for:

[0055] Semantic segmentation is performed on the first three-dimensional image to obtain x second candidate regions. The semantics of the voxels in the second candidate regions are first objects, and x is a positive integer greater than or equal to n.

[0056] From the x second candidate regions, regions whose shapes match the shape of the first object are determined to obtain the n first candidate regions.

[0057] In any embodiment of this application, the number of the first objects is i, where i and n are both integers greater than or equal to 2;

[0058] The processing unit is specifically used for:

[0059] From the x second candidate regions, regions whose shapes match the shape of the first object are determined to obtain y third candidate regions, where y is a positive integer, y is less than or equal to x, and y is greater than or equal to n;

[0060] Based on the relative positional relationship of the i first objects at the target time, n first candidate regions are determined from the y third candidate regions, where the target time is the time when the first three-dimensional image is acquired, and the relative positional relationship of the n first candidate regions matches the relative positional relationship of the target.

[0061] In conjunction with any embodiment of this application, the processing unit is further configured to:

[0062] Based on the matching degree between the shape of the y third candidate regions and the shape of the first object, y confidence scores of the y third candidate regions are obtained. The confidence scores correspond one-to-one with the third candidate regions, and the confidence scores are positively correlated with the matching degree.

[0063] Select the region with the highest confidence level from the y third candidate regions as the initial region;

[0064] The processing unit is further specifically used for:

[0065] Based on the relative positional relationship between the initial region and the target, j fourth candidate regions are determined from the y third candidate regions, wherein the relative positional relationship between the initial region and the j fourth candidate regions matches the relative positional relationship of the target, and j is a positive integer less than n;

[0066] Based on the initial region and the j fourth candidate regions, the n first candidate regions are obtained.

[0067] In any embodiment of this application, the first object is a ball, and the first object is attached to the skin of the second object;

[0068] The processing unit is further configured to:

[0069] Determine the skin region of the second object from the first three-dimensional image;

[0070] The processing unit is further specifically used for:

[0071] From the x second candidate regions, regions whose shapes match the shape of the first object are determined to obtain r fifth candidate regions, where r is a positive integer, r is less than or equal to x, and r is greater than or equal to y;

[0072] The centers of the r fifth candidate regions are determined respectively, resulting in r centers;

[0073] Based on the r centers, r sets of first rays are obtained, each set of first rays corresponding to one of the centers, and each set of first rays is a set of rays with the center as its endpoint.

[0074] The proportion of rays intersecting the skin region in each of the first ray sets is determined to obtain r proportions;

[0075] The first ray set corresponding to the proportion of the r proportions that is greater than the second threshold is determined as the second ray set;

[0076] Based on the second ray set, the y third candidate regions are determined from the r fifth candidate regions, and the center corresponding to the second ray set is the center of the third candidate region.

[0077] Thirdly, a surgical robot is provided, including the three-dimensional target detection device as described in the second aspect. In this third aspect, the surgical robot can perform a three-dimensional target detection method using the three-dimensional target detection device, thereby improving the accuracy of target detection results in three-dimensional images.

[0078] Fourthly, an electronic device is provided, comprising: a processor and a memory, the memory for storing computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs a method as described in the first aspect above and any possible implementation thereof.

[0079] Fifthly, another electronic device is provided, comprising: a processor, a transmitting device, an input device, an output device, and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs the method as described in the first aspect above and any possible implementation thereof.

[0080] In a sixth aspect, a computer-readable storage medium is provided, wherein a computer program is stored therein, the computer program including program instructions that, when executed by a processor, cause the processor to perform a method as described in the first aspect above and any possible implementation thereof.

[0081] In a seventh aspect, a computer program product is provided, the computer program product comprising a computer program or instructions, wherein, when the computer program or instructions are executed on a computer, the computer performs the method described in the first aspect and any possible implementation thereof.

[0082] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0083] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments of this application will be described below.

[0084] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0085] Figure 1 A flowchart illustrating a three-dimensional target detection method provided in an embodiment of this application;

[0086] Figure 2 A schematic diagram of a cross-section of a first candidate region provided in an embodiment of this application;

[0087] Figure 3 A schematic diagram of a false positive region provided in an embodiment of this application;

[0088] Figure 4 This is a schematic diagram of m target regions provided in an embodiment of this application;

[0089] Figure 5 A schematic diagram of n first candidate regions provided in an embodiment of this application;

[0090] Figure 6 A schematic diagram of another set of n first candidate regions provided in an embodiment of this application;

[0091] Figure 7 A schematic diagram of y third candidate regions provided in an embodiment of this application;

[0092] Figure 8 This is a schematic diagram of the structure of a three-dimensional target detection device provided in an embodiment of this application;

[0093] Figure 9 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0094] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0095] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0096] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. It should be understood that in this application, "at least one" means one or more, "more" means two or more, and "at least two" means two or three or more.

[0097] The execution subject of this application embodiment is a three-dimensional target detection device (hereinafter referred to as the detection device), wherein the detection device can be any electronic device capable of executing the technical solution disclosed in the method embodiment of this application. Optionally, the detection device can be one of the following: a computer, a platform server.

[0098] It should be understood that the method embodiments of this application can also be implemented by a processor executing computer program code. The embodiments of this application are described below with reference to the accompanying drawings. Please refer to... Figure 1 , Figure 1 This is a flowchart illustrating a three-dimensional target detection method provided in an embodiment of this application.

[0099] 101. Obtain the first three-dimensional image.

[0100] In this embodiment, the first three-dimensional image can be any three-dimensional image. Optionally, the first three-dimensional image is a three-dimensional medical image; for example, the first three-dimensional image is a three-dimensional computed tomography (CT) image, or a three-dimensional ultrasound image.

[0101] Optionally, the first three-dimensional image is obtained by photographing the first object, that is, the first three-dimensional image includes an image region corresponding to the first object. For example, the first object is an optical marker attached to the surface of the human body, and the first three-dimensional image is a three-dimensional CT image obtained by CT scanning the human body. In this case, the first three-dimensional image includes an image region corresponding to the optical marker.

[0102] 102. By performing semantic segmentation on the first three-dimensional image, n first candidate regions are obtained.

[0103] In this embodiment, the semantics of the voxels in the first candidate region are the first object, that is, the first candidate region is a candidate region of the image region corresponding to the first object in the first three-dimensional image. n is a positive integer, that is, the number of first candidate regions is at least one.

[0104] Optionally, the number of first objects is greater than or equal to 1, in which case the number of image regions corresponding to the first object in the first 3D image is greater than or equal to 1. Correspondingly, the number of candidate regions for the image regions corresponding to the first object in the first 3D image is greater than or equal to 1. Optionally, the number of first objects is 6.

[0105] Considering the potential errors in the semantic segmentation results obtained from the first 3D image, there may be image regions among the n first candidate regions that do not correspond to the first object. Therefore, the number of first candidate regions is greater than or equal to the number of first objects. For example, the first object is an optical marker attached to the surface of the human body, and the first 3D image is a 3D CT image obtained from a CT scan of the human body. When scanning the human body using a CT scanner, the person is wearing clothing including buttons; therefore, the first 3D image obtained includes image regions corresponding to the buttons. If, during semantic segmentation of the first 3D image, the image region corresponding to the buttons is mistakenly identified as a first candidate region, then the number of first candidate regions is greater than the number of first objects.

[0106] It should be understood that a voxel is a pixel in a three-dimensional image, and the voxels in the first candidate region are the pixels in the first three-dimensional image.

[0107] In some schemes, the detection device performs semantic segmentation on a first 3D image based on a segmentation model to obtain n first candidate regions, wherein the segmentation model is used to detect voxels in the 3D image that semantically represent a first object. Optionally, the segmentation model includes a U-Net architecture. For example, the segmentation model is (no-new-net, nnUNet).

[0108] Optionally, the first object is a sphere, and the shape of the first candidate region is similar to that of a sphere. Figure 2 This is a schematic diagram of a cross-section of a first candidate region provided in an embodiment of this application. Figure 2In the diagram, the white area represents the cross-section of the first candidate region, and the shape of this cross-section is close to that of a circle.

[0109] 103. Based on the number of voxels in each of the n first candidate regions and the volume of voxels in the first three-dimensional image, obtain the n first volumes of the n first candidate regions.

[0110] In this embodiment, the first volume corresponds one-to-one with the first candidate region; that is, the volume of the first candidate region is called the first volume. In step 103, the detection device can obtain the first volume based on the product of the number of voxels in the first candidate region and the volume of the voxels in the first three-dimensional image.

[0111] 104. Based on n first volumes and n first candidate regions, m target regions are obtained.

[0112] In this embodiment, the target region is the region in the first three-dimensional image corresponding to the first object; that is, the target region is the image region in the first three-dimensional image corresponding to the first object. m is a positive integer less than or equal to n. Optionally, m is equal to the number of first objects, that is, the number of target regions is equal to the number of first objects.

[0113] In some schemes, the detection device first determines whether there are false positive regions among n first candidate regions based on n first volumes. These false positive regions are not image regions corresponding to the first object, but are misidentified as image regions corresponding to the first object during semantic segmentation of the first 3D image. Then, if false positive regions exist among the n first candidate regions, they are removed, resulting in m target regions. Here, m is less than n. For example, Figure 3 This is a schematic diagram of a false positive region provided in an embodiment of this application. Figure 3 In the first three-dimensional image, the first three-dimensional image is a three-dimensional CT image obtained by performing a CT scan on a person. The first three-dimensional image includes the image region corresponding to the button, which is a false positive region.

[0114] In one optional implementation, the number of first objects is greater than 1, and the actual volume difference between any two first objects does not exceed a third threshold, meaning the actual volume difference between any two first objects is small. Because each first object has the same volume, the volume difference between the image regions corresponding to different first objects in the first 3D image should be small. Therefore, the detection device determines a first average value of n first volumes. The absolute value of the difference between the n first volumes and the first average value is determined to obtain n first absolute values. Each first absolute value corresponds one-to-one with a first volume, and since each first volume corresponds one-to-one with a first candidate region, each first absolute value corresponds one-to-one with a first candidate region. The larger the first absolute value, the greater the probability of an abnormal volume in the first candidate region, which also means a greater probability that the first candidate region is a false positive region. Therefore, the detection device designates at least one first candidate region corresponding to the largest k first absolute values ​​as a false positive region, where k is the difference between the number of first candidate regions and the number of first objects. For example, if the number of first candidate regions is 8 and the number of first objects is 6, then k is 2. In this case, the regions corresponding to the largest k first absolute values ​​are the two regions with the most outlier first volumes among the n first candidate regions.

[0115] In another optional implementation, the actual volume of the first object is within a first range. In this case, the volume of the image region corresponding to the first object in the first three-dimensional image should be within a second range corresponding to the first range. Optionally, the first range and the second range are the same. The detection device determines that the first candidate regions among n first candidate regions whose first volume is outside the second range are false positive regions.

[0116] In other schemes, if no false positive regions exist in the n first candidate regions, the detection device adjusts the contours of the n first candidate regions to obtain m target regions. For example, if the shape of the first object is a sphere, then the shape of the image region corresponding to the first object in the first 3D image should also be a sphere. Therefore, the detection device can improve the accuracy of the m target regions by adjusting the contours of the n first candidate regions so that the shapes of the m target regions obtained by adjusting the contours of the n first candidate regions are spheres. In this case, m equals n.

[0117] Optionally, the first object is an optical marker ball attached to a person's skin, and the first three-dimensional image is a three-dimensional CT image obtained by performing a CT scan on a person. Figure 4 This is a schematic diagram of m target regions provided in an embodiment of this application. Figure 4 As shown, m is 6, meaning there are 6 optical marker spheres in the first three-dimensional image.

[0118] exist Figure 1In the proposed 3D target detection method, after acquiring a first 3D image, the detection device performs semantic segmentation on the first 3D image to obtain n first candidate regions, where the voxels in each first candidate region represent the first object. Then, based on the number of voxels in each of the n first candidate regions and the volume of the voxels in the first 3D image, n first volumes of the n first candidate regions are obtained. Finally, based on the n first volumes and the n first candidate regions, m target regions are obtained, where the target regions are the regions in the first 3D image corresponding to the first object. This method achieves the acquisition of m target regions based on the semantics of voxels and the n first volumes, thereby reducing false positive regions among the m target regions and improving the accuracy of the m target regions.

[0119] As an optional implementation, based on n first volumes and n first candidate regions, m target regions are obtained, including: determining a third volume from the n first volumes whose difference from a second volume is less than or equal to a first threshold, wherein the second volume is obtained based on the actual volume of the first object; and obtaining m target regions based on the regions corresponding to the third volume among the n first candidate regions.

[0120] In this implementation, the actual volume of the first object refers to its volume in the real world. Optionally, the volume of any object in the first 3D image is the same as its volume in the real world; in this case, the first volume and the second volume are the same. If the third volume is less than or equal to the second volume (a first threshold), it indicates that the difference between the third and second volumes is small, meaning the probability that the third volume corresponds to the image region of the first object is relatively high. Therefore, the detection device obtains m target regions based on the regions corresponding to the third volume in n first candidate regions, thus improving the accuracy of the m target regions.

[0121] Optionally, the ratio of the first threshold to the second volume is a first preset value. For example, the first object is a sphere, and the second volume can be calculated based on the radius of the first object. If the first preset value is 50%, then the first threshold is half of the second volume.

[0122] As an optional implementation, the first object is a sphere, and the radius of the first object is a first radius. Obtaining m target regions based on the regions corresponding to the third volume among n first candidate regions includes the following steps: Firstly, determining the regions corresponding to the third volume among the n first candidate regions yields s first intermediate regions, where s is a positive integer, s is less than or equal to n, and s is greater than or equal to m. Secondly, determining the radii of the s first intermediate regions yields s second radii. Thirdly, determining a third radius from the s second radii whose difference from the first radius is less than or equal to a fourth threshold. Finally, based on the regions corresponding to the third radii among the s first intermediate regions, m target regions are obtained. At this point, the number of third radii is m.

[0123] Optionally, the absolute value of the difference between the fourth threshold and the first radius is a second preset value. For example, if the first radius is 5.7 mm and the second preset value is 1 mm, then the value of the fourth threshold ranges from 4.7 mm to 5.7 mm.

[0124] As an optional implementation, the first object is a sphere, and the detection device further performs the following steps: determining the minimum distance between voxels in m target regions and background regions in the first 3D image, obtaining m distance sets, wherein the semantics of voxels in the background region are not the first object, and the distance sets correspond one-to-one with the target regions. Based on the m distance sets, obtaining m distance subsets, wherein the distance subsets correspond one-to-one with the distance sets, and the distances in the distance subsets are the h largest distances in the distance sets. Based on the m distance subsets, determining m sub-regions from the m target regions, wherein the sub-regions correspond one-to-one with the distance subsets, and the voxels in the sub-regions correspond to the distances in the distance subsets. Based on the positions of the voxels in the m sub-regions in the first 3D image, obtaining m sphere center positions, wherein the sphere center positions correspond one-to-one with the sub-regions, and the sphere center positions are the positions of the sphere center of the first object in the first 3D image.

[0125] In this embodiment, the detection device first determines a distance set for each target region. The distances in the distance set reflect the distances from each voxel within the target region to the edge of the target region. Because the first object is a sphere, the distance from the position corresponding to the center of the sphere within the target region to the edge of the target region is relatively large. Therefore, the detection device first obtains a distance subset based on the distance set, and then determines a sub-region from the target region based on the distance subset. This sub-region is the one furthest from the edge of the target region, i.e., the central sub-region within the target region. Finally, based on the positions of the voxels in the sub-region in the first 3D image, the position of the sphere's center can be obtained.

[0126] Moreover, on the one hand, since there may be deviations between the shape of the target area and the sphere, the center of the target area may not be the position corresponding to the center of the sphere. On the other hand, the position corresponding to the center of the sphere is more likely to be located in the central sub-region of the target area. Therefore, the detection device can improve the accuracy of the position of the center of the sphere by obtaining the position of the center of the sphere based on the central sub-region of the target area.

[0127] Optionally, the ratio of h to the number of voxels in the target region is a third preset value, for example, the third preset value is 20%.

[0128] Optionally, the detection device determines the average position of voxels in the first 3D image across m sub-regions to obtain m sphere center positions. Since the accuracy of voxel positions in the first 3D image is at the voxel level, while the accuracy of the average position of voxels in the first 3D image across sub-regions is at the sub-voxel level, obtaining m sphere center positions using this implementation method can improve the accuracy of the m sphere center positions. For example, the sub-region includes voxels t1 and t2, where voxel t1 is located at (3, 4, 5) in the first 3D image, voxel t2 is located at (6, 5, 6), and the average position of voxels t1 and t2 in the first 3D image is (4.5, 4.5, 5.5).

[0129] In some schemes, after obtaining the m center positions of the spheres, the detection device can modify the shape of the m target regions based on the m center positions to improve the matching degree between the shape of the m target regions and the spheres.

[0130] In other schemes, after obtaining m sphere center positions, the detection device determines the distance between any two sphere center positions, resulting in d sphere center distances. If any of the d sphere center distances is less than a fifth threshold, the two sphere center positions corresponding to these distances are designated as positions to be merged. The two target regions corresponding to these positions in the m target regions are merged, resulting in e merged target regions. Based on the e merged target regions and the regions in the m regions excluding the two target regions corresponding to the positions to be merged, f optimized target regions are obtained. This allows for further optimization of the m target regions, resulting in f optimized target regions, thereby improving the accuracy of the f optimized target regions.

[0131] Optionally, based on the positions of voxels in the e merged target regions in the first three-dimensional image, the positions corresponding to the center of the sphere in the e merged target regions are obtained.

[0132] As an optional implementation, semantic segmentation of the first 3D image yields n first candidate regions, including the following steps: semantic segmentation of the first 3D image yields x second candidate regions, where the voxels in the second candidate regions represent a first object, and x is a positive integer greater than or equal to n. Regions whose shapes match the shape of the first object are determined from the x second candidate regions, resulting in n first candidate regions.

[0133] In this implementation, the detection device first obtains x second candidate regions through semantic segmentation, and then filters out n first candidate regions from the x second candidate regions based on the shape of the first object. In other words, after obtaining x second candidate regions, the detection device can remove false positive regions from the x second candidate regions based on the shape of the first object, obtaining n first candidate regions. This reduces the number of false positive regions in the n first candidate regions. Optionally, if the shape of the first object is a sphere, then the detection device determines regions whose shape matches the sphere from the x second candidate regions, obtaining n first candidate regions.

[0134] As an optional implementation, the number of first objects is i, where both i and n are at least two. In other words, the number of first objects and the number of first candidate regions are both at least two. Here, n is greater than or equal to i, meaning the number of first candidate regions is greater than or equal to the number of first objects.

[0135] To determine n first candidate regions by identifying regions whose shapes match the shape of the first object from x second candidate regions, the process includes the following steps: First, determine y third candidate regions from the x second candidate regions, where y is a positive integer, less than or equal to x, and greater than or equal to n. Second, based on the relative positional relationship of the i first objects at a target time, determine n first candidate regions from the y third candidate regions, where the target time is the time when the first 3D image was acquired, and the relative positional relationship of the n first candidate regions matches the relative positional relationship of the target.

[0136] In this embodiment, the detection device first selects y third candidate regions from x second candidate regions based on the shape of the first object. Then, based on the relative positional relationship of the first object in the real world (i.e., the aforementioned target relative positional relationship), it selects n first candidate regions from the y third candidate regions, thereby reducing false positive regions among the n first candidate regions.

[0137] Specifically, since the target time is the time when the first 3D image is acquired, and the relative positional relationship of the i first objects at the target time is the target relative positional relationship, the relative positional relationship between the image regions corresponding to the i first objects in the first 3D image should match the target relative positional relationship. Therefore, based on the target relative positional relationship, the detection device determines n first candidate regions from y third candidate regions so that the relative positional relationship of the n first candidate regions matches the target relative positional relationship.

[0138] It should be understood that when the number of first objects is at least two, the number of first candidate regions is also at least two. Optionally, the number of first candidate regions is greater than or equal to the number of first objects. For example, i is 4, meaning the number of first objects is 4. The relative positional relationship of the targets includes: if the four first objects are the four vertices of a rectangle, then at least one rectangle can be constructed based on n first candidate regions. Specifically, when n is 4, one rectangle can be constructed. When n is greater than 4, at least two rectangles can be constructed. Figure 5 This is a schematic diagram of n first candidate regions provided in an embodiment of this application. Figure 5 In the example, n=4, and the n first candidate regions include the following 4 first candidate regions: first candidate region y1, first candidate region y2, first candidate region y3, and first candidate region y4. These 4 first candidate regions are the 4 vertices of a rectangle that can be constructed, that is, these 4 first candidate regions can be the 4 vertices of a rectangle. Figure 6 This is a schematic diagram of another set of n first candidate regions provided in an embodiment of this application. Figure 6 In the problem, n=7, and the n first candidate regions include the following 7 first candidate regions: y5, y6, y7, y8, y9, y10, and y11. These 7 first candidate regions can construct two rectangles. Specifically, first candidate regions y5, y6, y7, and y8 can construct one rectangle, and y5, y6, y7, and y8 are the four vertices of the constructed rectangle. Similarly, first candidate regions y6, y9, y10, and y11 can construct one rectangle, and y6, y9, y10, and y11 are the four vertices of the constructed rectangle.

[0139] As an optional implementation, before determining n first candidate regions from y third candidate regions based on the relative positional relationship of i first objects at the target time, the detection device further performs the following steps: Based on the matching degree between the shapes of the y third candidate regions and the shapes of the first objects, obtain at least one confidence level for each of the y third candidate regions, wherein the confidence level corresponds one-to-one with the third candidate region, and the confidence level is positively correlated with the matching degree. Select the region with the highest confidence level from the y third candidate regions as the initial region. After obtaining the y confidence levels of the y third candidate regions, determine n first candidate regions from the y third candidate regions based on the relative positional relationship of i first objects at the target time, including the following steps: Based on the initial region and the relative positional relationship of the target, determine j fourth candidate regions from the y third candidate regions, wherein the relative positional relationship between the initial region and the j fourth candidate regions matches the relative positional relationship of the target, and j is a positive integer less than n. Based on the initial region and the j fourth candidate regions, obtain n first candidate regions.

[0140] In this embodiment, the detection device first obtains the confidence level of each third candidate region based on the matching degree between the shape of each third candidate region and the shape of the first object. The confidence level of a third candidate region represents its confidence as an image region corresponding to the first object. Then, the region with the highest confidence level is selected from the y third candidate regions as the initial region, i.e., the third candidate region with the highest confidence level as the image region corresponding to the first object is selected as the initial region. Next, based on the relative positional relationship between the initial region and the target, j fourth candidate regions are determined from the y third candidate regions. Based on the initial region and the j fourth candidate regions, n first candidate regions are obtained. This reduces the data processing volume for obtaining n first candidate regions, increases the speed of obtaining n first candidate regions, and improves the accuracy of obtaining n first candidate regions.

[0141] For example, Figure 7 This is a schematic diagram of y third candidate regions provided in an embodiment of this application. Figure 7As shown, y=10, and the y third candidate regions include the following 10 third candidate regions: third candidate region q1, third candidate region q2, third candidate region q3, third candidate region q4, third candidate region q5, third candidate region q6, third candidate region q7, third candidate region q8, third candidate region q9, and third candidate region q10. If the relative positional relationship of the target includes the four vertices of a rectangle for the four first objects, then if third candidate region q2 is the third candidate region with the highest confidence among at least two third candidate regions, the detection device obtains at least one first candidate region based on third candidate region q2 and the relative positional relationship of the target. Specifically, third candidate regions q1, q2, q3, and q4 can construct a rectangle. Third candidate regions q2, q5, q6, and q7 can construct a rectangle. A rectangle cannot be constructed based on third candidate regions q2, q8, and other third candidate regions, nor can a rectangle be constructed based on third candidate regions q2, q9, and other third candidate regions. Therefore, the detection device obtains at least one first candidate region based on the relative positional relationship between the third candidate region q2 and the target, including: third candidate region q1, third candidate region q2, third candidate region q3, third candidate region q4, third candidate region q5, third candidate region q6, and third candidate region q7. In this case, n=7.

[0142] As an optional implementation, the first object is a sphere, and the first object is attached to the skin of the second object. For example, the first object is an optical sphere, and the second object is a person. The optical tracking device can determine the position of the optical sphere in the world coordinate system by tracking the optical sphere, and then, based on the position of the optical sphere in the world coordinate system and its position in the first three-dimensional image, can obtain the transformation relationship between the world coordinate system and the image coordinate system of the first three-dimensional image.

[0143] Before determining regions whose shapes match the shape of the first object from x second candidate regions to obtain y third candidate regions, the detection device further performs the following step: determining the skin region of the second object from the first three-dimensional image. Optionally, the detection device determines voxels in the first three-dimensional image whose semantic meaning is skin by performing semantic segmentation on the first three-dimensional image, and then determines the skin region of the second object based on the voxels whose semantic meaning is skin.

[0144] After determining the skin region of the second object, regions whose shapes match the shape of the first object are selected from x second candidate regions to obtain y third candidate regions. This includes the following steps: First, regions whose shapes match the shape of the first object are selected from the x second candidate regions to obtain r fifth candidate regions, where r is a positive integer, less than or equal to x, and greater than or equal to y. Second, the centers of the r fifth candidate regions are determined to obtain r centers. Third, based on the r centers, r sets of first rays are obtained, where each set of first rays corresponds one-to-one with a center, and each set of first rays is a set of rays with a center as its endpoint. Fourth, the proportion of rays intersecting the skin region in each first ray set is determined to obtain r proportions. Fifth, the first ray set corresponding to the proportions of the r proportions greater than a second threshold is determined as the second ray set. Sixth, based on the second ray set, y third candidate regions are determined from the r fifth candidate regions, where the center corresponding to the second ray set is the center of the third candidate region.

[0145] Because the first object is attached to the skin of the second object, the image region corresponding to the first object in the first 3D image will come into contact with the skin region of the second object. At this time, some rays radiating from the center of the first object will intersect with the skin region; that is, a large proportion of the rays radiating from the center of the first object will intersect with the skin region. Therefore, in this embodiment, the detection device first obtains r fifth candidate regions whose shapes match the shape of the first object from x second candidate regions, thus achieving preliminary screening. Then, the centers of the r fifth candidate regions are determined to obtain r centers, and based on these r centers, r sets of first rays are obtained. Optionally, the first ray sets include all rays radiating from the centers of the fifth candidate regions. Then, the proportion of rays intersecting with the skin region in the r sets of first rays is calculated to obtain r proportions. Finally, a second threshold is used to determine whether the proportion is large or small; specifically, a proportion greater than the second threshold indicates a large proportion, and a proportion less than or equal to the second threshold indicates a small proportion. Therefore, the detection device determines the first ray set corresponding to the proportion of r proportions that is greater than the second threshold as the second ray set. Then, based on the second ray set, it determines y third candidate regions from the r fifth candidate regions. The center corresponding to the second ray set is the center of the third candidate region. That is, among the rays with the center of the third candidate region as the endpoint, the proportion of rays intersecting with the skin region is greater than or equal to the second threshold. This reduces the number of false positive regions among the y third candidate regions.

[0146] As an optional implementation, after obtaining r fifth candidate regions, the detection device determines the contact area between the surface of each of the r fifth candidate regions and the skin region, obtaining r areas. The ratio of each of the r areas to the surface area of ​​the r fifth candidate regions is then determined, resulting in r first ratios. The first ratio among the r first ratios that is greater than a sixth threshold is determined as a second ratio. Based on the fifth candidate regions among the r fifth candidate regions that correspond to the second ratios, y third candidate regions are obtained.

[0147] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0148] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, while using clear signs / information to inform users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, personal information processing may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0149] The methods of the embodiments of this application have been described in detail above, and the apparatus of the embodiments of this application is provided below.

[0150] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a three-dimensional target detection device provided in an embodiment of this application. The three-dimensional target detection device 1 includes: an acquisition unit 11 and a processing unit 12, wherein:

[0151] Acquisition unit 11 is used to acquire the first three-dimensional image;

[0152] Processing unit 12 is configured to obtain at least one first candidate region by performing semantic segmentation on the first three-dimensional image, wherein the semantics of the voxels in the first candidate region are first objects;

[0153] The processing unit 12 is further configured to obtain at least one first volume of the at least one first candidate region based on the number of voxels in each first candidate region and the volume of voxels in the first three-dimensional image, wherein the first volume corresponds one-to-one with the first candidate region.

[0154] The processing unit 12 is further configured to obtain at least one target region based on the at least one first volume and the at least one first candidate region, wherein the target region is the region in the first three-dimensional image corresponding to the first object.

[0155] In conjunction with any embodiment of this application, the processing unit 12 is specifically used for:

[0156] A third volume is determined from the n first volumes, wherein the difference between the second volume and the second volume is less than or equal to a first threshold, and the second volume is obtained based on the actual volume of the first object;

[0157] Based on the regions corresponding to the third volume in the n first candidate regions, the m target regions are obtained.

[0158] In any embodiment of this application, where the first object is a sphere, the processing unit 12 is further configured to:

[0159] Determine the minimum distance between the voxels in the m target regions and the background region in the first 3D image to obtain m distance sets. The semantics of the voxels in the background region are not the first object. The distance sets correspond one-to-one with the target regions.

[0160] Based on the m distance sets, m distance subsets are obtained, each distance subset corresponding to one of the distance sets, and the distances in each distance subset are the k largest distances in the distance sets.

[0161] Based on the m distance subsets, m sub-regions are determined from the m target regions, and the sub-regions correspond one-to-one with the distance subsets, and the voxels in the sub-regions correspond to the distances in the distance subsets;

[0162] Based on the positions of voxels in the m sub-regions in the first 3D image, m sphere center positions are obtained, each corresponding to one of the sub-regions, and each sphere center position is the position of the sphere center of the first object in the first 3D image.

[0163] In conjunction with any embodiment of this application, the processing unit 12 is specifically used for:

[0164] Semantic segmentation is performed on the first three-dimensional image to obtain x second candidate regions. The semantics of the voxels in the second candidate regions are first objects, and x is a positive integer greater than or equal to n.

[0165] From the x second candidate regions, regions whose shapes match the shape of the first object are determined to obtain the n first candidate regions.

[0166] In any embodiment of this application, the number of the first objects is i, where i and n are both integers greater than or equal to 2;

[0167] The processing unit 12 is specifically used for:

[0168] From the x second candidate regions, regions whose shapes match the shape of the first object are determined to obtain y third candidate regions, where y is a positive integer, y is less than or equal to x, and y is greater than or equal to n;

[0169] Based on the relative positional relationship of the i first objects at the target time, n first candidate regions are determined from the y third candidate regions, where the target time is the time when the first three-dimensional image is acquired, and the relative positional relationship of the n first candidate regions matches the relative positional relationship of the target.

[0170] In conjunction with any embodiment of this application, the processing unit 12 is further configured to:

[0171] Based on the matching degree between the shape of the y third candidate regions and the shape of the first object, y confidence scores of the y third candidate regions are obtained. The confidence scores correspond one-to-one with the third candidate regions, and the confidence scores are positively correlated with the matching degree.

[0172] Select the region with the highest confidence level from the y third candidate regions as the initial region;

[0173] The processing unit 12 is further specifically used for:

[0174] Based on the relative positional relationship between the initial region and the target, j fourth candidate regions are determined from the y third candidate regions, wherein the relative positional relationship between the initial region and the j fourth candidate regions matches the relative positional relationship of the target, and j is a positive integer less than n;

[0175] Based on the initial region and the j fourth candidate regions, the n first candidate regions are obtained.

[0176] In any embodiment of this application, the first object is a ball, and the first object is attached to the skin of the second object;

[0177] The processing unit 12 is further configured to:

[0178] Determine the skin region of the second object from the first three-dimensional image;

[0179] The processing unit 12 is further specifically used for:

[0180] From the x second candidate regions, regions whose shapes match the shape of the first object are determined to obtain r fifth candidate regions, where r is a positive integer, r is less than or equal to x, and r is greater than or equal to y;

[0181] The centers of the r fifth candidate regions are determined respectively, resulting in r centers;

[0182] Based on the r centers, r sets of first rays are obtained, each set of first rays corresponding to one of the centers, and each set of first rays is a set of rays with the center as its endpoint.

[0183] The proportion of rays intersecting the skin region in each of the first ray sets is determined to obtain r proportions;

[0184] The first ray set corresponding to the proportion of the r proportions that is greater than the second threshold is determined as the second ray set;

[0185] Based on the second ray set, the y third candidate regions are determined from the r fifth candidate regions, and the center corresponding to the second ray set is the center of the third candidate region.

[0186] In this embodiment, after acquiring a first three-dimensional image, the detection device performs semantic segmentation on the first three-dimensional image to obtain n first candidate regions, where the semantic meaning of voxels in the first candidate regions is a first object. Then, based on the number of voxels in each of the n first candidate regions and the volume of voxels in the first three-dimensional image, n first volumes of the n first candidate regions are obtained. Finally, based on the n first volumes and the n first candidate regions, m target regions are obtained, where the target regions are the regions in the first three-dimensional image corresponding to the first object. This allows for the generation of m target regions based on the semantic meaning of voxels and the n first volumes, thereby reducing false positive regions among the m target regions and improving the accuracy of the m target regions.

[0187] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0188] Figure 9This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 also includes an input device 23 and an output device 24. The processor 21, memory 22, input device 23, and output device 24 are coupled together via connectors, which include various interfaces, transmission lines, or buses, etc., and are not limited in this embodiment. It should be understood that in the various embodiments of this application, coupling refers to mutual connection in a specific way, including direct connection or indirect connection through other devices, such as through various interfaces, transmission lines, buses, etc.

[0189] The processor 21 can be one or more graphics processing units (GPUs). If the processor 21 is a GPU, the GPU can be a single-core GPU or a multi-core GPU. Optionally, the processor 21 can be a processor group composed of multiple GPUs, with the multiple processors coupled to each other via one or more buses. Optionally, the processor can also be other types of processors, etc., which are not limited in this embodiment.

[0190] The memory 22 can be used to store computer program instructions, as well as various types of computer program code, including program code for executing the scheme of this application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), which is used for related instructions and data.

[0191] Input device 23 is used to input data and / or signals, and output device 24 is used to output data and / or signals. Input device 23 and output device 24 can be independent devices or an integrated device.

[0192] It is understood that in this embodiment of the application, the memory 22 can be used not only to store related instructions, but also to store related data. For example, the memory 22 can be used to store the first three-dimensional image obtained by the input device 23, or the memory 22 can also be used to store m target regions obtained by the processor 21, etc. This embodiment of the application does not limit the specific data stored in the memory.

[0193] Understandable Figure 9This illustration only shows a simplified design of an electronic device. In practical applications, the electronic device may also include other necessary components, including, but not limited to, any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of this application are within the protection scope of this application.

[0194] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0195] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will also readily understand that the various embodiments of this application have different focuses, and for the sake of convenience and brevity, the same or similar parts may not be repeated in different embodiments. Therefore, parts not described or not described in detail in one embodiment can be referred to the descriptions in other embodiments.

[0196] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0197] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0198] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0199] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0200] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A three-dimensional target detection method, characterized in that, The method includes: Obtain the first 3D image; By performing semantic segmentation on the first three-dimensional image, n first candidate regions are obtained. The semantics of the voxels in the first candidate regions are first objects. n is a positive integer. The number of first objects is i. i and n are both integers greater than or equal to 2. The step of semantically segmenting the first 3D image to obtain n first candidate regions includes: performing semantic segmentation on the first 3D image to obtain x second candidate regions, where the voxels in the second candidate regions are semantically defined as a first object, and x is a positive integer greater than or equal to n; determining regions whose shapes match the shape of the first object from the x second candidate regions to obtain the n first candidate regions; The step of determining regions whose shapes match the shape of the first object from the x second candidate regions to obtain the n first candidate regions includes: determining regions whose shapes match the shape of the first object from the x second candidate regions to obtain y third candidate regions, where y is a positive integer, y is less than or equal to x, and y is greater than or equal to n; obtaining y confidence levels of the y third candidate regions based on the matching degree between the shapes of the y third candidate regions and the shapes of the first object, where the confidence level corresponds one-to-one with the third candidate region, and the confidence level is positively correlated with the matching degree; selecting the region with the highest confidence level from the y third candidate regions as the initial region; determining j fourth candidate regions from the y third candidate regions based on the target relative position relationship between the initial region and i first objects at the target time, where the target time is the time when the first three-dimensional image was acquired, the relative position relationship between the initial region and the j fourth candidate regions matches the target relative position relationship, and j is a positive integer less than n; and obtaining the n first candidate regions based on the initial region and the j fourth candidate regions. Based on the number of voxels in each of the n first candidate regions and the volume of the voxels in the first three-dimensional image, n first volumes of the n first candidate regions are obtained, and each first volume corresponds one-to-one with a first candidate region. Based on the n first volumes and the n first candidate regions, m target regions are obtained. The target regions are the regions in the first three-dimensional image that correspond to the first object, and m is a positive integer less than or equal to n.

2. The method according to claim 1, characterized in that, The process of obtaining m target regions based on the n first volumes and the n first candidate regions includes: A third volume is determined from the n first volumes, wherein the difference between the second volume and the second volume is less than or equal to a first threshold, and the second volume is obtained based on the actual volume of the first object; Based on the regions corresponding to the third volume in the n first candidate regions, the m target regions are obtained.

3. The method according to claim 2, characterized in that, The first object is a sphere, and the method further includes: Determine the minimum distance between the voxels in the m target regions and the background region in the first 3D image to obtain m distance sets. The semantics of the voxels in the background region are not the first object. The distance sets correspond one-to-one with the target regions. Based on the m distance sets, m distance subsets are obtained, each distance subset corresponding to one of the distance sets, and the distances in the distance subsets are the k largest distances in the distance sets; Based on the m distance subsets, m sub-regions are determined from the m target regions, and the sub-regions correspond one-to-one with the distance subsets, and the voxels in the sub-regions correspond to the distances in the distance subsets; Based on the positions of voxels in the m sub-regions in the first 3D image, m sphere center positions are obtained, each corresponding to one of the sub-regions, and each sphere center position is the position of the sphere center of the first object in the first 3D image.

4. The method according to claim 1, characterized in that, The first object is a ball, and the first object is attached to the skin of the second object; Before determining regions whose shapes match the shape of the first object from the x second candidate regions to obtain y third candidate regions, the method further includes: Determine the skin region of the second object from the first three-dimensional image; The step of determining regions whose shapes match the shape of the first object from the x second candidate regions to obtain y third candidate regions includes: From the x second candidate regions, regions whose shapes match the shape of the first object are determined to obtain r fifth candidate regions, where r is a positive integer, r is less than or equal to x, and r is greater than or equal to y; The centers of the r fifth candidate regions are determined respectively, resulting in r centers; Based on the r centers, r sets of first rays are obtained, each set of first rays corresponding to one of the centers, and each set of first rays is a set of rays with the center as its endpoint. The proportion of rays intersecting the skin region in each of the first ray sets is determined to obtain r proportions; The first ray set corresponding to the proportion of the r proportions that is greater than the second threshold is determined as the second ray set; Based on the second ray set, the y third candidate regions are determined from the r fifth candidate regions, and the center corresponding to the second ray set is the center of the third candidate region.

5. A three-dimensional target detection device, characterized in that, The device includes: The acquisition unit is used to acquire the first three-dimensional image; The processing unit is configured to perform semantic segmentation on the first three-dimensional image to obtain n first candidate regions, wherein the semantics of the voxels in the first candidate regions are first objects, the number of the first objects is i, and i and n are both integers greater than or equal to 2. The step of semantically segmenting the first 3D image to obtain n first candidate regions includes: performing semantic segmentation on the first 3D image to obtain x second candidate regions, where the voxels in the second candidate regions are semantically defined as a first object, and x is a positive integer greater than or equal to n; determining regions whose shapes match the shape of the first object from the x second candidate regions to obtain the n first candidate regions; The step of determining regions whose shapes match the shape of the first object from the x second candidate regions to obtain the n first candidate regions includes: determining regions whose shapes match the shape of the first object from the x second candidate regions to obtain y third candidate regions, where y is a positive integer, y is less than or equal to x, and y is greater than or equal to n; obtaining y confidence levels of the y third candidate regions based on the matching degree between the shapes of the y third candidate regions and the shapes of the first object, where the confidence level corresponds one-to-one with the third candidate region, and the confidence level is positively correlated with the matching degree; selecting the region with the highest confidence level from the y third candidate regions as the initial region; determining j fourth candidate regions from the y third candidate regions based on the target relative position relationship between the initial region and i first objects at the target time, where the target time is the time when the first three-dimensional image was acquired, the relative position relationship between the initial region and the j fourth candidate regions matches the target relative position relationship, and j is a positive integer less than n; and obtaining the n first candidate regions based on the initial region and the j fourth candidate regions. The processing unit is further configured to obtain n first volumes of the n first candidate regions based on the number of voxels in each of the n first candidate regions and the volume of the voxels in the first three-dimensional image, wherein each first volume corresponds one-to-one with a first candidate region. The processing unit is further configured to obtain m target regions based on the n first volumes and the n first candidate regions, wherein the target regions are the regions in the first three-dimensional image that correspond to the first object.

6. A surgical robot, characterized in that, Includes the three-dimensional target detection device as described in claim 5.

7. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs the method as described in any one of claims 1 to 4.

Images