Method and device for detecting landmark points on medical images, and electronic equipment

Abstract

The present invention provides a method and device for detecting marker points on a medical image, an electronic device, and a surgical robot system. A method for detecting marker points on a medical image comprises: determining N valid candidate marker point areas based on an initial marker point candidate area determined on the medical image, where N is an integer greater than zero; determining the centroid of each valid candidate marker point area in the N valid marker point candidate areas, so as to use the centroid as the center of the corresponding valid marker point candidate marker point area; matching the centers of the N valid candidate marker point areas with the centers of the corresponding actual marker points, so as to screen out K valid candidate marker point areas from the N valid candidate marker point areas; and detecting valid marker points on the medical image according to the centers of the K valid candidate marker point areas, thereby improving the accuracy of marker point detection.

Description

Technical Field

[0001] The present invention relates to the technical field of medical robots, and in particular to a method and device for detecting landmarks on medical images, electronic equipment, and a surgical robot system. Background Art

[0002] In order for the surgical robot to accurately locate and perform surgery on the patient according to the preoperative surgical plan, it is necessary to attach landmark points to the patient's body for medical image capture and detect the landmark points on the medical image so that the surgical process can be performed based on the detected landmark points. The accuracy of the detected landmark points greatly affects the accuracy of the surgery. Therefore, how to improve the accuracy of landmark point detection has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] Embodiments of the present invention provide a method and device for detecting landmarks on a medical image, an electronic device, and a surgical robot system, which are used to overcome or alleviate the above-mentioned problems in the prior art.

[0004] The technical solution adopted by the present invention is:

[0005] A method for detecting landmark points on a medical image, comprising:

[0006] Determine N valid candidate marker point regions based on the initial marker point candidate regions determined on the medical image, where N is an integer greater than zero;

[0007] Determine the centroid of each valid candidate marker point region among the N valid marker point candidate regions, and use the centroid as the center of the valid marker point candidate marker point region;

[0008] Matching the centers of the N valid candidate marker point regions with the centers of the corresponding actual marker points to screen out K valid candidate marker point regions from the N valid candidate marker point regions;

[0009] Valid landmark points on the medical image are detected according to the centers of the K valid candidate landmark point regions.

[0010] Optionally, determining N valid candidate marker point regions based on the initial marker point candidate regions determined on the medical image includes: performing iterative growth of candidate regions and iterative screening of regions based on the initial marker point candidate regions to determine N valid candidate marker point regions.

[0011] Optionally, the performing iterative growth of candidate regions and iterative screening of regions based on the initial marker point candidate regions to determine N valid candidate marker point regions includes:

[0012] During the j-th regional iterative growth, the target object is each candidate growth region obtained after the j-1-th regional iteration, and regional growth is performed on each candidate growth region obtained after the j-1-th iteration, 2≤j≤M, M is the total number of regional iterative growth;

[0013] According to the set regional constraints, the candidate growth regions obtained after the j-th regional iterative growth are screened to serve as the objects targeted during the j+1-th regional iteration, until N valid candidate landmark point regions are determined.

[0014] Optionally, the region growing threshold used in the j-th region iteration is different from that used in the j-1-th region iteration.

[0015] Optionally, the regional constraints include three-dimensional volume constraints and three-dimensional specification size constraints, so that when the growth candidate regions obtained after the j-th regional iterative growth are screened, the growth candidate regions that meet the three-dimensional volume constraints and the three-dimensional specification size constraints are used as the objects to be targeted during the j+1-th regional iteration.

[0016] Optionally, the method further includes: calculating the volume and three-dimensional specification size of the corresponding growth candidate region based on the voxel values ​​in the growth candidate region obtained after the j-th regional iterative growth is completed, so as to compare them with the three-dimensional volume constraint condition and the three-dimensional specification size constraint condition respectively.

[0017] Optionally, the three-dimensional size constraint condition includes: a ratio between the length, width and height of the growth candidate area.

[0018] Optionally, the method further includes: determining whether the total number of candidate growth regions obtained by the j-th regional iterative growth is not greater than a set limit on the total number of regions that can be obtained in a single regional growing process; if so, initiating the j-1-th regional iterative growth.

[0019] Optionally, the method also includes: determining whether the total number of growing candidate regions obtained by iterative growth of all sub-regions is not greater than the limit value of the total number of regions that can be obtained by multiple region growing processes; if so, executing the step of determining the center of mass of each valid candidate marker point region among the N valid marker point candidate regions.

[0020] Optionally, determining the centroid of each of the N valid marker point candidate areas includes: determining the corresponding centroid according to pixel values ​​and coordinates of pixel points in each of the N valid marker point candidate areas.

[0021] Optionally, the valid landmark points on the medical image are detected according to the centers of the K valid candidate landmark point regions, and then the method includes: determining the total number of the valid landmark points, and if the total number is less than a set total number threshold, performing region growing processing on the initial landmark point candidate region to use the corresponding growth region as a new initial landmark point candidate region, so as to re-determine N valid candidate landmark point regions.

[0022] Optionally, matching the centers of the N valid candidate marker point areas with the centers of the corresponding actual marker points to screen out K valid candidate marker point areas from the N valid candidate marker point areas includes: determining the single-point error between the center of each valid candidate marker point area in the N valid candidate marker point areas and the center of the corresponding actual marker point, and retaining the K valid candidate marker point areas whose single-point errors are less than a set single-point error threshold.

[0023] Optionally, matching the centers of the N valid candidate marker point areas with the centers of the corresponding actual marker points to screen out K valid candidate marker point areas from the N valid candidate marker point areas includes: determining the single-point error and the average error between the center of each valid candidate marker point area in the N valid candidate marker point areas and the center of the corresponding actual marker point, and retaining the K valid candidate marker point areas whose single-point error is less than a set single-point error threshold and whose average error is less than the average error threshold.

[0024] A device for detecting landmark points on a medical image, comprising:

[0025] A first processing unit, configured to determine N valid candidate marker point regions based on the initial marker point candidate region determined on the medical image, where N is an integer greater than zero;

[0026] A second processing unit is used to determine the centroid of each valid candidate marker point region in the N valid marker point candidate regions, so as to use the centroid as the center of the valid marker point candidate marker point region;

[0027] a third processing unit, configured to match the centers of the N valid candidate marker point regions with the centers of the corresponding actual marker points, so as to select K valid candidate marker point regions from the N valid candidate marker point regions;

[0028] The fourth processing unit is used to detect valid landmark points on the medical image according to the centers of the K valid candidate landmark point regions.

[0029] A computer storage medium stores a computer program for implementing any method described in any one of the embodiments of the present application.

[0030] An electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to run the computer program to implement any method described in any one of the embodiments of the present application.

[0031] A surgical robot system comprises an image processing device, wherein the image processing device is used to execute the method described in any one of the embodiments of the present application to detect landmarks, so as to perform a surgical procedure based on the landmarks.

[0032] In the technical solution provided by the embodiment of the present invention, N valid candidate marker point regions are determined based on the initial marker point candidate regions determined on the medical image, where N is an integer greater than zero; the centroid of each valid candidate marker point region among the N valid marker point candidate regions is determined, so that the centroid is used as the center of the corresponding valid marker point candidate marker point region; the centers of the N valid candidate marker point regions are matched with the centers of the corresponding actual marker points, so as to screen out K valid candidate marker point regions from the N valid candidate marker point regions; and the valid marker points on the medical image are detected according to the centers of the K valid candidate marker point regions, thereby improving the accuracy of marker point detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1A This is a flow chart of a method for detecting landmark points on a medical image according to an embodiment of the present application.

[0034] Figure 1B A schematic diagram of an iterative growth process according to an embodiment of the present application.

[0035] Figure 1C A schematic diagram of a process for screening valid candidate landmark point areas according to an embodiment of the present application.

[0036] Figure 2 This is a schematic diagram of the structure of a device for detecting landmarks on a medical image according to an embodiment of the present application;

[0037] Figure 3 This is a schematic diagram of the specific hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0039] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, devices, steps, etc. may be adopted. In other cases, well-known structures, methods, devices, implementations or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0040] In addition, the terms "estimated", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "estimated" or "second" may explicitly or implicitly include one or more of the features. In the description of the present disclosure, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. The symbol " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0041] In this disclosure, unless otherwise clearly specified and limited, terms such as "connection" should be understood in a broad sense, for example, it can be an electrical connection or can communicate with each other; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in this disclosure can be understood according to specific circumstances.

[0042] In the technical solution provided by the embodiment of the present invention, N valid candidate marker point regions are determined based on the initial marker point candidate regions determined on the medical image, where N is an integer greater than zero; the centroid of each valid candidate marker point region among the N valid marker point candidate regions is determined, so that the centroid is used as the center of the corresponding valid marker point candidate marker point region; the centers of the N valid candidate marker point regions are matched with the centers of the corresponding actual marker points, so as to screen out K valid candidate marker point regions from the N valid candidate marker point regions; and the valid marker points on the medical image are detected according to the centers of the K valid candidate marker point regions, thereby improving the accuracy of marker point detection.

[0043] Figure 1A FIG. 1 is a flow chart of a method for detecting landmarks on a medical image according to an embodiment of the present application; Figure 1A As shown, it includes:

[0044] S101, determining N valid candidate marker point regions based on the initial marker point candidate regions determined on the medical image, where N is an integer greater than zero;

[0045] Optionally, determining N valid candidate marker point regions based on the initial marker point candidate regions determined on the medical image includes: performing iterative growth of candidate regions and iterative screening of regions based on the initial marker point candidate regions to determine N valid candidate marker point regions.

[0046] In this embodiment, the initial marker point candidate region may be a region including marker points on a medical image. The medical image may be, for example, a CT image. By determining N valid candidate marker point regions based on the initial marker point candidate region, and performing the determination within a smaller and more accurate image range, the amount of data to be processed may be reduced, the detection time may be reduced, and the detection accuracy may be improved.

[0047] Figure 1B This is a schematic diagram of an iterative growth process of an embodiment of the present application. Figure 1B As shown, the iterative growth of candidate regions and iterative screening of regions based on the initial marker point candidate regions to determine N valid candidate marker point regions include:

[0048] S111, during the j-th regional iterative growth, the target object is each growth candidate region obtained after the j-1-th regional iteration is completed, and regional growth is performed on each growth candidate region obtained after the j-1-th iteration is completed;

[0049] In this embodiment, 2≤j≤M, where M is the total number of iterative growth of the region.

[0050] S121. According to the set regional constraint conditions, the candidate growth regions obtained after the j-th regional iterative growth are screened to serve as the objects targeted during the j+1-th regional iteration, until N valid candidate landmark point regions are determined.

[0051] In this embodiment, the problem of too many candidate landmark areas causing a long subsequent matching time can be reduced by setting area constraints.

[0052] Optionally, the region growing thresholds used in the j-th region iteration and the j-1-th region iteration are different, that is, the region growing thresholds used in two adjacent iterations are different, so that the candidate landmark point regions obtained by multiple region growing processes can cover all actual landmark points, while solving the problems that a single region growing process cannot avoid large differences in the pixel ranges of landmark points and cannot identify all of them, and the problem that a fixed threshold leads to a long detection time.

[0053] Here, it should be noted that the size of the region growing threshold used in the j-th region iteration and the j-1-th region iteration can be flexibly set according to the requirements of the application scenario.

[0054] Optionally, the regional constraints include three-dimensional volume constraints and three-dimensional specification size constraints, so that when the growth candidate regions obtained after the j-th regional iterative growth are screened, the growth candidate regions that meet the three-dimensional volume constraints and the three-dimensional specification size constraints are used as the objects to be targeted during the j+1-th regional iteration.

[0055] Optionally, the method further comprises: calculating the volume and three-dimensional size of the corresponding growth candidate region from the voxel values ​​in the growth candidate region obtained after the j-th regional iterative growth is completed, so as to compare them with the three-dimensional volume constraint condition and the three-dimensional size constraint condition respectively. This step may be performed before screening the growth candidate region obtained after the j-th regional iterative growth is completed, or may be included in screening the growth candidate region obtained after the j-th regional iterative growth is completed.

[0056] Optionally, the three-dimensional size constraint condition includes: a ratio of the length, width and height of the growth candidate region, and the length, width and height of the growth candidate region may be specifically based on a voxel value in the growth candidate region.

[0057] The three-dimensional volume constraint condition and the three-dimensional specification size constraint condition can be set according to empirical values. During screening, they can be a single value or a range of values, so that the growth candidate regions exceeding the single value or the range of values ​​are screened out and only the growth candidate regions that do not exceed the single value or the range of values ​​are retained.

[0058] As mentioned above, the medical image is a CT image, which has voxel values ​​(or also called CT values) with three-dimensional characteristics, so that the volume and three-dimensional specifications of the candidate growth area can be calculated based on the voxel values.

[0059] Of course, it should be noted here that CT images are merely examples of medical images and are not exclusive limitations. In other embodiments, medical images may include any image that can calculate the volume and three-dimensional specifications of the candidate growth area.

[0060] In this embodiment, the candidate growth regions are screened by comparing the volume and three-dimensional specification size of the candidate growth regions with the three-dimensional volume constraint conditions and the three-dimensional specification size constraint conditions, thereby avoiding the problem of long matching time due to too many candidate landmark point regions.

[0061] Optionally, the method further includes: judging whether the total number of candidate regions obtained by the j-th iterative regional growth is not greater than the set limit of the total number of regions that can be obtained in a single regional growth process, and if so, starting the j-1-th iterative regional growth. This judging step can be included in the j-th iterative regional growth step, thereby limiting the number of candidate landmark regions obtained by each regional growth, and avoiding the problem of long matching time due to too many candidate landmark regions.

[0062] Optionally, the method further includes: judging whether the total number of candidate regions obtained by iterative growth of all sub-regions is not greater than the total number of regions that can be obtained by multiple region growing processes, and if so, performing the step of determining the centroid of each valid candidate marker point region in the N valid marker point candidate regions. The judging step may be performed after all sub-region iterative growth is completed, or during the last region iterative growth, so as to limit the total number of candidate marker point regions obtained by all sub-region growth, and avoid the problem of long matching time due to too many total number of candidate marker point regions.

[0063] In this embodiment, the total number of region growing operations can be set according to the application scenario, or a certain total number threshold can be set, and when the total number threshold is reached, the region growing process is stopped. Alternatively, it can also be set according to the number of landmarks to be finally detected.

[0064] S102, determining the centroid of each valid candidate marker point region among the N valid marker point candidate regions, and taking the centroid as the center of the valid marker point candidate region;

[0065] Optionally, determining the centroid of each of the N valid marker point candidate areas includes: determining the corresponding centroid according to pixel values ​​and coordinates of pixel points in each of the N valid marker point candidate areas.

[0066] Specifically, it can be determined by referring to the following formula (1):

[0067]

[0068] m i Represents the pixel value of the Lth point in a valid landmark candidate area, r i Represents the coordinate value of the Lth point in a valid landmark candidate area, R m Represents the centroid of the candidate region corresponding to the valid landmark point.

[0069] It should be noted that in this embodiment, only formula (1) is used as an example for explanation, but it is not limited to a unique method. In other embodiments, other methods can also be used to calculate the centroid.

[0070] In this embodiment, the centroid is calculated based on the pixel value and coordinates of the pixel point, which can reflect the area with the highest pixel in the candidate marker point area, and can also reduce the problem of large pixel span range in the effective marker point area and deviation from the center of the actual marker point due to artifacts. The large pixel span range and artifacts include but are not limited to images caused by CT accuracy, patient shaking, etc.

[0071] S103 . Match the centers of the N valid candidate marker point regions with the centers of the corresponding actual marker points to select K valid candidate marker point regions from the N valid candidate marker point regions.

[0072] In this embodiment, 1≤K≤N.

[0073] In this embodiment, the centers of the N valid candidate marker point areas are matched with the centers of the corresponding actual marker points to screen out K valid candidate marker point areas from the N valid candidate marker point areas, including: determining the single-point error between the center of each valid candidate marker point area in the N valid candidate marker point areas and the center of the corresponding actual marker point, and retaining the K valid candidate marker point areas whose single-point errors are less than a set single-point error threshold.

[0074] or, Figure 1C This is a schematic diagram of a process for screening valid candidate landmark areas according to an embodiment of the present application. Figure 1C As shown, matching the centers of the N valid candidate marker point regions with the centers of the corresponding actual marker points to screen out K valid candidate marker point regions from the N valid candidate marker point regions includes:

[0075] S113, determining a single point error and an average error between the center of each valid candidate marker point region and the center of the corresponding actual marker point in the N valid candidate marker point regions;

[0076] S123, retaining K valid candidate marker point regions corresponding to single point errors less than a set single point error threshold and average errors less than an average error threshold.

[0077] Optionally, the single point error between the center of each valid candidate marker point region and the center of the corresponding actual marker point can be obtained by directly comparing the coordinate distance between the center of the valid candidate marker point region and the center of the corresponding actual marker point. Assume that the center of the valid candidate marker point region (x 11 ,y 11 , z 11 ) is the center of the valid candidate landmark point A 11 The three-dimensional coordinates in the world coordinate system, (x 21 ,y 21 , z21 ) is the corresponding actual landmark center B 1 , then the distance between two points (as a single point error) formula (2) is:

[0078]

[0079] Optionally, the average error between the center of each valid candidate marker point area and the center of the corresponding actual marker point can be calculated, for example, by calculating the average of all single point errors. Assuming that there are N valid candidate marker points, there are N single point errors. The average error of these N single point errors is calculated according to the following formula (3): A 1n represents the center of the nth valid candidate landmark point, B 2n Represents the center of the nth actual landmark point, 1≤n≤N.

[0080] Optionally, the three-dimensional coordinates of each actual marking point are preset, for example, by pasting the marking point on the guide plate and presetting the three-dimensional coordinates of the marking point based on the position of the marking point on the guide plate.

[0081] Based on the above-mentioned single point error and / or average error, the candidate landmark point areas that meet the error conditions can be retained, and valid landmark points are detected only based on the K valid candidate landmark point areas with higher accuracy, thereby further improving the detection accuracy and ensuring that the surgical robot system can complete the alignment work more accurately, thereby ensuring the smooth implementation of the surgical process.

[0082] S104: Detect valid landmark points on the medical image according to the centers of the K valid candidate landmark point regions.

[0083] In this embodiment, the center is determined according to the shape of the actual mark point. For example, if the actual mark point is a circle, the center is the center of the circle. To this end, when executing step S104, a circular area can be determined as a valid mark point with the center as the center of the circle. For example, a number of pixels outside the center are selected to form a circular area. For example, a number of pixels outside the center are limited based on the radius of the actual mark point.

[0084] Of course, in other embodiments, other methods may be used to limit a number of pixels outside the center.

[0085] Furthermore, the valid landmark points on the medical image are detected according to the centers of the K valid candidate landmark point regions, and then the method includes: determining the total number of the valid landmark points; if the total number is less than a set total number threshold, performing region growing processing on the initial landmark point candidate region to use the corresponding growth region as a new initial landmark point candidate region, so as to re-determine N valid candidate landmark point regions, until the detected valid landmark points reach the set total number threshold, thereby ensuring the success rate of the implementation of the detection scheme.

[0086] In this embodiment, the total number threshold can be determined according to the application scenario as long as the smooth implementation of the operation can be guaranteed.

[0087] It should be noted that in the above embodiments, the specific numerical values ​​are merely examples and are not intended to be exclusive limitations.

[0088] Figure 2 Schematic diagram of the structure of a device for detecting landmarks on a medical image according to an embodiment of the present application; Figure 2 As shown, it includes:

[0089] A first processing unit 201 is configured to determine N valid candidate marker point regions based on the initial marker point candidate regions determined on the medical image, where N is an integer greater than zero;

[0090] The second processing unit 202 is configured to determine the centroid of each valid candidate marker point region among the N valid marker point candidate regions, so as to use the centroid as the center of the valid marker point candidate region;

[0091] The third processing unit 203 is used to match the centers of the N valid candidate marker point regions with the centers of the corresponding actual marker points, so as to select K valid candidate marker point regions from the N valid candidate marker point regions;

[0092] The fourth processing unit 204 is configured to detect valid landmark points on the medical image according to the centers of the K valid candidate landmark point regions.

[0093] Optionally, the first processing unit 201 is specifically configured to: perform iterative growth of candidate regions and iterative screening of regions based on the initial marker point candidate regions to determine N valid candidate marker point regions.

[0094] The method according to claim 2, characterized in that the first processing unit 201 is specifically used to:

[0095] During the j-th regional iterative growth, the target object is each candidate growth region obtained after the j-1-th regional iteration, and regional growth is performed on each candidate growth region obtained after the j-1-th iteration, 2≤j≤M, M is the total number of regional iterative growth;

[0096] According to the set regional constraints, the candidate growth regions obtained after the j-th regional iterative growth are screened to serve as the objects targeted during the j+1-th regional iteration, until N valid candidate landmark point regions are determined.

[0097] Optionally, the region growing threshold used in the j-th region iteration is different from that used in the j-1-th region iteration.

[0098] Optionally, the regional constraints include three-dimensional volume constraints and three-dimensional specification size constraints, so that when the growth candidate regions obtained after the j-th regional iterative growth are screened, the growth candidate regions that meet the three-dimensional volume constraints and the three-dimensional specification size constraints are used as the objects to be targeted during the j+1-th regional iteration.

[0099] Optionally, the first processing unit 201 is also used to: calculate the volume and three-dimensional specification size of the corresponding growth candidate region based on the voxel values ​​in the growth candidate region obtained after the j-th region iterative growth is completed, so as to compare them with the three-dimensional volume constraint condition and the three-dimensional specification size constraint condition respectively.

[0100] Optionally, the three-dimensional size constraint condition includes: a ratio between the length, width and height of the growth candidate area.

[0101] Optionally, the first processing unit 201 is also used to determine whether the total number of candidate growth regions obtained by the j-th regional iterative growth is not greater than the set limit on the total number of regions that can be obtained in a single regional growing process, and if so, start the j-1-th regional iterative growth.

[0102] Optionally, the first processing unit 201 is also used to: determine whether the total number of growing candidate regions obtained by iterative growth of all sub-regions is not greater than the limit value of the total number of regions that can be obtained by multiple region growing processes; if so, execute the step of determining the center of mass of each valid candidate marker point region among the N valid marker point candidate regions.

[0103] Optionally, the second processing unit 202 is specifically configured to determine a corresponding centroid according to pixel values ​​and coordinates of pixels in each valid candidate marker point region of the N valid marker point candidate regions.

[0104] Optionally, the fourth processing unit 204 is also used to detect the valid landmark points on the medical image according to the centers of the K valid candidate landmark point regions, and determine the total number of the valid landmark points. If the total number is less than a set total number threshold, perform region growing processing on the initial landmark point candidate region to use the corresponding growth region as a new initial landmark point candidate region, so as to re-determine the N valid candidate landmark point regions.

[0105] Optionally, the third processing unit 203 is specifically used to determine the single-point error between the center of each valid candidate marker point area among the N valid candidate marker point areas and the center of the corresponding actual marker point, and retain K valid candidate marker point areas corresponding to the single-point errors that are less than a set single-point error threshold.

[0106] Optionally, the third processing unit 203 is specifically used to determine the single-point error and the average error between the center of each valid candidate marker point area among the N valid candidate marker point areas and the center of the corresponding actual marker point, and retain the K valid candidate marker point areas whose single-point errors are less than a set single-point error threshold and whose average errors are less than the average error threshold.

[0107] Figure 3 Schematic diagram of the specific hardware structure of an electronic device according to an embodiment of the present application; Figure 3 As shown, the electronic device may include: a processor (processor) 302 , a communication interface (Communications Interface) 304 , a memory (memory) 306 , and a communication bus 308 .

[0108] in:

[0109] The processor 302 , the communication interface 304 , and the memory 306 communicate with each other via the communication bus 308 .

[0110] The communication interface 304 is used to communicate with other electronic devices or servers.

[0111] The processor 302 is used to execute the program 310, and specifically can execute the relevant steps in the above-mentioned verification code generation method embodiment.

[0112] Specifically, the program 310 may include program codes, which include computer operation instructions.

[0113] The processor 302 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0114] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0115] The program 310 can be specifically used to enable the processor 302 to execute the method described in any of the above embodiments of the present application.

[0116] The specific implementation of each step in the program 310 can refer to the corresponding description of the corresponding steps and units in the method embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the aforementioned method embodiment, which will not be repeated here.

[0117] An embodiment of the present application further provides a computer storage medium, on which is stored a computer program for implementing any method described in any one of the embodiments of the present application.

[0118] An embodiment of the present application also provides a surgical robot system, which includes an image processing device, wherein the image processing device is used to execute the method described in any one of the embodiments of the present application to detect landmark points, so as to perform a surgical procedure based on the landmark points.

[0119] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the verification code generation method described herein is implemented. In addition, when a general-purpose computer accesses a code for implementing the verification code generation method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the verification code generation method shown here.

[0120] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0121] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. A method for detecting landmarks on a medical image, characterized in that: include: Determine N valid candidate marker point regions based on the initial marker point candidate regions determined on the medical image, where N is an integer greater than zero; Determine the centroid of each valid candidate marker point region among the N valid candidate marker point regions, and use the centroid as the center of the corresponding valid candidate marker point region; The centroid of the valid candidate landmark area is determined based on the following formula: mi represents the pixel value of the Lth point in a valid candidate marker point area, ri represents the coordinate value of the Lth point in a valid candidate marker point area, and Rm represents the centroid of the corresponding valid candidate marker point area; Matching the centers of the N valid candidate marker point regions with the centers of the corresponding actual marker points to screen out K valid candidate marker point regions from the N valid candidate marker point regions, including: determining the single point error and the average error between the center of each valid candidate marker point region in the N valid candidate marker point regions and the center of the corresponding actual marker point, and retaining the K valid candidate marker point regions corresponding to the single point error being less than a set single point error threshold and the average error being less than the average error threshold; wherein the average error is the average value of all the single point errors; The single-point error between the center of each valid candidate marker point area and the center of the corresponding actual marker point is obtained by directly comparing the coordinate distance between the center of the valid candidate marker point area and the center of the corresponding actual marker point. The center of the valid candidate marker point area (x11, y11, z11) is the three-dimensional coordinate of the center of the valid candidate marker point A11 in the world coordinate system, and (x21, y21, z21) is the corresponding actual marker point center B1. The formula for the distance between the two points as the single-point error is: Valid marker points on the medical image are detected according to the centers of the K valid candidate marker point regions.

2. The method according to claim 1, characterized in that: The determining of N valid candidate marker point regions based on the initial marker point candidate regions determined on the medical image includes: performing iterative growth of candidate regions and iterative screening of regions based on the initial marker point candidate regions to determine N valid candidate marker point regions.

3. The method according to claim 2, characterized in that The iterative growth of candidate regions and iterative screening of regions based on the initial marker point candidate regions to determine N valid candidate marker point regions include: During the j-th regional iterative growth, the target object is each candidate growth region obtained after the j-1-th regional iteration, and regional growth is performed on each candidate growth region obtained after the j-1-th iteration, 2≤j≤M, M is the total number of regional iterative growth; According to the set regional constraints, the candidate growth regions obtained after the j-th regional iterative growth are screened to serve as the objects targeted during the j+1-th regional iteration, until N valid candidate landmark point regions are determined.

4. The method according to claim 3, characterized in that The region growing thresholds used in the j-th region iteration and the j-1-th region iteration are different.

5. The method according to claim 3, characterized in that: The regional constraints include three-dimensional volume constraints and three-dimensional specification size constraints, so that when the growth candidate regions obtained after the j-th regional iterative growth are screened, the growth candidate regions that meet the three-dimensional volume constraints and the three-dimensional specification size constraints will be used as the objects to be targeted during the j+1-th regional iteration.

6. The method according to claim 5, characterized in that The method further comprises: The voxel values ​​in the growth candidate region obtained after the j-th region iterative growth are used to calculate the volume and three-dimensional specification size of the corresponding growth candidate region, so as to compare them with the three-dimensional volume constraint condition and the three-dimensional specification size constraint condition respectively.

7. The method according to claim 5, characterized in that The three-dimensional size constraint condition includes: the ratio between the length, width and height of the growth candidate area.

8. The method according to claim 7, characterized in that The method further comprises: Determine whether the total number of growing candidate regions obtained by iterative growth of all sub-regions is not greater than the limit value of the total number of regions that can be obtained by multiple region growing processes. If so, execute the step of determining the centroid of each valid candidate marker point region among the N valid candidate marker point regions.

9. The method according to claim 1, characterized in that: The determining the centroid of each valid candidate marker point region among the N valid candidate marker point regions comprises: The corresponding centroid is determined according to the pixel value and coordinates of the pixel point in each valid candidate marker point area of ​​the N valid candidate marker point areas.

10. The method according to claim 1, characterized in that The method detects valid marker points on the medical image based on the centers of the K valid candidate marker point regions, and then includes: determining the total number of the valid marker points, and if the total number is less than a set total number threshold, performing region growing processing on the initial marker point candidate region to use the corresponding grown region as a new initial marker point candidate region, so as to re-determine N valid candidate marker point regions.

11. A device for detecting landmark points on a medical image, characterized in that: include: A first processing unit, configured to determine N valid candidate marker point regions based on the initial marker point candidate region determined on the medical image, where N is an integer greater than zero; A second processing unit is used to determine the centroid of each valid candidate marker point region among the N valid candidate marker point regions, so as to use the centroid as the center of the corresponding valid candidate marker point region; a third processing unit, configured to match the centers of the N valid candidate marker point regions with the centers of the corresponding actual marker points, so as to select K valid candidate marker point regions from the N valid candidate marker point regions; a fourth processing unit, configured to detect valid marker points on the medical image according to the centers of the K valid candidate marker point regions; Wherein, the centroid of the valid candidate landmark point area is determined based on the following formula: mi represents the pixel value of the Lth point in a valid candidate marker point area, ri represents the coordinate value of the Lth point in a valid candidate marker point area, and Rm represents the centroid of the corresponding valid candidate marker point area; Matching the centers of the N valid candidate marker point regions with the centers of the corresponding actual marker points to screen out K valid candidate marker point regions from the N valid candidate marker point regions, including: determining the single point error and the average error between the center of each valid candidate marker point region in the N valid candidate marker point regions and the center of the corresponding actual marker point, and retaining the K valid candidate marker point regions corresponding to the single point error being less than a set single point error threshold and the average error being less than the average error threshold; wherein the average error is the average value of all the single point errors; The single-point error between the center of each valid candidate marker point area and the center of the corresponding actual marker point is obtained by directly comparing the coordinate distance between the center of the valid candidate marker point area and the center of the corresponding actual marker point. The center of the valid candidate marker point area (x11, y11, z11) is the three-dimensional coordinate of the center of the valid candidate marker point A11 in the world coordinate system, and (x21, y21, z21) is the corresponding actual marker point center B1. The formula for the distance between the two points as the single-point error is:

12. A computer storage medium, characterized in that: The computer storage medium stores a computer program for implementing the method according to any one of claims 1 to 10.

13. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to run the computer program to implement the method according to any one of claims 1 to 10.

14. A surgical robot system, characterized in that: include: image processing device, The image processing device is used to execute the method according to any one of claims 1 to 10 to detect the landmark points, so as to perform a surgical procedure based on the landmark points.

Images