Hip joint image segmentation method and device and surgical robot system

Abstract

The invention relates to a hip joint image segmentation method and device and a surgical robot system, and the method comprises the steps: obtaining a medical image of a hip joint; determining a segmentation model matched with an identification result according to whether the identification result of the metal prosthesis exists in the medical image; segmenting the medical image based on the segmentation model to obtain segmented hip joint regions; and based on the segmented hip joint region, identifying a segmented image of the hip joint. According to the method, the most suitable segmentation model can be selected for segmentation of the hip joint, so that the method can be suitable for different hip joint lesion scenes, the segmentation precision in a complex hip joint scene is improved, and the hip joint segmentation difficulty in the complex hip joint scene is reduced.

Description

Technical Field

[0001] The present application relates to the field of medical image processing, and in particular to a hip joint image segmentation method, device and surgical robot system. Background Art

[0002] Complex hip joints usually refer to hip joints with severe damage and deformity due to developmental abnormalities, fractures or arthritis. Hip replacement is one of the most widely used joint replacement surgeries. For hip joints with severe pathological conditions, hip replacement can be performed to restore hip joint function using an artificial hip joint, which is clinically defined as a restorative hip joint. Usually, the prosthesis in an artificial hip joint includes an acetabular cup, liner, ball head and femoral stem, and commonly used materials include metal materials such as titanium alloy, cobalt-chromium-molybdenum alloy, etc.

[0003] The segmentation of the hip joint area from medical images has a wide range of clinical applications. For damaged or deformed hip joint structures, or for hip joint structures using artificial hip joints, it is difficult to segment the hip joint area from medical images. Specifically, the quality of complex hip joint images is usually poor, with more noise and artifacts affecting the recognition of normal anatomical structures. Severe fractures and joint deformities pose great challenges to morphological-based segmentation methods. Injuries to soft tissues such as the iliopsoas muscle and ligaments directly affect the formulation of surgical plans, but they are poorly depicted in medical images and difficult to segment. Metal implants can also produce serious metal artifacts and binding shadows, making segmentation even more difficult.

[0004] At present, a semi-automatic segmentation algorithm based on traditional image processing is usually used. Among them, the semi-automatic segmentation algorithm can specifically include morphological methods that rely on morphological features such as boundaries and regions of the target structure for segmentation, graph cut algorithms, region growing algorithms, and random walk algorithms. Such methods have low segmentation accuracy for the above-mentioned pathological hip joint structures with damage or deformity, or restorative hip joint structures with metal prostheses. In summary, the current hip joint segmentation algorithm is not suitable for complex hip joint scenes with pathological hip joints or restorative hip joints.

[0005] With regard to the problem of difficulty in hip joint segmentation in complex hip joint scenarios in related technologies, no effective solution has been proposed so far. Summary of the invention

[0006] In this embodiment, a hip joint image segmentation method, device and surgical robot system are provided to solve the problem of difficulty in hip joint segmentation in complex hip joint scenes in related technologies.

[0007] In a first aspect, a hip joint image segmentation method is provided in this embodiment, comprising:

[0008] Obtain medical images of the hip joint;

[0009] According to the recognition result of whether there is a metal prosthesis in the medical image, determining a segmentation model matching the recognition result;

[0010] Segmenting the medical image based on the segmentation model to obtain a segmented hip joint area;

[0011] Based on the segmented hip joint region, a segmented image of the hip joint is identified.

[0012] In some embodiments, determining a segmentation model matching the recognition result based on the recognition result of whether a metal prosthesis exists in the medical image includes:

[0013] If the recognition result indicates that a metal prosthesis exists in the medical image, determining to use a revision segmentation model;

[0014] If the recognition result indicates that there is no metal prosthesis in the medical image, it is determined to adopt a pathological segmentation model.

[0015] In some of the embodiments, the pathological segmentation model is trained based on a data set including normal hip joints and pathological hip joints; the revision segmentation model is trained based on a data set including normal hip joints, pathological hip joints and hip joints with metal prostheses.

[0016] In some of the embodiments, before determining a segmentation model matching the recognition result of whether there is a metal prosthesis in the medical image, the method further includes:

[0017] Based on a preset density threshold, performing threshold segmentation on the medical image, and extracting all pixel regions in the medical image whose density is greater than the density threshold after the threshold segmentation;

[0018] When the pixel area is larger than a preset area threshold, the pixel area is identified as a metal prosthesis area, and it is determined that the metal prosthesis area exists in the medical image; otherwise, it is determined that the metal prosthesis area does not exist in the medical image.

[0019] In some embodiments, the method further comprises:

[0020] Dividing the data set of the hip joint of the metal prosthesis into a de-artifacted shadow set and an original subset;

[0021] The pre-labeled normal hip joint data set, the pathological hip joint data set, the artifact-removed shadow set and the original subset are input into a preset segmentation model for training until the segmentation model converges to obtain the revision segmentation model.

[0022] In some embodiments, identifying the segmented image of the hip joint based on the segmented hip joint region includes:

[0023] Acquire the mask to be corrected corresponding to the hip joint area;

[0024] In response to a correction operation on the mask to be corrected, based on the correction operation and the mask to be corrected, the hip joint region is corrected in combination with a preset segmentation algorithm to identify and obtain a segmented image of the hip joint.

[0025] In some embodiments, the method further comprises:

[0026] Based on a preset mapping rule, grayscale value mapping is performed on the image area corresponding to the mask to be corrected in the medical image to obtain a mapping result;

[0027] A correction operation is performed on the mask to be corrected based on the mapping result.

[0028] In some embodiments, in response to the correction operation on the mask to be corrected, based on the correction operation and the mask to be corrected, in combination with a preset segmentation algorithm, the hip joint region is corrected to identify and obtain a segmented image of the hip joint, including:

[0029] In response to mis-segmentation coordinates selected by a user from the mask to be corrected, obtaining a mis-segmentation region in the medical image based on the mis-segmentation coordinates;

[0030] Based on the mis-segmented area and the mask to be corrected, the hip joint area is corrected in combination with a preset segmentation algorithm to identify and obtain a segmented image of the hip joint.

[0031] In a second aspect, a hip joint image segmentation device is provided in this embodiment, comprising: an acquisition module, a matching module, a segmentation module and a recognition module; wherein:

[0032] The acquisition module is used to acquire a medical image of the hip joint;

[0033] The matching module is used to determine a segmentation model matching the recognition result according to the recognition result of whether there is a metal prosthesis in the medical image;

[0034] The segmentation module is used to segment the medical image based on the segmentation model to obtain a segmented hip joint area;

[0035] The recognition module is used to recognize the segmented image of the hip joint based on the segmented hip joint area.

[0036] According to a third aspect, a surgical robot system is provided in this embodiment, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the hip joint image segmentation method described in the first aspect.

[0037] Compared with the related art, the hip joint image segmentation method, device and surgical robot system provided in this embodiment first obtain a medical image of the hip joint; then determine a segmentation model matching the recognition result based on the recognition result of whether there is a metal prosthesis in the medical image; then segment the medical image based on the segmentation model to obtain a segmented hip joint area; finally, based on the segmented hip joint area, identify the segmented image of the hip joint. It can select the most suitable segmentation model for hip joint segmentation, so that it can be applied to different hip joint lesion scenarios, improve the segmentation accuracy in complex hip joint scenarios, and thus reduce the difficulty of hip joint segmentation in complex hip joint scenarios.

[0038] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 is a hardware structure block diagram of a terminal of the hip joint image segmentation method of this embodiment;

[0041] Figure 2 is a flow chart of the hip joint image segmentation method of this embodiment;

[0042] Figure 3 is a flow chart of the hip joint image segmentation method of the preferred embodiment;

[0043] Figure 4 is a flow chart of a data enhancement method for removing artifacts in the training process of the above-mentioned renovation segmentation model according to the preferred embodiment;

[0044] Figure 5 is a flow chart of the interactive segmentation method of hip joint images of the preferred embodiment;

[0045] Figure 6 It is a structural block diagram of the hip joint image segmentation device of this embodiment. DETAILED DESCRIPTION

[0046] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0047] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0048] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 4 is a hardware structure diagram of a terminal of the hip joint image segmentation method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0049] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the segmentation method of the hip joint image in the present embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, realizing the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0050] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0051] In this embodiment, a hip joint image segmentation method is provided. Figure 2 is a flow chart of the hip joint image segmentation method of this embodiment, such as Figure 2 As shown, the process includes the following steps:

[0052] Step S210, acquiring a medical image of the hip joint.

[0053] The medical image of the hip joint can specifically be an image obtained by scanning the hip joint part of the object under test using Computed Tomography (CT) technology or X-ray scanning technology. The medical image includes the density value HU corresponding to each pixel point, and there is a hip joint area in the medical image. The medical image may also contain other background areas. Therefore, the medical image is an image that requires hip joint area segmentation. The HU is the unit of heat capacity of the tube in the CT scan. The hip joint object in the medical image belongs to one of the three situations: a normal hip joint, a pathological hip joint, and a repaired hip joint with a metal prosthesis. A repaired hip joint with a metal prosthesis refers to a hip joint implanted with a metal prosthesis.

[0054] Step S220, according to the recognition result of whether there is a metal prosthesis in the medical image, determine a segmentation model that matches the recognition result.

[0055] Among them, after obtaining the medical image, the metal prosthesis area can be identified according to the metal prosthesis identification method, so as to select different segmentation models to segment the hip joint area to improve the accuracy of hip joint area segmentation. For example, the metal prosthesis identification can be realized by using a density threshold segmentation method, a gray value judgment method, and a deep learning method. Preferably, for the restorative hip joint of the metal prosthesis, the metal material contained therein will present a higher density value in the CT scan result, and the density value will be significantly different from the density value presented by the human tissue in the hip joint in the CT result. Therefore, after obtaining the medical image, the present embodiment first performs identification processing on the metal prosthesis area of ​​the medical image to determine whether the scanned hip joint is implanted with a metal prosthesis, and then selects different segmentation models for processing according to the identification result. Among them, the segmentation model matching the identification result refers to different segmentation models that are pre-trained for the presence or absence of metal prostheses in the medical image. For example, the medical image with a metal prosthesis is suitable for the first type of segmentation model, and the medical image without a metal prosthesis is suitable for the second type of segmentation model.

[0056] Furthermore, for medical images with metal prosthesis areas, a well-trained revision segmentation model can be used to identify the hip joint. The revision segmentation model is pre-trained based on a data set containing a metal prosthesis, a normal hip joint, and a pathological hip joint. For medical images without metal prosthesis areas, a well-trained pathological segmentation model can be used to identify the hip joint. The pathological segmentation model is pre-trained using a data set containing only normal hip joints and pathological hip joints. In addition to the difference in training data, the above-mentioned pathological segmentation model and the revision segmentation model can use the same or different machine learning model structures. Preferably, both the above-mentioned pathological segmentation model and the revision segmentation model can use a dual attention segmentation network model 3D-Unet model including two stages of coarse segmentation and fine segmentation. In addition, other segmentation network models such as fully convolutional neural networks for semantic segmentation (FCN for short), deep convolutional encoder SegNet, and multi-path reinforcement network RefineNet can also be selected based on the application scenario.

[0057] Step S230, segmenting the medical image based on the segmentation model to obtain a segmented hip joint area.

[0058] Step S240: identifying a segmented image of the hip joint based on the segmented hip joint region.

[0059] Through the above process, the hip joint area of ​​the medical image is identified using the segmentation model that matches the recognition result, and the hip joint area is obtained, thereby realizing automatic recognition of the hip joint area in the medical image. On this basis, the above hip joint area can be further corrected by manual adjustment or other correction methods to identify the segmented image of the hip joint.

[0060] Compared with the related art, the image segmentation of the hip joint is difficult due to the existence of damaged or deformed structures in the pathological hip joint, and the metal artifacts and restraint shadows produced by metal implants also cause segmentation difficulties. In this embodiment, corresponding segmentation models are established for different types of hip joints, which can achieve the segmentation of complex pathological hip joints accompanied by arthritis, fractures, necrosis, etc. and restorative hip joints implanted with metal prostheses. It can adapt to different hip joint disease scenarios, thereby improving the overall segmentation accuracy in complex hip joint scenarios, reducing the segmentation difficulty, and improving the segmentation robustness. It can also adaptively handle the image segmentation of complex pathological hip joints and restorative hip joints implanted with metal prostheses.

[0061] In the above steps S210 to S240, a medical image of the hip joint is obtained; according to the recognition result of whether there is a metal prosthesis in the medical image, a segmentation model matching the recognition result is determined; based on the segmentation model, the medical image is segmented to obtain a segmented hip joint region; based on the segmented hip joint region, a segmented image of the hip joint is identified. It can select the most suitable segmentation model for the segmentation of the hip joint, so that it can be applied to different hip joint disease scenarios, improve the segmentation accuracy in complex hip joint scenarios, and thus reduce the difficulty of hip joint segmentation in complex hip joint scenarios.

[0062] Further, in one embodiment, based on the above step S220, according to the recognition result of whether there is a metal prosthesis in the medical image, determining a segmentation model matching the recognition result may specifically include:

[0063] If the recognition result indicates that a metal prosthesis exists in the medical image, the revision segmentation model is determined to be used; if the recognition result indicates that no metal prosthesis exists in the medical image, the pathology segmentation model is determined to be used. In this embodiment, different segmentation models are selected for the medical image based on the recognition result to segment the hip joint area, which can improve the accuracy of hip joint area segmentation in different scenarios.

[0064] Furthermore, in one embodiment, the pathological segmentation model is trained based on a data set containing normal hip joints and pathological hip joints; and the revision segmentation model is trained based on a data set containing normal hip joints, pathological hip joints, and hip joints with metal prostheses. Based on this, a fully trained pathological segmentation model can be applicable to image segmentation of pathological hip joints and normal hip joints without metal prostheses, while a fully trained revision segmentation model can be applicable to segmentation of hip joints with metal prostheses and normal hip joints. Therefore, based on different training methods, this embodiment can train different targeted segmentation models to improve the accuracy and adaptability of hip joint area segmentation in different scenarios.

[0065] Furthermore, in one embodiment, based on the above step S220, before determining the segmentation model matching the recognition result based on the recognition result of whether a metal prosthesis exists in the medical image, the above hip joint image segmentation method may also include: performing threshold segmentation on the medical image based on a preset density threshold, and extracting all pixel areas in the medical image whose density is greater than the density threshold after threshold segmentation; when the pixel area is greater than a preset area threshold, identifying the pixel area as a metal prosthesis area, and determining that a metal prosthesis area exists in the medical image; otherwise, determining that no metal prosthesis area exists in the medical image.

[0066] Specifically, the density of metal materials is higher than that of human tissue. Therefore, metal materials will be displayed as small-area and high-density dot-like or linear structures on medical images. In addition, generally speaking, in medical images, the density value of the pixel points corresponding to the metal prosthesis is about 2000HU. In this regard, this embodiment adopts a threshold segmentation method to identify whether there is a metal prosthesis area in the medical image. Exemplarily, the medical image is first read to obtain image information including the image array and the density value corresponding to each pixel point. After that, a density threshold for identifying the metal prosthesis is set, for example, it can be set to 2000HU. Next, the medical image is threshold segmented according to the set density threshold, and all pixels with a density greater than the density threshold are extracted to form a preliminary threshold segmentation result. The area of ​​the pixel area composed of the extracted pixels is counted. When the area of ​​the pixel area is greater than a certain area threshold, it can be judged that the medical image contains a metal prosthesis area. This embodiment performs threshold segmentation on the medical image based on the set density threshold, thereby realizing the identification of the metal prosthesis area, and can accurately complete the identification of the metal prosthesis area before performing the image segmentation of the hip joint, thereby providing a reference for the subsequent selection of the corresponding segmentation model.

[0067] Additionally, in one embodiment, the above hip joint image segmentation method may further include:

[0068] The dataset of metal prosthetic hip joints is divided into a de-artifacted set and an original subset; the pre-labeled normal hip joint dataset, the pathological hip joint dataset, the de-artifacted set and the original subset are input into a preset segmentation model for training until the segmentation model converges to obtain a revision segmentation model.

[0069] This embodiment uses a data enhancement method based on artifact removal to train the segmentation model to obtain a revision segmentation model suitable for image recognition of restorative hip joints containing metal prostheses. Specifically, a preset artifact removal algorithm, such as a deep learning method based on a convolutional neural network, or a traditional image processing algorithm based on filtering, image transformation, etc., is used to perform artifact removal operations on a partial subset of the restorative hip joint data set of metal prostheses, thereby obtaining an artifact removal shadow set and an original subset. Afterwards, the data set of normal hip joints, the data set of pathological hip joints, the artifact removal shadow set, and the original subset are labeled respectively, and then input into the preset segmentation model for iterative training until the segmentation model is confirmed to converge, and a fully trained revision segmentation model is obtained.

[0070] This embodiment targets the image segmentation scenario of a prosthetic hip joint with a metal prosthesis, and obtains training data by combining an artifact removal algorithm, thereby obtaining an image segmentation model suitable for a prosthetic hip joint with an implanted metal prosthesis, which can restore the hip joint area obscured by metal artifacts to the greatest extent, and reduce the difficulty of image segmentation for complex hip joints. In addition, this embodiment uses a data enhancement algorithm based on artifact removal for training the segmentation model, which can also enhance the segmentation capability of the model in various complex hip joint image segmentation scenarios and improve segmentation robustness.

[0071] Furthermore, in each iteration of training, a data subset is randomly divided from the data set of the metal prosthetic hip joint for artifact removal, and an artifact-removed shadow set and an original subset are obtained. That is, in each iteration of training, a subset that needs to be artifact-removed is randomly divided from the data set of the metal prosthetic hip joint used for training to perform the artifact removal operation, so that in each iteration, the data set of the metal prosthetic hip joint is randomly divided into an artifact-removed shadow set and an original subset. For example, in each iteration of training, the data set U of the metal prosthetic hip joint is randomly divided into two subsets A and B. The relationship between subset A and subset B satisfies this condition: A∪B=U, and Is an empty set. Use the preset metal artifact removal algorithm to remove artifacts from subset B to obtain subset B'. Afterwards, the data set of normal hip joints, the data set of pathological hip joints, subset A and subset B' are sent to the preset segmentation model for training. In this embodiment, since the subsets used for artifact removal are randomly selected in each iteration, different subsets are added to the network training through the artifact removal algorithm in each iteration, thereby enhancing the robustness of the model.

[0072] In one embodiment, based on the above step S240, based on the segmented hip joint region, identifying the segmented image of the hip joint may specifically include:

[0073] The mask to be corrected corresponding to the hip joint area is obtained; in response to the correction operation on the mask to be corrected, the hip joint area is corrected based on the correction operation and the mask to be corrected in combination with a preset segmentation algorithm, and a segmented image of the hip joint is identified.

[0074] In order to improve the accuracy of the final segmentation of the hip joint area, this embodiment outputs the mask to be corrected corresponding to the hip joint area to the user after segmenting the hip joint area, so that the user can perform correction operations on the mask to be corrected, adjust the segmentation error area, and then obtain the corrected hip joint area. Exemplarily, the mask to be corrected can be a matrix of the same size as the medical image in mathematical form, in which the elements include the numbers 1 and 0, the element corresponding to the hip joint area is 1, and the other elements are 0. The mask to be corrected can also be expressed in other forms, which is not limited in this embodiment. In addition, the user's correction operation on the mask to be corrected can be specifically realized by editing the erroneously segmented area to correct the mask to be corrected.

[0075] The user can edit the corresponding area of ​​the mask to be corrected at any level in the coronal plane, sagittal plane and cross-section of the medical image. The editing operation performed by the user on a certain level is expanded to other levels of the preset number of layers in the Z-axis direction to obtain the overall regional contour of the mask to be corrected with the correction error. Afterwards, the hip joint area is segmented again in combination with the preset segmentation algorithm, such as graph cutting, region generation, random walk and other methods or a segmentation model based on deep learning, to identify the final target hip joint in the medical image. In view of the different pathological conditions of the hip joint, this embodiment further combines the interactive segmentation method that interacts with the user on the basis of automatic segmentation to correct the mask to be corrected corresponding to the hip joint area, which can be combined with the user's correction experience to further improve the accuracy of the segmentation result.

[0076] In one embodiment, the hip joint image segmentation method further includes:

[0077] Based on a preset mapping rule, grayscale value mapping is performed on the image area corresponding to the mask to be corrected in the medical image to obtain a mapping result; and a correction operation is performed on the mask to be corrected based on the mapping result.

[0078] Before outputting the mask to be corrected to the user, the image data of the hip joint area can be normalized, that is, linearly mapped. Specifically, first count the HU value of each pixel corresponding to the hip joint area in the medical image and sort them. In order to better show the bone area affected by metal artifacts, only the part whose HU value belongs to the preset range can be retained, such as the part whose HU value is greater than 0 and less than 2000, and the retained part is used to calculate the window width W. Specifically, based on the sorting result, the maximum HU value is obtained, expressed as MAX_HU, and the minimum HU value is expressed as MIN_HU. Then the window width is obtained according to the method of window width W = MAX_Hu-MIN_Hu. The obtained window width is then used to perform linear mapping on the image data of the hip joint area. The specific mapping method can refer to the following formula:

[0079] N_HU = [(O_HU - MIN_HU) / W]*255 (1)

[0080] Among them, O_HU is the original HU value of the pixel, N_HU is the HU value of the pixel after linear mapping, and W is the above-mentioned window width. Based on this, the value of each pixel in the new image is in the interval [0, 255]. After completing the above mapping, the mapping result is presented to the user through the output end, so that the user can refer to the output result and correct the mask to be corrected. This embodiment can provide a segmentation reference for the user's subsequent interactive segmentation and improve the accuracy of the final segmentation result by linearly mapping the area corresponding to the hip joint in the medical image and outputting the mapping result to the user.

[0081] In addition, in one embodiment, in response to the correction operation on the mask to be corrected, based on the correction operation and the mask to be corrected, the hip joint region is corrected in combination with a preset segmentation algorithm to identify and obtain a segmented image of the hip joint, which may specifically include:

[0082] In response to the mis-segmentation coordinates selected by the user from the mask to be corrected, the mis-segmentation area in the medical image is obtained based on the mis-segmentation coordinates; based on the mis-segmentation area and the mask to be corrected, the hip joint area is corrected in combination with a preset segmentation algorithm to identify and obtain a segmented image of the hip joint.

[0083] Specifically, the user edits the segmentation error area of ​​any layer in the coronal plane, sagittal plane or cross section in the mask to be corrected. Taking the cross section as an example, the user edits the position coordinates (x1, y1, z) of the lower left corner and the position coordinates (x2, y2, z) of the upper right corner of the cross section, and expands the layer up and down in the z direction to obtain the entire segmentation error contour (x1, y1, z1) to (x2, y2, z2) of the mask to be corrected. The above-mentioned segmentation error contour is combined with the medical image to obtain the image block of the mis-segmented area in the medical image. Combined with the image block of the mis-segmented area and the mask to be corrected, the hip joint area is segmented using a preset segmentation algorithm to obtain a correction result. The correction result is updated to the mask to be corrected to obtain a corrected mask. The corrected mask is output to the user. If the user evaluates that the corrected mask corresponds to the accurate hip joint area, the correction process is terminated to obtain the target hip joint. If the user evaluates that the hip joint area corresponding to the corrected mask is inaccurate at this time, the corrected mask is identified as the mask to be corrected, and the above correction process is repeated again until the accurate hip joint area is obtained. This embodiment adopts a combination of automatic segmentation and interactive segmentation, making full use of the automatic segmentation capability of the well-trained segmentation model and the user's segmentation experience, so that the two complement each other and are suitable for image segmentation of hip joints under different pathological conditions. It makes up for the problem that the segmentation model is difficult to achieve accurate segmentation in all complex pathological conditions, and can obtain more accurate segmentation results.

[0084] In addition, when it is determined that there is a metal prosthesis area in the medical image, the medical image is de-artifacted to obtain a de-artifacted image; the de-artifacted image and the medical image are displayed to the user for comparison. The de-artifacted image and the medical image are displayed to the user for image comparison, which can facilitate the user to observe the pathological areas such as bone and cortex affected by metal artifacts in the medical image, and restore the damaged bone tissue area blocked by metal artifacts to the greatest extent, thereby improving the user's judgment in performing correction operations during subsequent interactive segmentation, and assisting the user to confirm the real pathological area.

[0085] The present embodiment is described and illustrated below through preferred embodiments.

[0086] Figure 3 Flow chart of the hip joint image segmentation method of the preferred embodiment. Figure 3 As shown, the hip joint image segmentation method includes the following steps:

[0087] Step S301, obtaining an original medical image of the hip joint;

[0088] Step S302, performing metal prosthesis recognition on the medical image to obtain a recognition result;

[0089] Step S303, judging whether there is a metal prosthesis area in the medical image based on the recognition result; if so, executing step S304; otherwise, executing step S305;

[0090] Step S304, segmenting the medical image using a well-trained revision segmentation model to obtain a hip joint region; wherein the revision segmentation model can be specifically implemented based on a 3D-Unet model, which can be trained based on a data set containing normal hip joints and pathological hip joints;

[0091] Step S305, segmenting the medical image using a well-trained pathological segmentation model to obtain a hip joint region; wherein the pathological segmentation model may be implemented based on a 3D-Unet model, which may be trained based on a data set including a normal hip joint, a pathological hip joint, and a hip joint with a metal prosthesis;

[0092] Step S306, outputting the mask to be corrected corresponding to the hip joint area to the user;

[0093] Step S307: Based on the user's correction operation on the mask to be corrected, the hip joint area is corrected in combination with a preset segmentation algorithm to identify and obtain a segmented image of the hip joint.

[0094] Figure 4 : is a flowchart of a data enhancement method for removing artifacts in the training process of the above-mentioned renovation segmentation model according to the preferred embodiment of the present invention, such as Figure 4 As shown, the data enhancement method for removing artifacts includes the following steps:

[0095] Step S401, in the current training iteration process, randomly divide the data set of the hip joint with metal prosthesis into subset A and subset B;

[0096] Step S402, performing artifact removal processing on the subset B using a preset metal artifact removal algorithm to obtain an artifact removal shadow set B';

[0097] Step S403, marking the subset A as the original subset, and inputting the marked artifact-removed shadow set B', the data set of normal hip joints, and the data set of pathological hip joints into a preset segmentation model for training;

[0098] Step S404, determining whether the segmentation model has converged, if so, stopping the training to obtain a fully trained renovation segmentation model; otherwise, returning to execute the above step S401.

[0099] Figure 5 Flow chart of the interactive segmentation method of hip joint image of the preferred embodiment, as shown in FIG. Figure 5 As shown, the interactive segmentation method of the hip joint image includes the following steps:

[0100] Step S501, normalizing the image data in the hip joint area corresponding to the mask to be corrected to obtain normalized data;

[0101] Step S502, outputting normalized data to the user;

[0102] Step S503, if there is a metal prosthesis area in the medical image, then perform a de-artifacting operation on the medical image, and output the medical image and the de-artifacting image to the user for comparison;

[0103] Step S504, obtaining the correction operation of the mask to be corrected by the user with reference to the output results of step S502 and step S503, and obtaining a corrected mask;

[0104] Step S505, obtaining the evaluation result of the user on the modified mask, and determining whether the hip joint area expected by the user is obtained; if so, executing step S506; otherwise, returning to executing step S501;

[0105] Step S506: output the target hip joint.

[0106] In this embodiment, a hip joint image segmentation device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware for predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0107] Figure 6 60 is a structural block diagram of the hip joint image segmentation device of this embodiment. Figure 6 As shown, the hip joint image segmentation device 60 includes: an acquisition module 62, a matching module 64, a segmentation module 66 and a recognition module 68; wherein: the acquisition module 62 is used to acquire a medical image of the hip joint; the matching module 64 is used to determine a segmentation model that matches the recognition result based on the recognition result of whether there is a metal prosthesis in the medical image; the segmentation module 66 is used to segment the medical image based on the segmentation model to obtain a segmented hip joint area; the recognition module 68 is used to identify a segmented image of the hip joint based on the segmented hip joint area.

[0108] The above-mentioned hip joint image segmentation device 60 can select the most suitable segmentation model for hip joint segmentation, so that it can be applicable to different hip joint disease scenarios, improve the segmentation accuracy in complex hip joint scenarios, and then reduce the difficulty of hip joint segmentation in complex hip joint scenarios.

[0109] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0110] In addition, in combination with the hip joint image segmentation method provided in the above embodiment, a surgical robot system can also be provided in this embodiment to implement the method, and the surgical robot system includes a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to execute any one of the hip joint image segmentation methods in the above embodiment.

[0111] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0112] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0113] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0114] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0115] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A hip joint image segmentation method, characterized in that: include: Obtain medical images of the hip joint; According to the recognition result of whether there is a metal prosthesis in the medical image, determining a segmentation model matching the recognition result; Segmenting the medical image based on the segmentation model to obtain a segmented hip joint area; Based on the segmented hip joint region, a segmented image of the hip joint is identified.

2. The hip joint image segmentation method according to claim 1, characterized in that: The step of determining a segmentation model matching the recognition result based on the recognition result of whether a metal prosthesis exists in the medical image comprises: If the recognition result indicates that a metal prosthesis exists in the medical image, determining to use a revision segmentation model; If the recognition result indicates that there is no metal prosthesis in the medical image, it is determined to adopt a pathological segmentation model.

3. The method for segmenting a hip joint image according to claim 2, characterized in that: The pathological segmentation model is trained based on a data set containing normal hip joints and pathological hip joints; the revision segmentation model is trained based on a data set containing normal hip joints, pathological hip joints and hip joints with metal prostheses.

4. The hip joint image segmentation method according to claim 1, characterized in that: Before determining a segmentation model matching the recognition result of whether a metal prosthesis exists in the medical image, the method further includes: Based on a preset density threshold, performing threshold segmentation on the medical image, and extracting all pixel regions in the medical image whose density is greater than the density threshold after the threshold segmentation; When the pixel area is larger than a preset area threshold, the pixel area is identified as a metal prosthesis area, and it is determined that a metal prosthesis exists in the medical image; otherwise, it is determined that the metal prosthesis does not exist in the medical image.

5. The hip joint image segmentation method according to claim 3, characterized in that: The method further comprises: Dividing the data set of the hip joint of the metal prosthesis into a de-artifacted shadow set and an original subset; The pre-labeled normal hip joint data set, the pathological hip joint data set, the artifact-removed shadow set and the original subset are input into a preset segmentation model for training until the segmentation model converges to obtain the revision segmentation model.

6. The method for segmenting a hip joint image according to any one of claims 1 to 5, characterized in that: The step of identifying a segmented image of the hip joint based on the segmented hip joint region comprises: Acquire the mask to be corrected corresponding to the hip joint area; In response to a correction operation on the mask to be corrected, based on the correction operation and the mask to be corrected, the hip joint region is corrected in combination with a preset segmentation algorithm to identify and obtain a segmented image of the hip joint.

7. The hip joint image segmentation method according to claim 6, characterized in that: The method further comprises: Based on a preset mapping rule, grayscale value mapping is performed on the image area corresponding to the mask to be corrected in the medical image to obtain a mapping result; A correction operation is performed on the mask to be corrected based on the mapping result.

8. The hip joint image segmentation method according to claim 6, characterized in that: In response to the correction operation on the mask to be corrected, based on the correction operation and the mask to be corrected, the hip joint region is corrected in combination with a preset segmentation algorithm to identify and obtain a segmented image of the hip joint, including: In response to mis-segmentation coordinates selected by a user from the mask to be corrected, obtaining a mis-segmentation region in the medical image based on the mis-segmentation coordinates; Based on the mis-segmented area and the mask to be corrected, the hip joint area is corrected in combination with a preset segmentation algorithm to identify and obtain a segmented image of the hip joint.

9. A hip joint image segmentation device, characterized in that: include: Acquisition module, matching module, segmentation module and recognition module; wherein: The acquisition module is used to acquire a medical image of the hip joint; The matching module is used to determine a segmentation model matching the recognition result according to the recognition result of whether there is a metal prosthesis in the medical image; The segmentation module is used to segment the medical image based on the segmentation model to obtain a segmented hip joint area; The recognition module is used to recognize the segmented image of the hip joint based on the segmented hip joint area.

10. A surgical robot system, comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the hip joint image segmentation method according to any one of claims 1 to 8.

Images