Hip joint data processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a hip joint data processing method and device, a storage medium and electronic equipment. Relates to the technical field of artificial intelligence. Performing metal artifact removal processing on the initial CT image through a target network structure to obtain a target CT image and a target confidence coefficient matrix corresponding to the target CT image; performing three-dimensional reconstruction according to the target CT image and the target confidence coefficient matrix to obtain a target hip joint model, and determining a registration point set according to the target hip joint model; according to the registration point set, a target affine matrix is determined, and the target affine matrix is used for providing data reference for hip joint surgery. According to the method and the device, the problem that the accuracy of determining the affine matrix is relatively low due to the fact that the affine matrix is determined by performing three-dimensional reconstruction on the CT image with the metal artifact in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a hip joint data processing method and device, a storage medium and an electronic device. Background Art

[0002] In the treatment of osteoarthritis, secondary hip replacement surgery (also known as hip revision surgery) has been proven to be an effective treatment for symptoms such as acetabular enlargement, prosthesis loosening, and prosthesis wear after primary hip replacement.

[0003] However, good registration is essential for hip revision surgery with the help of modern navigation robots. During the registration process of hip revision surgery, the patient's implanted prosthesis causes metal artifacts in the CT, which has always been a difficult problem in CT diagnosis. Since the patient's implanted prosthesis is difficult to remove before surgery, and the revision surgery assisted by modern navigation robots relies on CT to reconstruct the patient's bone prosthesis, and plans, registers and guides the normal operation based on the reconstructed three-dimensional model, therefore, in the existing technology, the three-dimensional reconstruction of CT images with metal artifacts and then the determination of the affine matrix has the problem of low accuracy.

[0004] Currently, no effective solution has been proposed to solve the problem that the accuracy of determining the affine matrix is ​​relatively low when three-dimensional reconstruction is performed on CT images with metal artifacts in related technologies. Summary of the invention

[0005] The main purpose of the present application is to provide a hip joint data processing method and device, a storage medium and an electronic device to solve the problem in the related art that the accuracy of determining the affine matrix is ​​relatively low due to three-dimensional reconstruction of CT images with metal artifacts and then determining the affine matrix.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a hip joint data processing method is provided. The method comprises: obtaining an initial CT image of a hip joint; performing metal artifact removal processing on the initial CT image through a target network structure to obtain a target CT image and a target confidence matrix corresponding to the target CT image, wherein the target confidence matrix is ​​composed of the confidence corresponding to each voxel point in the target CT image; performing three-dimensional reconstruction based on the target CT image and the target confidence matrix to obtain a target hip joint model, and determining a set of registration points based on the target hip joint model; determining a target affine matrix based on the set of registration points, wherein the target affine matrix is ​​used to provide data reference for hip joint surgery.

[0007] Furthermore, the initial CT image is processed to remove metal artifacts through the target network structure to obtain a target CT image and a target confidence matrix corresponding to the target CT image, including: extracting features of the initial CT image and the sinusoidal graph corresponding to the initial CT image through the sinusoidal graph enhancement network in the target network structure to obtain first feature information; and processing the first feature information through the image enhancement network in the target network structure to obtain the target CT image and the target confidence matrix.

[0008] Further, the first feature information is processed by the image enhancement network in the target network structure to obtain the target CT image and the target confidence matrix, including: downsampling the first feature information by multiple downsampling layers of the image enhancement network to obtain the second feature information corresponding to each downsampling layer; upsampling the second feature information corresponding to each downsampling layer by multiple first upsampling layers of the image enhancement network to obtain the target CT image; processing the second feature information corresponding to each downsampling layer and the output information of the target first upsampling layer by multiple second upsampling layers of the image enhancement network to obtain the target confidence matrix, wherein the target first upsampling layer is a sampling layer among the multiple first upsampling layers.

[0009] Furthermore, the target network structure is trained by the following steps: obtaining a training sample set, wherein the training sample set includes: a sample CT image, a real CT image corresponding to the sample CT image, and a real confidence matrix corresponding to the real CT image; processing the training sample set through the initial network structure to obtain a predicted CT image and a predicted confidence matrix corresponding to the predicted CT image; iterating the initial network structure according to the predicted CT image and the predicted confidence matrix to obtain the target network structure.

[0010] Furthermore, the initial network structure is iterated according to the predicted CT image and the predicted confidence matrix to obtain the target network structure, including: performing loss calculation according to the predicted CT image and the real CT image to obtain a first loss function; performing loss calculation according to the predicted confidence matrix and the real confidence matrix to obtain a second loss function; performing calculation according to the first loss function, the second loss function and the target weight to obtain a target loss function, wherein the target weight is calculated based on the first loss function and the second loss function; and iterating the initial network structure according to the target loss function to obtain the target network structure.

[0011] Furthermore, obtaining the training sample set includes: obtaining the real CT image, and processing the real CT image to obtain the sample CT image; calculating the difference between each voxel point in the real CT image and each voxel point in the sample CT image to obtain a single voxel difference set; obtaining the real confidence matrix based on the single voxel difference set; obtaining the training sample set based on the sample CT image, the real CT image and the real confidence matrix.

[0012] Furthermore, performing three-dimensional reconstruction based on the target CT image and the target confidence matrix to obtain a target hip joint model includes: performing three-dimensional reconstruction based on the target CT image to obtain an initial hip joint model; and performing confidence mapping on the initial hip joint model based on the target confidence matrix to obtain the target hip joint model.

[0013] Furthermore, according to the target hip joint model, determining the registration point set includes: screening the acetabulum side of the target hip joint model to obtain the acetabulum side candidate point set; screening the femoral side of the target hip joint model to obtain the femoral side candidate point set; screening based on the acetabulum side candidate point set and the femoral side candidate point set to obtain the registration point set.

[0014] Furthermore, the acetabulum side of the target hip joint model is screened to obtain a set of candidate points on the acetabulum side, including: determining the direction vector corresponding to the center point of the acetabular cup and the implantation direction of the acetabular cup based on the target hip joint model; determining the normal vector corresponding to the first screening point within the target range centered at the center point of the acetabular cup, and calculating the angle value between the normal vector and the direction vector to obtain the target angle value; screening the first screening point within the target range based on the target angle value and the confidence level corresponding to the first screening point to obtain the set of candidate points on the acetabulum side.

[0015] Furthermore, the femoral side of the target hip joint model is screened to obtain a femoral side candidate point set, including: determining a target screening area based on the innermost point of the femoral stem, the outermost point of the femoral side, the apex of the greater trochanter and the bottom point of the lesser trochanter in the target hip joint model; judging whether a second screening point in the target screening area belongs to the femoral area to obtain a first judgment result; judging whether the confidence corresponding to the second screening point is greater than or equal to a preset threshold to obtain a second judgment result; screening the second screening point based on the first judgment result and the second judgment result to obtain the femoral side candidate point set.

[0016] Further, screening is performed based on the acetabulum side candidate point set and the femoral side candidate point set to obtain the registration point set, including: determining a coarse registration point set corresponding to the coarse registration from the candidate points in the candidate point set based on the selection indication of the first object, wherein the candidate point set is the acetabulum side candidate point set, or the femoral side candidate point set; clustering the candidate points in the candidate point set to obtain a fine registration point set corresponding to the fine registration; and obtaining the registration point set based on the coarse registration point set and the fine registration point set.

[0017] Furthermore, clustering is performed on the candidate points in the candidate point set to obtain a set of precise alignment points corresponding to the precise alignment, including: clustering is performed on the candidate points in the candidate point set to obtain multiple candidate point subsets; the spatial volume of each candidate point subset is calculated to obtain multiple spatial volume values; the sampling ratio of each candidate point subset is calculated based on the multiple spatial volume values ​​to obtain multiple ratio values; and the multiple candidate point subsets are randomly sampled based on the multiple ratio values ​​and the required number of precise alignment points to obtain the set of precise alignment points.

[0018] Further, determining the target affine matrix according to the registration point set includes: coarsely registering a coarse registration point set in the registration point set with a first acquisition point set corresponding to the coarse registration point set to obtain a coarse registration affine matrix, wherein the acquisition points in the first acquisition point set are obtained from a second object; based on the coarse registration affine matrix, registering a fine registration point set in the registration point set with a second acquisition point set corresponding to the fine registration point set to obtain a fine registration affine matrix, wherein the acquisition points in the second acquisition point set are obtained from a second object; and obtaining the target affine matrix based on the fine registration affine matrix and the coarse registration affine matrix.

[0019] Further, performing rough registration on a coarse registration point set in the registration point set and a first acquisition point set corresponding to the coarse registration point set to obtain a coarse registration affine matrix includes: sorting the coarse registration points in the coarse registration point set to obtain sorted coarse registration points, and sorting the first acquisition points in the first acquisition point set to obtain sorted first acquisition points; determining a first vector and a second vector based on a line between the sorted coarse registration points, and determining a third vector and a fourth vector based on a line between the sorted first acquisition points; obtaining a target rotation matrix based on the first vector, the second vector, the third vector and the fourth vector; performing rotation processing on the first acquisition point based on the target rotation matrix to obtain a processed first acquisition point; obtaining a translation matrix based on a distance between the processed first acquisition point and the coarse registration point; and obtaining the coarse registration affine matrix based on the target rotation matrix and the translation matrix.

[0020] Furthermore, based on the coarse registration affine matrix, the fine registration point set in the registration point set and the second acquisition point set corresponding to the fine registration point set are registered to obtain the fine registration affine matrix, which includes: applying random perturbations to the coarse registration affine matrix based on a normal distribution to obtain multiple first affine matrices; optimizing the multiple first affine matrices through a data matching algorithm to obtain multiple second affine matrices; when similar affine matrices exist in the multiple second affine matrices, merging similar affine matrices in the multiple second affine matrices to obtain multiple third affine matrices; and obtaining the fine registration affine matrix based on the multiple third affine matrices and the fine registration point set.

[0021] Further, obtaining the fine registration affine matrix according to the multiple third affine matrices and the fine registration point set includes: mapping the second acquisition points in the second acquisition point set based on the coarse registration affine matrix to obtain a third acquisition point set; mapping the third acquisition points in the third acquisition point set based on each third affine matrix to obtain a fourth acquisition point set corresponding to each third affine matrix; calculating the root mean square error of the distance between the fourth acquisition point in the fourth acquisition point set and the registration point in the fine registration point set to obtain multiple root mean square error sets; if there is an error in the target root mean square error set in the multiple root mean square error sets that is less than an error threshold, determining the third affine matrix corresponding to the target root mean square error set as the fine registration affine matrix.

[0022] Furthermore, the method also includes: if there is no target root mean square error set in the multiple root mean square error sets whose error is less than an error threshold, then determining that one iteration is completed, and judging whether the number of iterations is greater than or equal to a preset number; if the number of iterations is less than the preset number, then repeating the step of applying random perturbations to the third affine matrix with the smallest error in the root mean square error set based on a normal distribution until the number of iterations is greater than or equal to the preset number, or the root mean square error set corresponding to the current affine matrix is ​​less than the error threshold.

[0023] Furthermore, after determining whether the number of iterations is greater than or equal to a preset number of times, the method further includes: determining whether it is necessary to filter the set of precise alignment points; if it is necessary to filter the set of precise alignment points, filtering the set of precise alignment points based on preset conditions to obtain a filtered set of precise alignment points, wherein the next iteration process is performed based on the filtered set of precise alignment points.

[0024] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a hip joint data processing device is provided. The device comprises: a first acquisition unit, used to acquire an initial CT image of a hip joint; a first processing unit, used to perform metal artifact removal processing on the initial CT image through a target network structure, and obtain a target CT image and a target confidence matrix corresponding to the target CT image, wherein the target confidence matrix is ​​composed of the confidence corresponding to each voxel point in the target CT image; a reconstruction unit, used to perform three-dimensional reconstruction based on the target CT image and the target confidence matrix, obtain a target hip joint model, and determine a set of registration points based on the target hip joint model; a first determination unit, used to determine a target affine matrix based on the set of registration points, wherein the target affine matrix is ​​used to provide data reference for hip joint surgery.

[0025] Furthermore, the first processing unit includes: an extraction subunit, used to extract features of the initial CT image and the sinusoidal graph corresponding to the initial CT image through the sinusoidal graph enhancement network in the target network structure to obtain first feature information; a processing subunit, used to process the first feature information through the image enhancement network in the target network structure to obtain the target CT image and the target confidence matrix.

[0026] Furthermore, the processing subunit includes: a first sampling module, used to downsample the first feature information through multiple downsampling layers of the image enhancement network to obtain second feature information corresponding to each downsampling layer; a second sampling module, used to upsample the second feature information corresponding to each downsampling layer through multiple first upsampling layers of the image enhancement network to obtain the target CT image; a processing module, used to process the second feature information corresponding to each downsampling layer and the output information of the target first upsampling layer through multiple second upsampling layers of the image enhancement network to obtain the target confidence matrix, wherein the target first upsampling layer is a sampling layer among the multiple first upsampling layers.

[0027] Furthermore, the target network structure is obtained by training using the following devices: a second acquisition unit, used to acquire a training sample set, wherein the training sample set includes: a sample CT image, a real CT image corresponding to the sample CT image, and a real confidence matrix corresponding to the real CT image; a second processing unit, used to process the training sample set through the initial network structure to obtain a predicted CT image and a predicted confidence matrix corresponding to the predicted CT image; an iteration unit, used to iterate the initial network structure according to the predicted CT image and the predicted confidence matrix to obtain the target network structure.

[0028] Furthermore, the iteration unit includes: a first calculation subunit, used to perform loss calculation based on the predicted CT image and the real CT image to obtain a first loss function; a second calculation subunit, used to perform loss calculation based on the predicted confidence matrix and the real confidence matrix to obtain a second loss function; a third calculation subunit, used to perform calculation based on the first loss function, the second loss function and the target weight to obtain a target loss function, wherein the target weight is calculated based on the first loss function and the second loss function; an iteration subunit, used to iterate the initial network structure based on the target loss function to obtain the target network structure.

[0029] Furthermore, the second acquisition unit includes: an acquisition subunit, used to acquire the real CT image and process the real CT image to obtain the sample CT image; a fourth calculation subunit, used to calculate the difference between each voxel point in the real CT image and each voxel point in the sample CT image to obtain a single voxel difference set; a first determination subunit, used to obtain the real confidence matrix based on the single voxel difference set; and a second determination subunit, used to obtain the training sample set based on the sample CT image, the real CT image and the real confidence matrix.

[0030] Furthermore, the reconstruction unit includes: a reconstruction subunit, used to perform three-dimensional reconstruction based on the target CT image to obtain an initial hip joint model; and a mapping subunit, used to perform confidence mapping on the initial hip joint model based on the target confidence matrix to obtain the target hip joint model.

[0031] Furthermore, the first determination unit includes: a first screening subunit, used to screen the acetabulum side of the target hip joint model to obtain an acetabulum side candidate point set; a second screening subunit, used to screen the femoral side of the target hip joint model to obtain a femoral side candidate point set; a third screening subunit, used to screen based on the acetabulum side candidate point set and the femoral side candidate point set to obtain the registration point set.

[0032] Furthermore, the first screening subunit includes: a first determination module, used to determine the direction vector corresponding to the acetabular cup center point and the acetabular cup implantation direction based on the target hip joint model; a second determination module, used to determine the normal vector corresponding to the first screening point within the target range centered on the acetabular cup center point, and calculate the angle value between the normal vector and the direction vector to obtain the target angle value; a first screening module, used to screen the first screening point within the target range according to the target angle value and the confidence corresponding to the first screening point to obtain the acetabular side candidate point set.

[0033] Furthermore, the second screening sub-unit includes: a third determination module, used to determine the target screening area based on the innermost point of the femoral stem, the outermost point on the femoral side, the apex of the greater trochanter and the bottom point of the lesser trochanter in the target hip joint model; a first judgment module, used to judge whether the second screening point in the target screening area belongs to the femoral area, and obtain a first judgment result; a second judgment module, used to judge whether the confidence corresponding to the second screening point is greater than or equal to a preset threshold, and obtain a second judgment result; a second screening module, used to screen the second screening point according to the first judgment result and the second judgment result, and obtain the femoral side candidate point set.

[0034] Further, the third screening sub-unit includes: a fourth determination module, used to determine a coarse registration point set corresponding to the coarse registration from the candidate points in the candidate point set based on the selection indication of the first object, wherein the candidate point set is the acetabulum side candidate point set, or the femoral side candidate point set; a clustering module, used to cluster the candidate points in the candidate point set to obtain a fine registration point set corresponding to the fine registration; a fifth determination module, used to obtain the registration point set based on the coarse registration point set and the fine registration point set.

[0035] Furthermore, the clustering module includes: a clustering submodule, used to cluster the candidate points in the candidate point set to obtain multiple candidate point subsets; a first calculation submodule, used to calculate the spatial volume of each candidate point subset to obtain multiple spatial volume values; a second calculation submodule, used to calculate the sampling ratio of each candidate point subset based on the multiple spatial volume values ​​to obtain multiple ratio values; a sampling submodule, used to randomly sample the multiple candidate point subsets based on the multiple ratio values ​​and the required number of precise alignment points to obtain the precise alignment point set.

[0036] Further, the determination unit includes: a first registration subunit, used to perform coarse registration on a coarse registration point set in the registration point set and a first acquisition point set corresponding to the coarse registration point set to obtain a coarse registration affine matrix, wherein the acquisition points in the first acquisition point set are obtained from a second object; a second registration subunit, used to perform, based on the coarse registration affine matrix, a fine registration point set in the registration point set and a second acquisition point set corresponding to the fine registration point set to obtain a fine registration affine matrix, wherein the acquisition points in the second acquisition point set are obtained from a second object; and a third determination subunit, used to obtain the target affine matrix based on the fine registration affine matrix and the coarse registration affine matrix.

[0037] Further, the first registration subunit includes: a sorting module, which is used to sort the coarse registration points in the coarse registration point set to obtain sorted coarse registration points, and sort the first acquisition points in the first acquisition point set to obtain sorted first acquisition points; a sixth determination module, which is used to determine the first vector and the second vector based on the connection between the sorted coarse registration points, and determine the third vector and the fourth vector based on the connection between the sorted first acquisition points; a seventh determination module, which is used to obtain a target rotation matrix based on the first vector, the second vector, the third vector and the fourth vector; a processing module, which is used to perform rotation processing on the first acquisition point based on the target rotation matrix to obtain a processed first acquisition point; an eighth determination module, which is used to obtain a translation matrix based on the distance between the processed first acquisition point and the coarse registration point; and a ninth determination module, which is used to obtain the coarse registration affine matrix based on the target rotation matrix and the translation matrix.

[0038] Furthermore, the second registration subunit includes: a perturbation module, which is used to apply random perturbations to the coarse registration affine matrix based on a normal distribution to obtain multiple first affine matrices; an optimization module, which is used to optimize the multiple first affine matrices through a data matching algorithm to obtain multiple second affine matrices; a merging module, which is used to merge similar affine matrices in the multiple second affine matrices when similar affine matrices exist in the multiple second affine matrices to obtain multiple third affine matrices; and a tenth determination module, which is used to obtain the fine registration affine matrix based on the multiple third affine matrices and the fine registration point set.

[0039] Further, the tenth determination module includes: a first mapping submodule, used to map the second acquisition points in the second acquisition point set based on the coarse registration affine matrix to obtain a third acquisition point set; a second mapping submodule, used to map the third acquisition points in the third acquisition point set based on each third affine matrix to obtain a fourth acquisition point set corresponding to each third affine matrix; a third calculation submodule, used to calculate the root mean square error of the distance between the fourth acquisition point in the fourth acquisition point set and the registration point in the fine registration point set to obtain multiple root mean square error sets; a determination submodule, used to determine the third affine matrix corresponding to the target root mean square error set as the fine registration affine matrix if there is an error in the target root mean square error set in the multiple root mean square error sets that is less than the error threshold.

[0040] Furthermore, the device also includes: a second determination unit, used to determine that an iteration is completed and judge whether the number of iterations is greater than or equal to a preset number if there is no target root mean square error set in the multiple root mean square error sets whose error is less than an error threshold; an execution unit, used to repeatedly execute the step of applying random perturbations to the third affine matrix with the smallest error of the root mean square error set based on a normal distribution if the number of iterations is less than the preset number, until the number of iterations is greater than or equal to the preset number, or the root mean square error set corresponding to the current affine matrix is ​​less than the error threshold.

[0041] Furthermore, the device also includes: a judgment unit, used to judge whether it is necessary to filter the set of precise alignment points after judging whether the number of iterations is greater than or equal to a preset number of times; a screening unit, used to filter the set of precise alignment points based on preset conditions when it is necessary to filter the set of precise alignment points to obtain a filtered set of precise alignment points, wherein the next iteration process is performed based on the filtered set of precise alignment points.

[0042] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer-readable storage medium is also provided, wherein the computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the steps of any one of the above-mentioned method embodiments.

[0043] In order to achieve the above-mentioned purpose, according to another aspect of the present application, there is further provided an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above-mentioned method embodiments.

[0044] In an embodiment of the present application, an initial CT image of a hip joint is acquired; the initial CT image is processed to remove metal artifacts through a target network structure to obtain a target CT image and a target confidence matrix corresponding to the target CT image, wherein the target confidence matrix is ​​composed of the confidence corresponding to each voxel point in the target CT image; three-dimensional reconstruction is performed based on the target CT image and the target confidence matrix to obtain a target hip joint model, and a set of registration points is determined based on the target hip joint model; a target affine matrix is ​​determined based on the set of registration points, wherein the target affine matrix is ​​used to provide data reference for hip joint surgery, thereby solving the technical problem in the related art of performing three-dimensional reconstruction through CT images with metal artifacts and then determining the affine matrix, resulting in relatively low accuracy in determining the affine matrix. In this scheme, after acquiring the initial CT image of the hip joint, the target network structure is used to remove metal artifacts from the initial CT image to obtain a processed target CT image and a target confidence matrix corresponding to the target CT image. Finally, three-dimensional reconstruction is performed based on the target CT image and the target confidence matrix, thereby improving the accuracy of the target hip joint model. Furthermore, the target affine matrix can be obtained more accurately based on the target hip joint model, thereby achieving the effect of improving the accuracy of determining the affine matrix. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 A hardware structure block diagram of a computer terminal for implementing a hip joint data processing method is shown;

[0047] Figure 2 The process of the hip joint data processing method provided in the embodiment of the present application is as follows Figure 1 ;

[0048] Figure 3 is a schematic diagram before and after removing metal artifacts according to an embodiment of the present application;

[0049] Figure 4 is a schematic diagram of a target network structure provided according to an embodiment of the present application;

[0050] Figure 5 The process of the hip joint data processing method provided in the embodiment of the present application is as follows Figure 2 ;

[0051] Figure 6 The process of the hip joint data processing method provided in the embodiment of the present application is as follows Figure 3 ;

[0052] Figure 7The process of the hip joint data processing method provided in the embodiment of the present application is as follows Figure 4 ;

[0053] Figure 8 The process of the hip joint data processing method provided in the embodiment of the present application is as follows Figure 5 ;

[0054] Fig. 9 The process of the hip joint data processing method provided in the embodiment of the present application is as follows Figure 6 ;

[0055] Fig.10 is a schematic diagram of a hip joint data processing device provided according to an embodiment of the present application;

[0056] Fig.11 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0058] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0059] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0060] Affine matrix: used to transform points in one space to another space.

[0061] It should be noted that the collected information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with relevant laws, regulations and standards, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation entrances are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions to provide users with corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered.

[0062] Example 1

[0063] According to an embodiment of the present application, a method embodiment of hip joint data processing is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0064] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a hip joint data processing method. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0065] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0066] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the hip joint data processing method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned hip joint data processing method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0067] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0068] The display may be a touch screen type liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0069] Under the above operating environment, this application provides Figure 2 Hip joint data processing method shown. Figure 2 The process of the hip joint data processing method according to the first embodiment of the present application is as follows Figure 1 .

[0070] Step S201, obtaining an initial CT image of the hip joint.

[0071] Optionally, an initial CT image of the hip joint of the target object is obtained. It should be noted that the initial CT image includes an image of the prosthesis.

[0072] Step S202, performing metal artifact removal processing on the initial CT image through the target network structure to obtain a target CT image and a target confidence matrix corresponding to the target CT image, wherein the target confidence matrix is ​​composed of the confidence corresponding to each voxel point in the target CT image.

[0073] Optionally, the initial CT image is input into the target network structure, and the initial CT image is processed by the target network structure to remove metal artifacts to obtain a target CT image and a target confidence matrix corresponding to the target CT image. It should be noted that the target confidence matrix is ​​composed of the confidence corresponding to each voxel point in the target CT image, and the confidence is used to indicate the degree of closeness between the voxel point and the actual situation of the target object. The higher the confidence, the more the point can reflect the actual situation of the target object. That is, a high confidence can indicate that the point is less affected by metal artifacts, and it still has high robustness after removing metal artifacts, and can reflect the actual bone situation at that position of the target object.

[0074] Step S203, performing three-dimensional reconstruction based on the target CT image and the target confidence matrix to obtain a target hip joint model, and determining a registration point set based on the target hip joint model.

[0075] Optionally, a 3D reconstruction model of the hip joint of the target object is generated by performing 3D reconstruction using the target CT image and the target confidence matrix, that is, obtaining the above-mentioned target hip joint model. Then, the registration points that can be used for registration are determined from the target hip joint model, thereby obtaining the above-mentioned registration point set.

[0076] Step S204, determining a target affine matrix based on the registration point set, wherein the target affine matrix is ​​used to provide data reference for hip joint surgery.

[0077] Optionally, a target affine matrix is ​​determined based on the above-mentioned set of registration points to provide data reference for hip joint surgery.

[0078] In summary, after acquiring the initial CT image of the hip joint, the target network structure is used to remove metal artifacts from the initial CT image to obtain the processed target CT image and the target confidence matrix corresponding to the target CT image. Finally, three-dimensional reconstruction is performed based on the target CT image and the target confidence matrix, which improves the accuracy of the target hip joint model. Further, the target affine matrix can be obtained more accurately based on the target hip joint model, thereby achieving the effect of improving the accuracy of determining the affine matrix.

[0079] Optionally, in the hip joint data processing method provided in an embodiment of the present application, the initial CT image is processed to remove metal artifacts through a target network structure to obtain a target CT image and a target confidence matrix corresponding to the target CT image, including: extracting features of the initial CT image and the sinusoidal graph corresponding to the initial CT image through a sinusoidal graph enhancement network in the target network structure to obtain first feature information; processing the first feature information through an image enhancement network in the target network structure to obtain a target CT image and a target confidence matrix.

[0080] In an optional embodiment, the following steps are used to remove metal artifacts from the initial CT image: the target network structure is composed of a sinogram enhancement network (Sinogram Enhancement Net, SE-Net) and an image enhancement network (Image Enhancement Net, IE-Net). The sinogram enhancement network is used to extract features from the initial CT image and the sinogram corresponding to the initial CT image to obtain first feature information, and then the image enhancement network in the target network structure is used to process the first feature information to obtain a target CT image and a target confidence matrix. For example, Figure 3 Shown are CT images before and after metal artifact removal (MAR) processing.

[0081] Processing the initial CT images through the target network structure can effectively remove metal artifacts and improve the accuracy of subsequent three-dimensional reconstruction.

[0082] Optionally, in the hip joint data processing method provided in an embodiment of the present application, the first feature information is processed by the image enhancement network in the target network structure to obtain a target CT image and a target confidence matrix, including: downsampling the first feature information through multiple downsampling layers of the image enhancement network to obtain second feature information corresponding to each downsampling layer; upsampling the second feature information corresponding to each downsampling layer through multiple first upsampling layers of the image enhancement network to obtain a target CT image; processing the second feature information corresponding to each downsampling layer and the output information of the target first upsampling layer through multiple second upsampling layers of the image enhancement network to obtain a target confidence matrix, wherein the target first upsampling layer is a sampling layer among the multiple first upsampling layers.

[0083] In an optional embodiment, the target CT image and target confidence matrix are obtained by the following steps: the image enhancement network is composed of a plurality of downsampling layers, a plurality of first upsampling layers and a plurality of second upsampling layers. Figure 4The schematic diagram of the target network structure shown in the figure includes a sinusoidal enhancement network and an image enhancement network. The image enhancement network includes multiple downsampling layers, multiple first upsampling layers and multiple second upsampling layers. The multiple second upsampling layers are connected layer by layer using a residual connection method. The first feature information is downsampled by multiple downsampling layers to obtain the second feature information corresponding to each downsampling layer, such as Figure 4 As shown, the output of each downsampling layer is connected to the input of each first upsampling layer. Therefore, the second feature information corresponding to each downsampling layer is upsampled through multiple first upsampling layers to obtain the target CT image.

[0084] like Figure 4 As shown, in order to obtain the target confidence matrix, the output of each downsampling layer is connected to the input of each second upsampling layer, and the output of the target first upsampling layer is also connected to the input of the second upsampling layer. Therefore, the second feature information corresponding to each downsampling layer and the output information of the target first upsampling layer are processed by multiple second upsampling layers to obtain the target confidence matrix. It should be noted that the target first upsampling layer is the first upsampling layer except the last first upsampling layer.

[0085] By processing the initial CT image through the target network structure, in addition to obtaining the target CT image excluding metal artifacts, the corresponding target confidence matrix can also be obtained. The target confidence matrix can be used to more accurately construct the subsequent three-dimensional model and improve the accuracy of selecting the registration points.

[0086] Optionally, in the hip joint data processing method provided in the embodiment of the present application, the following steps are used to train to obtain the target network structure: obtain a training sample set, wherein the training sample set includes: a sample CT image, a real CT image corresponding to the sample CT image, and a real confidence matrix corresponding to the real CT image; process the training sample set through the initial network structure to obtain a predicted CT image and a predicted confidence matrix corresponding to the predicted CT image; iterate the initial network structure according to the predicted CT image and the predicted confidence matrix to obtain the target network structure.

[0087] In an optional embodiment, the above-mentioned target network structure can be trained by the following steps: constructing a training sample set through sample CT images, real CT images corresponding to the sample CT images, and real confidence matrices corresponding to the real CT images, and then processing the training sample set through the initial network structure to obtain predicted CT images and predicted confidence matrices corresponding to the predicted CT images, and finally iterating the initial network structure according to the predicted CT images and the predicted confidence matrix to obtain the target network structure.

[0088] Through the above training method, the target network structure can accurately remove metal artifacts from CT images and more accurately obtain the confidence matrix of the CT image after metal artifacts are removed.

[0089] Optionally, in the hip joint data processing method provided in an embodiment of the present application, the initial network structure is iterated based on the predicted CT image and the predicted confidence matrix to obtain the target network structure, including: calculating the loss based on the predicted CT image and the real CT image to obtain a first loss function; calculating the loss based on the predicted confidence matrix and the real confidence matrix to obtain a second loss function; calculating based on the first loss function, the second loss function and the target weight to obtain a target loss function, wherein the target weight is calculated based on the first loss function and the second loss function; iterating the initial network structure based on the target loss function to obtain the target network structure.

[0090] In an optional embodiment, iterating the initial network structure by predicting the CT image and the prediction confidence matrix includes the following steps: firstly, performing loss calculation based on the predicted CT image and the real CT image to obtain a first loss function L out Then, the loss is calculated based on the predicted confidence matrix and the true confidence matrix to obtain the second loss function L conf .

[0091] In an optional embodiment, the second loss function may be expressed as follows:

[0092]

[0093] in, is the prediction confidence matrix of the predicted output, C conf is the label data (i.e., the true confidence matrix), and δ is a hyperparameter.

[0094] Finally, the target loss function is obtained by calculating the first loss function, the second loss function and the target weight, and the initial network structure is iterated through the target loss function to obtain the target network structure.

[0095] It should be noted that through Figure 4 It can be seen that the two task objectives of confidence matrix output and metal artifact-free CT image output are not of equal weight, and additional weights need to be introduced, namely the target weights mentioned above. For example, the target weight ω = [1, σ], where σ is an adaptive co-weight, which is used to balance the loss bias of the confidence matrix and the metal artifact-free CT image to ensure that the model can take into account both artifact removal and output confidence weights.

[0096] In an optional embodiment, the objective loss function may be expressed as follows:

[0097]

[0098] The initial value of σ is 1, and σ will be generated gradually and adaptively as the training progresses. Let L′ out and L′ conf is the loss of the previous step, then the σ of this step can be expressed as: σ = L′ out / L′ conf .

[0099] The above-mentioned target loss function can effectively balance the confidence matrix and the loss bias of CT images without metal artifacts, thereby improving the image processing performance of the target network structure.

[0100] Optionally, in the hip joint data processing method provided in the embodiment of the present application, obtaining a training sample set includes: obtaining a real CT image, and processing the real CT image to obtain a sample CT image; calculating the difference between each voxel point in the real CT image and each voxel point in the sample CT image to obtain a single voxel difference set; obtaining a real confidence matrix based on the single voxel difference set; obtaining a training sample set based on the sample CT image, the real CT image and the real confidence matrix.

[0101] In an optional embodiment, the following steps may be used to obtain the above training sample set: Acquire a real CT image, that is, a CT image without metal artifacts. truth Then, the sample CT image CT can be obtained by processing the real CT image, such as applying the photon starvation effect or the X-ray beam hardening effect simulation. in It should be noted that it is also possible to first obtain a CT image with metal artifacts. in , and then optimize the CT by iterative reconstruction method in Methods to obtain CT truth .

[0102] Then, the Hu (Hu value, also known as Hounsfield unit, is a unit used to quantify different tissue densities. It reflects the degree of tissue absorption of X-rays) threshold window T can be set. The design principle of the threshold window is: half of the difference between the upper and lower limits of the Hu value of the object to be extracted or segmented in CT, that is: T = (HU Up -HU down ) / 2.

[0103] Subsequently, the CT truth With CT in The difference between the two is recorded as S, that is, the difference between each voxel point in the real CT image and each voxel point in the sample CT image is calculated to obtain a set of single voxel differences. The single voxel confidence C can be expressed as:

[0104]

[0105] The true confidence matrix is ​​obtained through the single pixel difference set. Finally, the training sample set is obtained according to the sample CT image, the true CT image and the true confidence matrix.

[0106] Through the above steps, the training sample set can be quickly constructed to improve the efficiency of subsequent training network structure.

[0107] In an optional embodiment, the following may be used: Figure 5 The flowchart shown implements the training of the initial network structure, specifically including: collecting patient CT data, constructing metal artifact-free CT data, and generating a true confidence matrix, training the initial network structure model through patient CT data, metal artifact-free CT data and the true confidence matrix, calculating the loss function, and judging whether the training has been completed based on the loss function. If not, optimizing the model until the target network structure is obtained.

[0108] Optionally, in the hip joint data processing method provided in an embodiment of the present application, three-dimensional reconstruction is performed based on the target CT image and the target confidence matrix to obtain a target hip joint model, including: performing three-dimensional reconstruction based on the target CT image to obtain an initial hip joint model; and performing confidence mapping on the initial hip joint model based on the target confidence matrix to obtain a target hip joint model.

[0109] In an optional embodiment, the target hip joint model is obtained by the following steps: the target CT image is used to perform three-dimensional reconstruction to obtain an initial hip joint model, and then the ilium and femur models are used as mask layers, and the above mask layers are defined as M. ili and M fe , for C conf Perform masking to obtain the confidence Conf of the ilium and femur ili and Conf fe The masking process is as follows:

[0110] Conf=C·M

[0111] Where M is M ili and M fe By using the ilium and femur models as mask layers, the confidence of the ilium and femur in the target hip joint model can be obtained. ili and Conf fe. After masking, each feature point in the reconstructed bone model has a confidence level, which is used to indicate how close the voxel point is to the actual situation. The higher the confidence level, the more the point can reflect the actual situation. That is, a high confidence level can indicate that the point is less affected by metal artifacts, and it still has high robustness after MAR, and can reflect the actual bone situation at that location. In order to intuitively reflect the role of confidence, you can use Heatmap (a data visualization technology that displays the numerical value in the data matrix through color changes) to render the confidence level, that is, the target confidence matrix performs confidence mapping on the initial hip joint model. Blue indicates that the confidence level of the point is very high, and vice versa, red indicates that the confidence level of the point is low. The rendering result is as follows: Figure 6 As shown in the figure, it can be clearly seen from the rendering that the closer the part is to the metal prosthesis, the greater the impact of the metal artifact, which is consistent with the cause of the metal artifact. It should be noted that the initial hip joint model after rendering is the target hip joint model.

[0112] Three-dimensional reconstruction using the target confidence matrix and target CT images can more intuitively display the confidence in the target hip joint model and improve the accuracy of subsequent selection of registration points.

[0113] Optionally, in the hip joint data processing method provided in an embodiment of the present application, determining the alignment point set based on the target hip joint model includes: screening the acetabulum side of the target hip joint model to obtain an acetabulum side candidate point set; screening the femoral side of the target hip joint model to obtain a femoral side candidate point set; screening based on the acetabulum side candidate point set and the femoral side candidate point set to obtain a alignment point set.

[0114] In an optional embodiment, the following steps are used to select the registration points: first, the acetabulum side of the target hip joint model is screened to obtain an acetabulum side candidate point set, and the femoral side of the target hip joint model is screened to obtain a femoral side candidate point set; finally, the acetabulum side candidate point set and the femoral side candidate point set are screened to obtain the final registration point set.

[0115] By distinguishing the parts and then selecting candidate points according to different parts, the accuracy of the registration point set is improved.

[0116] Optionally, in the hip joint data processing method provided in the embodiment of the present application, the acetabulum side of the target hip joint model is screened to obtain a set of candidate points on the acetabulum side, including: determining the direction vector corresponding to the center point of the acetabulum cup and the implantation direction of the acetabulum cup based on the target hip joint model; determining the normal vector corresponding to the first screening point within the target range centered at the center point of the acetabulum cup, and calculating the angle value between the normal vector and the direction vector to obtain the target angle value; screening the first screening point within the target range based on the target angle value and the confidence level corresponding to the first screening point to obtain a set of candidate points on the acetabulum side.

[0117] In an optional embodiment, screening the acetabulum side of the target hip joint model includes: determining the acetabulum cup center point P according to the target hip joint model ilium The direction vector V corresponding to the acetabular cup implantation direction ilium Then, the acetabular cup center P ilium As the starting point of expansion, its coordinates in the target hip joint model are recorded as [x0, y0, z0], and the point set P has been selected. set = {P ilium}, acetabular side candidate point set, initial screening area radius r = 0. Traverse P set The newly added screening point P i , with P i The eight neighborhoods of P are used as the expansion candidate points ik , the corresponding coordinates are [x ik ,y ik ,z ik ], if P ik If the point belongs to the acetabulum area, it is determined whether the point is within the screening area (i.e., the above-mentioned target range). If the point is within the above-mentioned target range, the point is used as the above-mentioned first screening point. The following formula can be used to determine whether the point is within the target range:

[0118]

[0119] Then, the first screening point in the target range is screened according to the angle between the normal vector corresponding to the first screening point and the direction vector and the confidence corresponding to the first screening point to obtain a set of candidate points on the acetabulum side. For example, first determine the angle between the normal line of the point and the implantation direction. Let the normal vector of the first screening point be H ijk , calculate H ijk and V ilium The angle θ between ijk (That is, the target angle value mentioned above): If θ ijk >90°, then the point cannot be used as an alternative point. Determine whether the minimum confidence requirement is met. If P ik To be eligible for screening, it is necessary to determine whether the confidence of the point meets the minimum confidence requirement: Confik ≥γ, if P ik Can meet the alternative requirements, then P ik is added to the acetabular side candidate point set. ik Whether the requirements are met, P ik will be added to P set The screening area is gradually expanded. Assume that the maximum screening boundary is R, repeat the above steps and gradually expand the screening area radius r until r = R. R can be set to the diameter d of the prosthesis model, or the size of the surgical window that can be measured.

[0120] By distinguishing the locations and screening the candidate points according to the confidence level, the accuracy of determining the candidate point set on the acetabulum side is improved.

[0121] Optionally, in the hip joint data processing method provided in an embodiment of the present application, the femoral side of the target hip joint model is screened to obtain a femoral side candidate point set including: determining a target screening area based on the innermost point of the femoral stem, the outermost point of the femoral side, the apex of the greater trochanter and the bottom point of the lesser trochanter in the target hip joint model; judging whether a second screening point in the target screening area belongs to the femoral area to obtain a first judgment result; judging whether the confidence level corresponding to the second screening point is greater than or equal to a preset threshold to obtain a second judgment result; and screening the second screening point based on the first judgment result and the second judgment result to obtain a femoral side candidate point set.

[0122] In an optional embodiment, the following steps are used to obtain a set of candidate points on the femoral side: Different from the hip side, there is an obvious screening interval on the femoral side. The innermost point of the femoral stem, the outermost point on the femoral side, the apex of the greater trochanter, and the bottom point of the lesser trochanter constitute the registration screening area S (i.e., the target screening area mentioned above). All voxel points in the area are traversed to determine whether the point belongs to the femoral area; if it belongs to the femoral area, determine whether the confidence corresponding to the second screening point is greater than or equal to a preset threshold (i.e., Conf ik ≥γ), if the confidence corresponding to the second screening point is greater than or equal to the preset threshold and belongs to the femoral area, then the second screening point is a femoral candidate point and can be added to the femoral candidate point set.

[0123] By distinguishing the parts and screening the candidate points according to the confidence level, the accuracy of determining the candidate point set on the femoral side is improved.

[0124] Optionally, in the hip joint data processing method provided in the embodiment of the present application, screening is performed based on the acetabulum side candidate point set and the femoral side candidate point set to obtain the registration point set, including: determining a coarse registration point set corresponding to the coarse registration from the candidate points in the candidate point set based on the selection indication of the first object, wherein the candidate point set is the acetabulum side candidate point set, or the femoral side candidate point set; clustering the candidate points in the candidate point set to obtain a fine registration point set corresponding to the fine registration; and obtaining the registration point set based on the coarse registration point set and the fine registration point set.

[0125] In an optional embodiment, screening from the acetabulum side candidate point set and the femoral side candidate point set includes the following steps: according to the selection indication of the first object, from the candidate points in the acetabulum side candidate point set or the femoral side candidate point set, determine the coarse registration point set corresponding to the coarse registration. For example, manually select 3 from the candidate point set, and require that the lengths of the lines between the three feature points are unequal. When the candidate point set is the acetabulum side candidate point set, the coarse registration point set corresponding to the coarse registration is the coarse registration point set corresponding to the acetabulum side, and when the candidate point set is the femoral side candidate point set, the coarse registration point set corresponding to the coarse registration is the coarse registration point set corresponding to the femoral side.

[0126] Then, the candidate points in the candidate point set can be clustered by the DBSCAN method to obtain a set of fine registration points corresponding to the fine registration, and finally the registration point set can be obtained according to the coarse registration point set and the fine registration point set. It should be noted that when the candidate point set is the acetabular side candidate point set, the fine registration point set is the fine registration point set corresponding to the acetabular side, and when the candidate point set is the femoral side candidate point set, the fine registration point set is the fine registration point set corresponding to the femoral side.

[0127] The coarse registration point set and the fine registration point set facilitate the subsequent more accurate determination of the target affine matrix.

[0128] Optionally, in the hip joint data processing method provided in the embodiment of the present application, clustering is performed on the candidate points in the candidate point set to obtain a set of precise alignment points corresponding to the precise alignment, including: clustering is performed on the candidate points in the candidate point set to obtain multiple candidate point subsets; the spatial volume of each candidate point subset is calculated to obtain multiple spatial volume values; the sampling ratio of each candidate point subset is calculated based on the multiple spatial volume values ​​to obtain multiple ratio values; and multiple candidate point subsets are randomly sampled based on the multiple ratio values ​​and the required number of precise alignment points to obtain a set of precise alignment points.

[0129] In an optional embodiment, the following steps can be used to cluster the candidate points in the candidate point set to obtain a set of precise registration points corresponding to the precise registration: cluster the candidate points on the femoral side and the acetabulum side using the DBSCAN method, and cluster the point set OPset Divide into k sub-clusters, each sub-cluster is represented by C j In other words, clustering is performed on the candidate points in the candidate point set to obtain multiple candidate point subsets.

[0130] Then, the volume m of the space represented by each subcluster can be calculated by the Quickhull algorithm: j , that is, the spatial volume value of each candidate point subset. The sampling ratio of each candidate point subset is calculated according to multiple spatial volume values ​​to obtain multiple ratio values, for example, the sampling ratio p of each subcluster i It can be expressed as follows: in, It represents the sum of the spatial volumes occupied by all subclusters.

[0131] Finally, according to the multiple ratio values ​​and the required number of precise alignment points, multiple subsets of candidate points are randomly sampled to obtain a set of precise alignment points. For example, if the required number of precise alignment points is N, the number of points that need to be sampled in each subset is n. j , then n j It can be expressed by the following formula: In the formula Indicates rounding down.

[0132] In an optional embodiment, the planner can adjust the number of samples of each sub-cluster according to the actual situation of each sub-cluster. For example, the number of sampling points for sparse sub-cluster is increased, and sub-cluster that is too small is not sampled. The planner needs to ensure that after the division,

[0133] When the number of samples is determined, random sampling can be used to select n j ≠0 for random sampling: P = Sample (C j ,n j )for j=1,2,…,k, P is the precise alignment point set.

[0134] It should be noted that the Quickhull algorithm and the DBSCAN algorithm are not required algorithms for this solution, and users can also choose clustering methods and volume calculation methods by themselves.

[0135] By clustering the candidate points and calculating the proportion of the clustered sub-clusters, the accuracy of selecting the precise alignment points is improved.

[0136] In an optional embodiment, the following may be used: Figure 7The flowchart shown realizes the selection of precise registration points, specifically including: inputting the acquired initial CT, obtaining MAR images (i.e. CT without metal artifacts) and confidence matrix through model prediction, performing CT segmentation and 3D reconstruction based on MAR images, mapping the confidence matrix to the 3D skeleton model, and then obtaining a set of candidate points based on the 3D skeleton model. Clustering the candidate set to obtain candidate subclusters, assigning sampling weights to the selected subclusters and calculating and fine-tuning the number of subclusters sampled, sampling precise registration points according to the sampling number, and obtaining a set of precise registration points.

[0137] Optionally, in the hip joint data processing method provided in the embodiment of the present application, determining the target affine matrix based on the registration point set includes: coarsely aligning a coarse registration point set in the registration point set with a first acquisition point set corresponding to the coarse registration point set to obtain a coarse registration affine matrix, wherein the acquisition points in the first acquisition point set are obtained from the second object; based on the coarse registration affine matrix, aligning a fine registration point set in the registration point set with a second acquisition point set corresponding to the fine registration point set to obtain a fine registration affine matrix, wherein the acquisition points in the second acquisition point set are obtained from the second object; and obtaining the target affine matrix based on the fine registration affine matrix and the coarse registration affine matrix.

[0138] In an optional embodiment, the target affine matrix is ​​obtained by the following steps: First, a coarse registration point set in the registration point set is coarsely registered with the first acquisition point set corresponding to the coarse registration point set to obtain a coarse registration affine matrix T R It should be noted that the first acquisition point can be acquired from the second object. Then, through the coarse registration affine matrix, the fine registration point set in the registration point set and the second acquisition point set corresponding to the fine registration point set are finely registered to obtain the fine registration affine matrix T fine Finally, the target affine matrix is ​​obtained based on the fine registration affine matrix and the coarse registration affine matrix. For example, the target affine matrix T final It can be obtained by the following formula: T final =T fine T R It should be noted that the target affine matrix T final It is also divided into the target affine matrix corresponding to the acetabulum side and the target affine matrix corresponding to the femoral side.

[0139] Through the two-step registration process of coarse registration and fine registration, the accuracy of the target affine matrix can be effectively improved.

[0140] Optionally, in the hip joint data processing method provided in the embodiment of the present application, a coarse registration point set in the registration point set and a first acquisition point set corresponding to the coarse registration point set are coarsely registered to obtain a coarse registration affine matrix, including: sorting the coarse registration points in the coarse registration point set to obtain sorted coarse registration points, and sorting the first acquisition points in the first acquisition point set to obtain sorted first acquisition points; determining a first vector and a second vector based on the connecting lines between the sorted coarse registration points, and determining a third vector and a fourth vector based on the connecting lines between the sorted first acquisition points; obtaining a target rotation matrix based on the first vector, the second vector, the third vector and the fourth vector; rotating the first acquisition point based on the target rotation matrix to obtain a processed first acquisition point; obtaining a translation matrix based on the distance between the processed first acquisition point and the coarse registration point; and obtaining a coarse registration affine matrix based on the target rotation matrix and the translation matrix.

[0141] In an optional embodiment, the following steps are used to implement the coarse registration: sorting the coarse registration points in the coarse registration point set to obtain sorted coarse registration points, and sorting the first acquisition point in the first acquisition point set to obtain the sorted first acquisition point. plan =[A0, B0, C0], where A0, B0, C0 represent the coordinates of the three registration points in the target hip joint model; let the first acquisition point acquired be P world =[A w ,B w ,C w ], where A w ,B w ,C w Represents the coordinates of the three registration points collected in the optical navigator space. Calculate the Euclidean distance between the three feature points A, B, and C, and let the two points with the largest distance be the AC candidate points, and the remaining point be B; then, select the point farther from point B as A and the point closer to point B as C from the AC candidate points, complete the sorting process, and get the sorted P plan and P world .

[0142] Then, the first vector and the second vector are determined based on the connection between the sorted rough registration points, and the third vector and the fourth vector are determined based on the connection between the sorted first acquisition points. For example, let the vector from point A to point C in the sorted rough registration points be V AC , the vector from point A to point B is V AB ; Let the corresponding vector of the first collected point after sorting be V′ AC and V′ AB .

[0143] Then, according to the first vector, the second vector, the third vector and the fourth vector, the target rotation matrix is ​​obtained. For example, the first vector, the second vector, the third vector and the fourth vector are first normalized and the normalized V AC and V AB The vector group A = [V AC ,V AB ], and correspondingly construct the vectors in the collection point set into a vector group B = [V A ' C ,V A ' B ] Then the covariance matrix H can be expressed as: H = A·B T ; Perform singular value decomposition on the covariance matrix and construct the rotation matrix R. After obtaining the covariance matrix, use singular value decomposition to obtain U, L, V t , that is, U, L, V t =SVD(H); Then, we can get the rotation matrix R:

[0144] It should be noted that after obtaining the rotation matrix R, it is also necessary to ensure that its determinant is positive. If the determinant is negative, V t The last row of is multiplied by -1, and the rotation matrix is ​​recalculated to ensure that the obtained rotation matrix R is an orthogonal matrix.

[0145] After obtaining the target rotation matrix, the first acquisition point is rotated according to the target rotation matrix to obtain the processed first acquisition point, and then the translation matrix is ​​obtained according to the distance between the processed first acquisition point and the coarse registration point. Finally, the coarse registration affine matrix is ​​obtained according to the target rotation matrix and the translation matrix. For example, let the point in the coarse registration indicator set be P, and the midpoint of the acquisition point set be P'. Use R to rotate P' to obtain the new position P' R , and calculate P and P′ R The distance between them is taken as the translation amount M: M = PP' R , P′ R =R·P′. Finally, the obtained rotation matrix and translation matrix are combined and written into homogeneous form to obtain the affine matrix T of the rough registration R :

[0146] In an optional embodiment, in order to improve the accuracy of the coarse registration affine matrix, an angle constraint may be added, that is, the initial coarse registration affine matrix T is obtained according to the target rotation matrix and translation matrix. R , then, T R Applied to P world , get the sampling point P in the CT coordinate system ct :

[0147] Let P ct The geometric center of world , P plan The geometric center of plan , then the implantation point position (MI) of the implanted fixation nail can be obtained. w ) of the vector V world and V plan , calculate V world and V plan The angle θ between ili , if Θ ili If the angle is less than 30°, the hip side registration can be used to obtain the expected rough registration result. Similarly, for the rough registration point set on the femoral side, the midpoint E of the line connecting the ligament attachment points of the medial and lateral femoral condyles is selected. w As a reference point, get p to E w Vector, if the vector angle Θ fe If the angle is less than 30°, the femoral registration can be considered to have achieved the expected result. Otherwise, it is necessary to re-collect the registration points and repeat the above steps to obtain a rough registration point set.

[0148] In an optional embodiment, the following may be used: Figure 8 The flowchart shown implements group registration, specifically including: determining coarse registration points and acquiring first acquisition points, sorting the first acquisition points and coarse registration points, constructing vectors based on the sorted points, and obtaining a rotation matrix based on the vectors, and then calculating a translation matrix, constructing a coarse registration affine matrix based on the rotation matrix and the translation matrix, and judging whether the angle constraint conditions are met. If so, the process ends; if not, the coarse registration points are re-determined and the first acquisition points are acquired.

[0149] Optionally, in the hip joint data processing method provided in the embodiment of the present application, based on the coarse registration affine matrix, the fine registration point set in the registration point set and the second acquisition point set corresponding to the fine registration point set are registered to obtain the fine registration affine matrix, including: applying random perturbations to the coarse registration affine matrix based on the normal distribution to obtain multiple first affine matrices; optimizing the multiple first affine matrices through a data matching algorithm to obtain multiple second affine matrices; when similar affine matrices exist in the multiple second affine matrices, merging the similar affine matrices in the multiple second affine matrices to obtain multiple third affine matrices; and obtaining the fine registration affine matrix based on the multiple third affine matrices and the fine registration point set.

[0150] In an optional embodiment, the fine registration affine matrix is ​​obtained by the following steps: applying random perturbations to the coarse registration affine matrix through normal distribution to obtain a plurality of first affine matrices, for example, the plurality of first affine matrices are a perturbation matrix group T D ={T D1 …T DnThen, the multiple first affine matrices are optimized by the ICP (Iterative Closest Point) iterative closest point algorithm (i.e., the above-mentioned data matching algorithm) to obtain multiple second affine matrices. For example, the multiple second affine matrices are G T ={T1…T m}.

[0151] First, the second collection point in the second collection point set is mapped according to the coarse registration affine matrix to obtain the third collection point set, and then, G T The affine matrix in the third collection point set U one by one P Perform affine transformation to obtain the moved affine point set U TP = {U T1 …U Tn}. Then, TP The midpoints are compared one by one. If two affine point subsets U Ti and U Tk If the error of all the moved precision alignment points in is less than 1 mm, the two point sets are considered to be close point sets, and the corresponding affine matrix T can be deleted. Dk , keep T Di , that is, similar affine matrices in the multiple second affine matrices are merged to obtain multiple third affine matrices, and finally a precise registration affine matrix is ​​obtained according to the multiple third affine matrices and the precise registration point set.

[0152] The fine registration affine matrix can be determined more accurately by applying random perturbations to the coarse registration affine matrix and using a data matching algorithm.

[0153] Optionally, in the hip joint data processing method provided in the embodiment of the present application, obtaining a fine registration affine matrix based on multiple third affine matrices and fine registration point sets includes: mapping the second acquisition point in the second acquisition point set based on the coarse registration affine matrix to obtain a third acquisition point set; mapping the third acquisition point in the third acquisition point set based on each third affine matrix to obtain a fourth acquisition point set corresponding to each third affine matrix; calculating the root mean square error of the distance between the fourth acquisition point in the fourth acquisition point set and the registration point in the fine registration point set to obtain multiple root mean square error sets; if there is an error in the target root mean square error set in the multiple root mean square error sets that is less than the error threshold, then the third affine matrix corresponding to the target root mean square error set is determined as the fine registration affine matrix.

[0154] If there is no target root mean square error set in the multiple root mean square error sets whose error is less than the error threshold, it is determined that one iteration is completed, and it is judged whether the number of iterations is greater than or equal to the preset number; if the number of iterations is less than the preset number, the step of applying random perturbations to the third affine matrix with the smallest error in the root mean square error set based on the normal distribution is repeated until the number of iterations is greater than or equal to the preset number, or the root mean square error set corresponding to the current affine matrix is ​​less than the error threshold.

[0155] In an optional embodiment, the following steps are used to obtain the fine registration affine matrix: first, the second collection points in the second collection point set are mapped according to the coarse registration affine matrix to obtain a third collection point set U P , U P = {P1…P n Then, the third acquisition points in the third acquisition point set are mapped according to each third affine matrix to obtain a fourth acquisition point set corresponding to each third affine matrix, and the root mean square error of the distance between the fourth acquisition point and the registration point in the precise registration point set is calculated. For example, the root mean square error (RMSE) is Where d is the distance between each point and the registration point.

[0156] Determine whether there is a target root mean square error set whose error is less than the error threshold in multiple root mean square error sets. If there is a target root mean square error set whose error is less than the error threshold ε, that is, RMSE imin <ε, the termination condition is met, and the corresponding three affine matrices are the optimal registration matrix T fine (That is, the precise registration affine matrix mentioned above).

[0157] If the termination condition is not met, that is, there is no target root mean square error set whose error is less than the error threshold, then it is determined that one iteration is completed, and it is determined whether the number of iterations is greater than or equal to the preset number β max If the number of iterations is less than the preset number, the step of applying random perturbations to the third affine matrix with the smallest error of the root mean square error set based on the normal distribution is repeated until the number of iterations is greater than or equal to the preset number, or the root mean square error set corresponding to the current affine matrix is ​​less than the error threshold.

[0158] The error threshold can be used to accurately evaluate the accuracy of the affine matrix and improve the accuracy of the final precise registration affine matrix.

[0159] Optionally, in the hip joint data processing method provided in the embodiment of the present application, after determining whether the number of iterations is greater than or equal to a preset number of times, the method further includes: determining whether it is necessary to filter the set of precise alignment points; if it is necessary to filter the set of precise alignment points, filtering the set of precise alignment points based on preset conditions to obtain a filtered set of precise alignment points, wherein the next iteration process is executed based on the filtered set of precise alignment points.

[0160] In an optional embodiment, after determining whether the number of iterations is greater than or equal to the preset number, if the number of iterations is less than the preset number, it is necessary to determine whether the set of precise alignment points needs to be screened. If the RMSE imin ≥2ε, the optimization interval has not been reached yet, and there is no need to screen the precise registration point set. The next iteration can be performed directly. If screening is required, the third affine matrix with the smallest error of the root mean square error set of the previous iteration is determined, and the third acquisition point set is mapped according to the third affine matrix to obtain the affine point set U fine .

[0161] Traverse U fine Extract point P i , calculate the surface subgroup G that is closest to i The distance D i , forming a distance set U dis ={D1…D n}. When RMSE imin ≥2ε means that the error has been reduced to a lower value in this iteration. Due to the existence of artificial sampling errors, if the outliers are not removed in time, the final registration accuracy may be affected. i It is extracted from the mesh model of the reconstructed 3D model. First, all surface points are extracted from the mesh model to form the bone surface point set U m ={m1…m n}, according to the k sub-clusters obtained by the precise matching point extraction, the surface sub-clusters set G belonging to the sub-cluster containing the sampling point is obtained. U = {G1…G k}, that is, the surface subgroup G i .

[0162] Therefore, it is necessary to screen and remove outliers and corresponding subgroups. dis , point P in the following three cases i The corresponding accurate point or subgroup G i Should be deleted:

[0163] 1. P i The distance D to its nearest subcluster i ≥2ε, then Pi It can be considered as an outlier point that cannot be aligned correctly and needs to be deleted. i When an outlier point is deleted, if the subcluster to which the outlier point corresponds no longer contains the corresponding fine alignment point, the subcluster should also be deleted synchronously.

[0164] 2. Multiple points [P i …P j ]'s nearest subgroups are all G i , then [P i …P j ], there are multiple abnormal points, because it is impossible to further determine which point has an abnormality or G i There is a large distortion in the reconstruction itself, so [P i …P j ] The corresponding precise alignment points should be deleted, and G i should also be deleted at the same time.

[0165] 3. Most of them are related to G i The relevant point distance [D i …D j ] are greater than 2δ, which means that G i The area represented by G is greatly distorted during reconstruction. i and G i The corresponding registration point.

[0166] It should be noted that after the precise registration point set is screened, it is necessary to determine whether the screened precise registration point set can be used for the next iteration. The number of remaining registration points is less than N / 2 or the sum of the spatial volumes occupied by the remaining subclusters is less than It is considered that the precise registration cannot be continued, and forced registration will only lead to falling into the local optimum. The precise registration points should be collected again and re-registered.

[0167] In an optional embodiment, applying random perturbations to the coarse registration affine matrix using a normal distribution includes:

[0168] Let the current number of iterations be k, the maximum number of iterations be n, and the number of perturbations to be generated this time be p. Initialize the translation perturbation component. Generate the translation perturbation matrix T according to the normal distribution p , the standard deviation of each perturbation is 10% of the model size and decreases gradually with the number of iterations: [p,3]). Initialize the rotation perturbation component. Use normal distribution to generate the rotation angle θ around the x, y, and z axes x ,θ y ,θ z , the standard deviation of each angle is 10°, and it decreases gradually with the increase of the number of iterations: [1,p]) / 180 corresponds to the rotation disturbance component R of the axis x , R y , R z for:

[0169]

[0170] The total rotational disturbance component R is: R = R x ·R y ·R z , according to the above translation perturbation matrix T p And the total rotation perturbation component R obtains the multiple first affine matrices mentioned above.

[0171] In an optional embodiment, the following may be used: Fig. 9 The flowchart shown implements the precise registration, specifically including the following steps: collecting the second collection point set and determining the precise registration point set, mapping the second collection point in the second collection point set according to the coarse registration affine matrix to obtain the third collection point set, setting the maximum number of iterations and the exit iteration threshold ε, randomly generating multiple perturbation affine matrices, optimizing each perturbation affine matrix using the ICP algorithm to obtain an optimized perturbation affine matrix, merging the optimized perturbation affine matrices with similar results, finding the optimized perturbation affine matrix with the minimum error, judging whether the minimum error is less than the exit iteration threshold, if less than, determining the optimized perturbation affine matrix with the minimum error as the final precise registration affine matrix. If not less than, judging whether the number of iterations reaches the maximum number of iterations, if reached, determining the optimized perturbation affine matrix with the minimum error as the final precise registration affine matrix, if not reached, judging whether it is necessary to screen the precise registration points, if necessary, screening the precise registration points, and judging whether the remaining precise registration points support continued iteration, if not supported, re-collecting the second collection point set and determining the precise registration point set. If the screening of precise alignment points is supported or not required, the process of randomly generating multiple perturbed affine matrices is repeated.

[0172] The hip joint data processing method provided in the embodiment of the present application obtains an initial CT image of the hip joint; performs metal artifact removal processing on the initial CT image through a target network structure to obtain a target CT image and a target confidence matrix corresponding to the target CT image, wherein the target confidence matrix is ​​composed of the confidence corresponding to each voxel point in the target CT image; performs three-dimensional reconstruction based on the target CT image and the target confidence matrix to obtain a target hip joint model, and determines a set of alignment points based on the target hip joint model; determines a target affine matrix based on the set of alignment points, wherein the target affine matrix is ​​used to provide data reference for hip joint surgery, thereby solving the technical problem in the related art of performing three-dimensional reconstruction through CT images with metal artifacts and then determining the affine matrix, resulting in relatively low accuracy in determining the affine matrix. In this scheme, after acquiring the initial CT image of the hip joint, the target network structure is used to remove metal artifacts from the initial CT image to obtain a processed target CT image and a target confidence matrix corresponding to the target CT image. Finally, three-dimensional reconstruction is performed based on the target CT image and the target confidence matrix, thereby improving the accuracy of the target hip joint model. Furthermore, the target affine matrix can be obtained more accurately based on the target hip joint model, thereby achieving the effect of improving the accuracy of determining the affine matrix.

[0173] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0174] Example 2

[0175] The embodiment of the present application also provides a hip joint data processing device. It should be noted that the hip joint data processing device of the embodiment of the present application can be used to execute the hip joint data processing method provided in the embodiment of the present application. The hip joint data processing device provided in the embodiment of the present application is introduced below.

[0176] According to an embodiment of the present application, a device for implementing the above hip joint data processing method is also provided, such as Fig.10 As shown, the device includes: a first acquisition unit 1001, a first processing unit 1002, a reconstruction unit 1003 and a first determination unit 1004.

[0177] A first acquisition unit 1001 is used to acquire an initial CT image of a hip joint;

[0178] The first processing unit 1002 is used to perform metal artifact removal processing on the initial CT image through the target network structure to obtain a target CT image and a target confidence matrix corresponding to the target CT image, wherein the target confidence matrix is ​​composed of confidences corresponding to each voxel point in the target CT image;

[0179] The reconstruction unit 1003 is used to perform three-dimensional reconstruction based on the target CT image and the target confidence matrix to obtain a target hip joint model, and determine a registration point set based on the target hip joint model;

[0180] The first determination unit 1004 is used to determine a target affine matrix based on the set of registration points, wherein the target affine matrix is ​​used to provide data reference for hip joint surgery.

[0181] The hip joint data processing device provided in the embodiment of the present application acquires the initial CT image of the hip joint through the first acquisition unit 1001; the first processing unit 1002 performs metal artifact removal processing on the initial CT image through the target network structure to obtain the target CT image and the target confidence matrix corresponding to the target CT image, wherein the target confidence matrix is ​​composed of the confidence corresponding to each voxel point in the target CT image; the reconstruction unit 1003 performs three-dimensional reconstruction based on the target CT image and the target confidence matrix to obtain the target hip joint model, and determines the registration point set based on the target hip joint model; the first determination unit 1004 solves the technical problem in the related art that the affine matrix is ​​determined by three-dimensional reconstruction of the CT image with metal artifacts based on the registration point set, resulting in relatively low accuracy in determining the affine matrix. In this scheme, after acquiring the initial CT image of the hip joint, the target network structure is used to remove metal artifacts from the initial CT image to obtain a processed target CT image and a target confidence matrix corresponding to the target CT image. Finally, three-dimensional reconstruction is performed based on the target CT image and the target confidence matrix, thereby improving the accuracy of the target hip joint model. Furthermore, the target affine matrix can be obtained more accurately based on the target hip joint model, thereby achieving the effect of improving the accuracy of determining the affine matrix.

[0182] Optionally, in the hip joint data processing device provided in an embodiment of the present application, the first processing unit includes: an extraction subunit, used to perform feature extraction on the initial CT image and the sinusoidal graph corresponding to the initial CT image through a sinusoidal graph enhancement network in a target network structure to obtain first feature information; a processing subunit, used to process the first feature information through an image enhancement network in the target network structure to obtain a target CT image and a target confidence matrix.

[0183] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the processing subunit includes: a first sampling module, used to downsample the first feature information through multiple downsampling layers of the image enhancement network to obtain second feature information corresponding to each downsampling layer; a second sampling module, used to upsample the second feature information corresponding to each downsampling layer through multiple first upsampling layers of the image enhancement network to obtain a target CT image; a processing module, used to process the second feature information corresponding to each downsampling layer and the output information of the target first upsampling layer through multiple second upsampling layers of the image enhancement network to obtain a target confidence matrix, wherein the target first upsampling layer is a sampling layer among the multiple first upsampling layers.

[0184] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the following devices are used for training to obtain the target network structure: a second acquisition unit, used to acquire a training sample set, wherein the training sample set includes: a sample CT image, a real CT image corresponding to the sample CT image, and a real confidence matrix corresponding to the real CT image; a second processing unit, used to process the training sample set through the initial network structure to obtain a predicted CT image and a predicted confidence matrix corresponding to the predicted CT image; an iteration unit, used to iterate the initial network structure according to the predicted CT image and the predicted confidence matrix to obtain the target network structure.

[0185] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the iteration unit includes: a first calculation subunit, used to perform loss calculation based on the predicted CT image and the real CT image to obtain a first loss function; a second calculation subunit, used to perform loss calculation based on the predicted confidence matrix and the real confidence matrix to obtain a second loss function; a third calculation subunit, used to perform calculation based on the first loss function, the second loss function and the target weight to obtain a target loss function, wherein the target weight is calculated based on the first loss function and the second loss function; an iteration subunit, used to iterate the initial network structure based on the target loss function to obtain a target network structure.

[0186] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the second acquisition unit includes: an acquisition subunit, used to acquire a real CT image, and process the real CT image to obtain a sample CT image; a fourth calculation subunit, used to calculate the difference between each voxel point in the real CT image and each voxel point in the sample CT image, to obtain a single voxel difference set; a first determination subunit, used to obtain a real confidence matrix based on the single voxel difference set; a second determination subunit, used to obtain a training sample set based on the sample CT image, the real CT image and the real confidence matrix.

[0187] Optionally, in the hip joint data processing device provided in an embodiment of the present application, the reconstruction unit includes: a reconstruction subunit, used to perform three-dimensional reconstruction based on the target CT image to obtain an initial hip joint model; and a mapping subunit, used to perform confidence mapping on the initial hip joint model based on a target confidence matrix to obtain a target hip joint model.

[0188] Optionally, in the hip joint data processing device provided in an embodiment of the present application, the first determination unit includes: a first screening subunit, used to screen the acetabulum side of the target hip joint model to obtain an acetabulum side candidate point set; a second screening subunit, used to screen the femoral side of the target hip joint model to obtain a femoral side candidate point set; a third screening subunit, used to screen based on the acetabulum side candidate point set and the femoral side candidate point set to obtain a registration point set.

[0189] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the first screening subunit includes: a first determination module, used to determine the direction vector corresponding to the acetabular cup center point and the acetabular cup implantation direction based on the target hip joint model; a second determination module, used to determine the normal vector corresponding to the first screening point within the target range centered on the acetabular cup center point, and calculate the angle value between the normal vector and the direction vector to obtain the target angle value; a first screening module, used to screen the first screening point within the target range according to the target angle value and the confidence level corresponding to the first screening point to obtain a set of acetabular side candidate points.

[0190] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the second screening sub-unit includes: a third determination module, used to determine the target screening area based on the innermost point of the femoral stem, the outermost point on the femoral side, the apex of the greater trochanter and the bottom point of the lesser trochanter in the target hip joint model; a first judgment module, used to judge whether the second screening point in the target screening area belongs to the femoral area, and obtain a first judgment result; a second judgment module, used to judge whether the confidence corresponding to the second screening point is greater than or equal to a preset threshold, and obtain a second judgment result; a second screening module, used to screen the second screening point according to the first judgment result and the second judgment result, and obtain a femoral side candidate point set.

[0191] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the third screening sub-unit includes: a fourth determination module, used to determine a coarse registration point set corresponding to the coarse registration from the candidate points in the candidate point set based on the selection indication of the first object, wherein the candidate point set is the acetabulum side candidate point set, or the femoral side candidate point set; a clustering module, used to cluster the candidate points in the candidate point set to obtain a fine registration point set corresponding to the fine registration; a fifth determination module, used to obtain a registration point set based on the coarse registration point set and the fine registration point set.

[0192] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the clustering module includes: a clustering submodule, used to cluster the candidate points in the candidate point set to obtain multiple candidate point subsets; a first calculation submodule, used to calculate the spatial volume of each candidate point subset to obtain multiple spatial volume values; a second calculation submodule, used to calculate the sampling ratio of each candidate point subset based on the multiple spatial volume values ​​to obtain multiple ratio values; a sampling submodule, used to randomly sample multiple candidate point subsets based on the multiple ratio values ​​and the required number of precise alignment points to obtain a precise alignment point set.

[0193] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the determination unit includes: a first registration subunit, used to perform coarse registration on a coarse registration point set in the registration point set and a first acquisition point set corresponding to the coarse registration point set to obtain a coarse registration affine matrix, wherein the acquisition points in the first acquisition point set are obtained from the second object; a second registration subunit, used to perform, based on the coarse registration affine matrix, a fine registration point set in the registration point set and a second acquisition point set corresponding to the fine registration point set to obtain a fine registration affine matrix, wherein the acquisition points in the second acquisition point set are obtained from the second object; a third determination subunit, used to obtain a target affine matrix based on the fine registration affine matrix and the coarse registration affine matrix.

[0194] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the first registration subunit includes: a sorting module, which is used to sort the coarse registration points in the coarse registration point set to obtain sorted coarse registration points, and sort the first acquisition points in the first acquisition point set to obtain sorted first acquisition points; a sixth determination module, which is used to determine the first vector and the second vector based on the connecting line between the sorted coarse registration points, and determine the third vector and the fourth vector based on the connecting line between the sorted first acquisition points; a seventh determination module, which is used to obtain a target rotation matrix based on the first vector, the second vector, the third vector and the fourth vector; a processing module, which is used to rotate the first acquisition point based on the target rotation matrix to obtain the processed first acquisition point; an eighth determination module, which is used to obtain a translation matrix based on the distance between the processed first acquisition point and the coarse registration point; and a ninth determination module, which is used to obtain a coarse registration affine matrix based on the target rotation matrix and the translation matrix.

[0195] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the second registration subunit includes: a perturbation module, which is used to apply random perturbations to the coarse registration affine matrix based on a normal distribution to obtain multiple first affine matrices; an optimization module, which is used to optimize the multiple first affine matrices through a data matching algorithm to obtain multiple second affine matrices; a merging module, which is used to merge similar affine matrices in the multiple second affine matrices when similar affine matrices exist in the multiple second affine matrices to obtain multiple third affine matrices; and a tenth determination module, which is used to obtain a fine registration affine matrix based on the multiple third affine matrices and a set of fine registration points.

[0196] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the tenth determination module includes: a first mapping submodule, used to map the second acquisition point in the second acquisition point set based on the coarse registration affine matrix to obtain a third acquisition point set; a second mapping submodule, used to map the third acquisition point in the third acquisition point set based on each third affine matrix to obtain a fourth acquisition point set corresponding to each third affine matrix; a third calculation submodule, used to calculate the root mean square error of the distance between the fourth acquisition point in the fourth acquisition point set and the registration point in the fine registration point set to obtain multiple root mean square error sets; a determination submodule, used to determine the third affine matrix corresponding to the target root mean square error set as the fine registration affine matrix if the error in the target root mean square error set in the multiple root mean square error sets is less than the error threshold.

[0197] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the device also includes: a second determination unit, used to determine that one iteration is completed and judge whether the number of iterations is greater than or equal to a preset number if there is no target root mean square error set in multiple root mean square error sets whose error is less than an error threshold; an execution unit, used to repeatedly execute the step of applying random perturbations to the third affine matrix with the smallest error of the root mean square error set based on a normal distribution if the number of iterations is less than a preset number, until the number of iterations is greater than or equal to the preset number, or the root mean square error set corresponding to the current affine matrix is ​​less than the error threshold.

[0198] Optionally, in the hip joint data processing device provided in the embodiment of the present application, the device also includes: a judgment unit, used to judge whether it is necessary to filter the set of precise alignment points after judging whether the number of iterations is greater than or equal to a preset number of times; a screening unit, used to filter the set of precise alignment points based on preset conditions when it is necessary to filter the set of precise alignment points, to obtain a filtered set of precise alignment points, wherein the next iteration process is executed based on the filtered set of precise alignment points.

[0199] It should be noted that the above-mentioned first acquisition unit 1001, first processing unit 1002, reconstruction unit 1003 and first determination unit 1004 correspond to steps S201 to S204 in the first embodiment, and the four units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the first embodiment. It should be noted that the above-mentioned modules or units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n), and the above-mentioned modules can also be run in the computer terminal 10 provided in the first embodiment as part of the device.

[0200] Example 3

[0201] An embodiment of the present application may provide an electronic device, Fig.11 is a structural block diagram of an electronic device according to an embodiment of the present application. Fig.11 As shown, the electronic device may include: one or more ( Fig.11 (only one is shown) processor 1102, memory 1104, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0202] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0203] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: obtain an initial CT image of the hip joint; perform metal artifact removal processing on the initial CT image through the target network structure to obtain a target CT image and a target confidence matrix corresponding to the target CT image, wherein the target confidence matrix is ​​composed of the confidence corresponding to each voxel point in the target CT image; perform three-dimensional reconstruction based on the target CT image and the target confidence matrix to obtain a target hip joint model, and determine a set of registration points based on the target hip joint model; determine a target affine matrix based on the set of registration points, wherein the target affine matrix is ​​used to provide data reference for hip joint surgery.

[0204] The processor can call the information and application program stored in the memory through the transmission device to perform the following steps: removing metal artifacts from the initial CT image through the target network structure to obtain the target CT image and the target confidence matrix corresponding to the target CT image, including: extracting features of the initial CT image and the sinusoidal graph corresponding to the initial CT image through the sinusoidal graph enhancement network in the target network structure to obtain first feature information; processing the first feature information through the image enhancement network in the target network structure to obtain the target CT image and the target confidence matrix.

[0205] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: processing the first feature information through the image enhancement network in the target network structure to obtain the target CT image and the target confidence matrix, including: downsampling the first feature information through multiple downsampling layers of the image enhancement network to obtain the second feature information corresponding to each downsampling layer; upsampling the second feature information corresponding to each downsampling layer through multiple first upsampling layers of the image enhancement network to obtain the target CT image; processing the second feature information corresponding to each downsampling layer and the output information of the target first upsampling layer through multiple second upsampling layers of the image enhancement network to obtain the target confidence matrix, wherein the target first upsampling layer is a sampling layer among the multiple first upsampling layers.

[0206] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: the target network structure is trained by the following steps: a training sample set is obtained, wherein the training sample set includes: a sample CT image, a real CT image corresponding to the sample CT image, and a real confidence matrix corresponding to the real CT image; the training sample set is processed through the initial network structure to obtain a predicted CT image and a predicted confidence matrix corresponding to the predicted CT image; the initial network structure is iterated according to the predicted CT image and the predicted confidence matrix to obtain the target network structure.

[0207] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: iterate the initial network structure according to the predicted CT image and the predicted confidence matrix to obtain the target network structure, including: calculate the loss according to the predicted CT image and the real CT image to obtain the first loss function; calculate the loss according to the predicted confidence matrix and the real confidence matrix to obtain the second loss function; calculate according to the first loss function, the second loss function and the target weight to obtain the target loss function, wherein the target weight is calculated based on the first loss function and the second loss function; iterate the initial network structure according to the target loss function to obtain the target network structure.

[0208] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtaining the training sample set includes: obtaining a real CT image, and processing the real CT image to obtain a sample CT image; calculating the difference between each voxel point in the real CT image and each voxel point in the sample CT image to obtain a single voxel difference set; obtaining a real confidence matrix based on the single voxel difference set; obtaining a training sample set based on the sample CT image, the real CT image and the real confidence matrix.

[0209] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: performing three-dimensional reconstruction based on the target CT image and the target confidence matrix to obtain the target hip joint model, including: performing three-dimensional reconstruction based on the target CT image to obtain the initial hip joint model; performing confidence mapping on the initial hip joint model based on the target confidence matrix to obtain the target hip joint model.

[0210] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: According to the target hip joint model, determine the alignment point set including: screening the acetabulum side of the target hip joint model to obtain the acetabulum side candidate point set; screening the femoral side of the target hip joint model to obtain the femoral side candidate point set; screening based on the acetabulum side candidate point set and the femoral side candidate point set to obtain the alignment point set.

[0211] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: screen the acetabulum side of the target hip joint model to obtain a set of candidate points on the acetabulum side, including: determining the direction vector corresponding to the center point of the acetabular cup and the implantation direction of the acetabular cup based on the target hip joint model; determining the normal vector corresponding to the first screening point within the target range centered at the center point of the acetabular cup, and calculating the angle value between the normal vector and the direction vector to obtain the target angle value; screening the first screening point within the target range based on the target angle value and the confidence corresponding to the first screening point to obtain a set of candidate points on the acetabulum side.

[0212] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: screen the femoral side of the target hip joint model to obtain a set of candidate points on the femoral side, including: determining the target screening area based on the innermost point of the femoral stem, the outermost point of the femoral side, the apex of the greater trochanter and the bottom point of the lesser trochanter in the target hip joint model; judging whether the second screening point in the target screening area belongs to the femoral area to obtain a first judgment result; judging whether the confidence corresponding to the second screening point is greater than or equal to a preset threshold to obtain a second judgment result; screening the second screening point based on the first judgment result and the second judgment result to obtain a set of candidate points on the femoral side.

[0213] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: screening according to the acetabulum side candidate point set and the femoral side candidate point set to obtain the registration point set including: determining a coarse registration point set corresponding to the coarse registration from the candidate points in the candidate point set based on the selection indication of the first object, wherein the candidate point set is the acetabulum side candidate point set, or the femoral side candidate point set; clustering the candidate points in the candidate point set to obtain a fine registration point set corresponding to the fine registration; obtaining the registration point set based on the coarse registration point set and the fine registration point set.

[0214] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: clustering the candidate points in the candidate point set to obtain a set of precise alignment points corresponding to the precise alignment, including: clustering the candidate points in the candidate point set to obtain multiple candidate point subsets; calculating the spatial volume of each candidate point subset to obtain multiple spatial volume values; calculating the sampling ratio of each candidate point subset based on the multiple spatial volume values ​​to obtain multiple ratio values; randomly sampling multiple candidate point subsets based on the multiple ratio values ​​and the required number of precise alignment points to obtain a set of precise alignment points.

[0215] The processor can call the information and application program stored in the memory through the transmission device to perform the following steps: determining the target affine matrix according to the registration point set includes: roughly registering a coarse registration point set in the registration point set with a first acquisition point set corresponding to the coarse registration point set to obtain a coarse registration affine matrix, wherein the acquisition points in the first acquisition point set are obtained from the second object; based on the coarse registration affine matrix, registering a fine registration point set in the registration point set with a second acquisition point set corresponding to the fine registration point set to obtain a fine registration affine matrix, wherein the acquisition points in the second acquisition point set are obtained from the second object; and obtaining the target affine matrix based on the fine registration affine matrix and the coarse registration affine matrix.

[0216] The processor can call the information and application program stored in the memory through the transmission device to perform the following steps: roughly aligning the coarse registration point set in the registration point set and the first acquisition point set corresponding to the coarse registration point set to obtain a coarse registration affine matrix, including: sorting the coarse registration points in the coarse registration point set to obtain sorted coarse registration points, and sorting the first acquisition points in the first acquisition point set to obtain sorted first acquisition points; determining the first vector and the second vector based on the connection between the sorted coarse registration points, and determining the third vector and the fourth vector based on the connection between the sorted first acquisition points; obtaining a target rotation matrix based on the first vector, the second vector, the third vector and the fourth vector; performing rotation processing on the first acquisition point based on the target rotation matrix to obtain a processed first acquisition point; obtaining a translation matrix based on the distance between the processed first acquisition point and the coarse registration point; and obtaining a coarse registration affine matrix based on the target rotation matrix and the translation matrix.

[0217] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: based on the coarse registration affine matrix, the fine registration point set in the registration point set and the second acquisition point set corresponding to the fine registration point set are registered to obtain the fine registration affine matrix, including: applying random perturbations to the coarse registration affine matrix based on the normal distribution to obtain multiple first affine matrices; optimizing the multiple first affine matrices through the data matching algorithm to obtain multiple second affine matrices; when similar affine matrices exist in the multiple second affine matrices, merging the similar affine matrices in the multiple second affine matrices to obtain multiple third affine matrices; obtaining the fine registration affine matrix based on the multiple third affine matrices and the fine registration point set.

[0218] The processor can call the information and application program stored in the memory through the transmission device to perform the following steps: obtaining the fine registration affine matrix according to multiple third affine matrices and fine registration point sets includes: mapping the second acquisition points in the second acquisition point set based on the coarse registration affine matrix to obtain the third acquisition point set; mapping the third acquisition points in the third acquisition point set based on each third affine matrix to obtain the fourth acquisition point set corresponding to each third affine matrix; calculating the root mean square error of the distance between the fourth acquisition point in the fourth acquisition point set and the registration point in the fine registration point set to obtain multiple root mean square error sets; if there is an error in the target root mean square error set in the multiple root mean square error sets that is less than the error threshold, then determining the third affine matrix corresponding to the target root mean square error set as the fine registration affine matrix.

[0219] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: the method also includes: if there is no target root mean square error set in the multiple root mean square error sets whose error is less than the error threshold, it is determined that one iteration is completed, and it is judged whether the number of iterations is greater than or equal to the preset number; if the number of iterations is less than the preset number, the step of applying random perturbations to the third affine matrix with the smallest error of the root mean square error set based on the normal distribution is repeated until the number of iterations is greater than or equal to the preset number, or the root mean square error set corresponding to the current affine matrix is ​​less than the error threshold.

[0220] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: after determining whether the number of iterations is greater than or equal to the preset number of times, the method also includes: determining whether it is necessary to filter the set of precise alignment points; if it is necessary to filter the set of precise alignment points, filter the set of precise alignment points based on preset conditions to obtain a filtered set of precise alignment points, wherein the next iteration process is performed based on the filtered set of precise alignment points.

[0221] It can be understood by those skilled in the art that Fig.11 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Fig.11 The structure of the electronic device is not limited. Fig.11 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Fig.11 Different configurations are shown.

[0222] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0223] Example 4

[0224] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the hip joint data processing method provided in the first embodiment.

[0225] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0226] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the steps of the hip joint data processing method.

[0227] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0228] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0229] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0230] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0231] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0232] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0233] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A hip joint data processing method, characterized in that: include: Obtain initial CT images of the hip joint; Performing metal artifact removal processing on the initial CT image through a target network structure to obtain a target CT image and a target confidence matrix corresponding to the target CT image, wherein the target confidence matrix is ​​composed of confidences corresponding to each voxel point in the target CT image; Performing three-dimensional reconstruction based on the target CT image and the target confidence matrix to obtain a target hip joint model, and determining a registration point set based on the target hip joint model; A target affine matrix is ​​determined based on the set of registration points, wherein the target affine matrix is ​​used to provide data reference for hip joint surgery.

2. The method according to claim 1, characterized in that The initial CT image is processed by removing metal artifacts through the target network structure to obtain a target CT image and a target confidence matrix corresponding to the target CT image, including: Extracting features of the initial CT image and the sinogram corresponding to the initial CT image through the sinogram enhancement network in the target network structure to obtain first feature information; The first feature information is processed by an image enhancement network in the target network structure to obtain the target CT image and the target confidence matrix.

3. The method according to claim 2, characterized in that Processing the first feature information through the image enhancement network in the target network structure to obtain the target CT image and the target confidence matrix includes: Downsampling the first feature information through multiple downsampling layers of the image enhancement network to obtain second feature information corresponding to each downsampling layer; Performing upsampling processing on the second feature information corresponding to each downsampling layer through the multiple first upsampling layers of the image enhancement network to obtain the target CT image; The target confidence matrix is ​​obtained by processing the second feature information corresponding to each downsampling layer and the output information of the target first upsampling layer through multiple second upsampling layers of the image enhancement network, wherein the target first upsampling layer is a sampling layer among the multiple first upsampling layers.

4. The method according to claim 1, characterized in that The target network structure is obtained by training using the following steps: Acquire a training sample set, wherein the training sample set includes: a sample CT image, a real CT image corresponding to the sample CT image, and a real confidence matrix corresponding to the real CT image; Processing the training sample set through an initial network structure to obtain a predicted CT image and a prediction confidence matrix corresponding to the predicted CT image; The initial network structure is iterated according to the predicted CT image and the predicted confidence matrix to obtain the target network structure.

5. The method according to claim 4, characterized in that Iterating the initial network structure according to the predicted CT image and the predicted confidence matrix to obtain the target network structure includes: Performing loss calculation based on the predicted CT image and the real CT image to obtain a first loss function; Performing loss calculation based on the predicted confidence matrix and the true confidence matrix to obtain a second loss function; Calculating according to the first loss function, the second loss function and a target weight to obtain a target loss function, wherein the target weight is calculated based on the first loss function and the second loss function; The initial network structure is iterated according to the target loss function to obtain the target network structure.

6. The method according to claim 4, characterized in that Obtaining a training sample set includes: Acquiring the real CT image, and processing the real CT image to obtain the sample CT image; Calculating the difference between each voxel point in the real CT image and each voxel point in the sample CT image to obtain a single voxel difference set; Obtaining the true confidence matrix according to the single pixel difference set; The training sample set is obtained according to the sample CT image, the real CT image and the real confidence matrix.

7. The method according to claim 1, characterized in that Performing three-dimensional reconstruction based on the target CT image and the target confidence matrix to obtain a target hip joint model includes: Performing three-dimensional reconstruction based on the target CT image to obtain an initial hip joint model; Confidence mapping is performed on the initial hip joint model according to the target confidence matrix to obtain the target hip joint model.

8. The method according to claim 1, characterized in that According to the target hip joint model, determining the registration point set includes: Screening the acetabulum side of the target hip joint model to obtain a set of candidate points on the acetabulum side; Screening the femoral side of the target hip joint model to obtain a femoral side candidate point set; The registration point set is obtained by screening the acetabulum side candidate point set and the femoral side candidate point set.

9. The method according to claim 8, characterized in that The acetabulum side of the target hip joint model is screened to obtain a set of candidate points on the acetabulum side including: According to the target hip joint model, determining the direction vector corresponding to the acetabular cup center point and the acetabular cup implantation direction; Determine a normal vector corresponding to a first screening point within a target range centered at the center point of the acetabular cup, and calculate an angle value between the normal vector and the direction vector to obtain a target angle value; The first screening points within the target range are screened according to the target angle value and the confidence level corresponding to the first screening points to obtain the acetabulum side candidate point set.

10. The method according to claim 8, characterized in that The femoral side of the target hip joint model is screened to obtain a femoral side candidate point set including: Determine a target screening area based on the innermost point of the femoral stem, the outermost point of the femoral side, the apex of the greater trochanter and the bottom point of the lesser trochanter in the target hip joint model; Determine whether the second screening point in the target screening area belongs to the femur area, and obtain a first determination result; Determine whether the confidence level corresponding to the second screening point is greater than or equal to a preset threshold, and obtain a second determination result; The second screening points are screened according to the first judgment result and the second judgment result to obtain the femoral side candidate point set.

11. The method according to claim 8, characterized in that The registration point set is obtained by screening according to the acetabulum side candidate point set and the femoral side candidate point set, and includes: Determine a coarse registration point set corresponding to the coarse registration from the candidate points in the candidate point set based on the selection indication of the first object, wherein the candidate point set is the acetabulum side candidate point set or the femoral side candidate point set; Performing clustering processing on the candidate points in the candidate point set to obtain a precise registration point set corresponding to the precise registration; The registration point set is obtained according to the coarse registration point set and the fine registration point set.

12. The method according to claim 11, characterized in that The candidate points in the candidate point set are clustered to obtain a set of precise registration points corresponding to the precise registration, including: Performing clustering processing on the candidate points in the candidate point set to obtain multiple candidate point subsets; Calculate the spatial volume of each candidate point subset to obtain multiple spatial volume values; Calculating the sampling ratio of each candidate point subset according to the multiple spatial volume values ​​to obtain multiple ratio values; The multiple candidate point subsets are randomly sampled according to the multiple ratio values ​​and the required number of precise alignment points to obtain the precise alignment point set.

13. The method according to claim 12, characterized in that Determining the target affine matrix according to the set of registration points includes: Performing rough registration on a coarse registration point set in the registration point set and a first collection point set corresponding to the coarse registration point set to obtain a coarse registration affine matrix, wherein the collection points in the first collection point set are obtained from the second object; Based on the coarse registration affine matrix, a fine registration point set in the registration point set is registered with a second collection point set corresponding to the fine registration point set to obtain a fine registration affine matrix, wherein the collection points in the second collection point set are obtained from a second object; The target affine matrix is ​​obtained based on the fine registration affine matrix and the coarse registration affine matrix.

14. The method according to claim 13, characterized in that The coarse registration point set in the registration point set and the first acquisition point set corresponding to the coarse registration point set are coarsely registered to obtain a coarse registration affine matrix, which includes: sorting the coarse registration points in the coarse registration point set to obtain sorted coarse registration points, and sorting the first acquisition point in the first acquisition point set to obtain the sorted first acquisition point; Determine a first vector and a second vector based on a connection line between the sorted coarse registration points, and determine a third vector and a fourth vector based on a connection line between the sorted first acquisition points; Obtaining a target rotation matrix according to the first vector, the second vector, the third vector and the fourth vector; Rotating the first acquisition point according to the target rotation matrix to obtain a processed first acquisition point; Obtaining a translation matrix according to the distance between the processed first acquisition point and the coarse registration point; The coarse registration affine matrix is ​​obtained according to the target rotation matrix and the translation matrix.

15. The method according to claim 13, characterized in that Based on the coarse registration affine matrix, the fine registration point set in the registration point set is registered with the second acquisition point set corresponding to the fine registration point set, and the fine registration affine matrix is ​​obtained, which includes: Applying random perturbations to the coarse registration affine matrix based on normal distribution to obtain a plurality of first affine matrices; Optimizing the multiple first affine matrices by a data matching algorithm to obtain multiple second affine matrices; When similar affine matrices exist in the plurality of second affine matrices, merging the similar affine matrices in the plurality of second affine matrices to obtain a plurality of third affine matrices; The precise registration affine matrix is ​​obtained according to the plurality of third affine matrices and the precise registration point set.

16. The method according to claim 15, characterized in that Obtaining the precise registration affine matrix according to the plurality of third affine matrices and the precise registration point set comprises: Mapping the second collection points in the second collection point set based on the coarse registration affine matrix to obtain a third collection point set; Mapping the third acquisition points in the third acquisition point set based on each third affine matrix to obtain a fourth acquisition point set corresponding to each third affine matrix; Calculating a root mean square error of a distance between a fourth acquisition point in the fourth acquisition point set and a registration point in the precise registration point set to obtain a plurality of root mean square error sets; If an error in a target root mean square error set exists in the multiple root mean square error sets and is smaller than an error threshold, a third affine matrix corresponding to the target root mean square error set is determined as the fine registration affine matrix.

17. The method according to claim 16, characterized in that The method further comprises: If there is no target root mean square error set in the multiple root mean square error sets whose error is less than the error threshold, it is determined that one iteration is completed, and whether the number of iterations is greater than or equal to a preset number; If the number of iterations is less than the preset number, the step of applying random perturbations to the third affine matrix with the smallest error in the root mean square error set based on normal distribution is repeated until the number of iterations is greater than or equal to the preset number, or the root mean square error set corresponding to the current affine matrix is ​​less than the error threshold.

18. The method according to claim 17, characterized in that After determining whether the number of iterations is greater than or equal to a preset number, the method further includes: Determining whether the set of precise registration points needs to be screened; When the precise alignment point set needs to be screened, the precise alignment point set is screened based on preset conditions to obtain a screened precise alignment point set, wherein the next iteration process is performed based on the screened precise alignment point set.

19. A hip joint data processing device, characterized in that: include: A first acquisition unit is used to acquire an initial CT image of the hip joint; A first processing unit is used to perform metal artifact removal processing on the initial CT image through a target network structure to obtain a target CT image and a target confidence matrix corresponding to the target CT image, wherein the target confidence matrix is ​​composed of confidences corresponding to each voxel point in the target CT image; A reconstruction unit, configured to perform three-dimensional reconstruction based on the target CT image and the target confidence matrix to obtain a target hip joint model, and determine a registration point set based on the target hip joint model; The first determination unit is used to determine a target affine matrix based on the set of registration points, wherein the target affine matrix is ​​used to provide data reference for hip joint surgery.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute the hip joint data processing method described in any one of claims 1 to 18.

21. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the program executes the hip joint data processing method described in any one of claims 1 to 18 when running.

Images