Key point labeling method and device, equipment and storage medium

Abstract

The invention relates to a key point labeling method and device, equipment and a storage medium. The method comprises the following steps: acquiring a three-dimensional tissue model of a target body tissue; projecting the surface of the three-dimensional tissue model to a preset projection plane to obtain a two-dimensional surface image, and displaying the two-dimensional surface image; in response to a key point triggering operation of a user on the two-dimensional surface image, determining two-dimensional coordinates of the surface key points on the two-dimensional surface image; converting the two-dimensional coordinates from the two-dimensional surface image to a three-dimensional tissue model to obtain three-dimensional coordinates; wherein the three-dimensional coordinates are corresponding seats of the surface key points on the surface of the three-dimensional tissue model. Therefore, the requirement for anatomical knowledge of the user is lowered, the difficulty of key point labeling of the body tissue surface by the user is lowered, the user can quickly perform key point labeling, and the accuracy of characterizing the corresponding key points by the three-dimensional coordinates finally determined through coordinate conversion is also improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and particularly to a key point annotation method, apparatus, device, and storage medium. Background Art

[0002] With the development of software technologies, key point annotation has become increasingly important. Based on the annotated key points, functions such as surgical navigation can be realized. In related technologies, key points are selected on the three-view sectional drawings generated by Multi-Planar Reconstruction (MPR). However, the key point selection method based on sectional drawings has poor intuitiveness, and users who need to select key points require high anatomical knowledge, resulting in low accuracy of the finally determined key points. Summary of the Invention

[0003] To solve the above technical problems, the present disclosure provides a key point annotation method, apparatus, device, and storage medium.

[0004] In a first aspect, the present disclosure provides a key point annotation method, which includes:

[0005] Obtain a three-dimensional tissue model of a target body tissue;

[0006] Project the surface of the three-dimensional tissue model onto a preset projection plane to obtain a two-dimensional surface image, and display the two-dimensional surface image;

[0007] In response to a user's key point trigger operation on the two-dimensional surface image, determine the two-dimensional coordinates of the surface key point on the two-dimensional surface image;

[0008] Convert the two-dimensional coordinates from the two-dimensional surface image to the three-dimensional tissue model to obtain three-dimensional coordinates; wherein, the three-dimensional coordinates are the coordinates corresponding to the surface key point on the surface of the three-dimensional tissue model.

[0009] In a second aspect, the present disclosure provides a key point annotation apparatus, which includes:

[0010] An obtaining module, configured to obtain a three-dimensional tissue model of a target body tissue;

[0011] A projection module, configured to project the surface of the three-dimensional tissue model onto a preset projection plane to obtain a two-dimensional surface image, and display the two-dimensional surface image;

[0012] A trigger module, configured to, in response to a user's key point trigger operation on the two-dimensional surface image, determine the two-dimensional coordinates of the surface key point on the two-dimensional surface image;

[0013] A conversion module for converting the two-dimensional coordinates from the two-dimensional surface image to the three-dimensional tissue model to obtain three-dimensional coordinates, where the three-dimensional coordinates are the coordinates corresponding to the surface key points on the surface of the three-dimensional tissue model.

[0014] In a third aspect, an embodiment of the present disclosure further provides an electronic device, which includes:

[0015] One or more processors;

[0016] A storage device for storing one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect.

[0018] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method provided in the first aspect is implemented.

[0019] The technical solutions provided by the embodiments of the present disclosure have the following advantages compared with the prior art:

[0020] A key point annotation method, device, device and storage medium according to an embodiment of the present disclosure obtain a three-dimensional tissue model of a target body tissue; project the surface of the three-dimensional tissue model onto a preset projection plane to obtain a two-dimensional surface image and display the two-dimensional surface image; in response to a user's key point trigger operation on the two-dimensional surface image, determine the two-dimensional coordinates of the surface key points on the two-dimensional surface image; convert the two-dimensional coordinates from the two-dimensional surface image to the three-dimensional tissue model to obtain three-dimensional coordinates, where the three-dimensional coordinates are the coordinates corresponding to the surface key points on the surface of the three-dimensional tissue model. Thus, by projecting the surface of the three-dimensional tissue model into a two-dimensional surface image and displaying the two-dimensional surface image, determining the coordinates of the key points on the two-dimensional surface image according to the user's key point trigger operation, and performing a two-dimensional to three-dimensional conversion of the coordinates to obtain the coordinates corresponding to the surface of the three-dimensional tissue model, the user views a two-dimensional image of the body tissue surface through the projection of the three-dimensional model surface, reducing the requirement for the user's anatomical knowledge, reducing the difficulty of the user in performing key point annotation on the body tissue surface, enabling the user to quickly get started with key point annotation, and also improving the accuracy of the three-dimensional coordinates finally determined through coordinate conversion in representing the corresponding key points. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0022] To more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0023] Figure 1 A flowchart of a key point annotation method provided by an embodiment of the present disclosure;

[0024] Figure 2 A schematic diagram of a three-dimensional tissue model provided by an embodiment of the present disclosure;

[0025] Figure 3 A flowchart of another key point annotation method provided in an embodiment of the present disclosure;

[0026] Figure 4 A schematic diagram of a three-dimensional isosurface model provided by an embodiment of the present disclosure;

[0027] Figure 5 A schematic diagram of another three-dimensional isosurface model provided by an embodiment of the present disclosure;

[0028] Figure 6 A schematic diagram of determining a two-dimensional surface image provided by an embodiment of the present disclosure;

[0029] Figure 7 A flowchart of yet another key point annotation method provided by an embodiment of the present disclosure;

[0030] Figure 8 A schematic diagram of a decomposition block provided by an embodiment of the present disclosure;

[0031] Figure 9 A schematic diagram of the structure of a key point annotation device provided by an embodiment of the present disclosure;

[0032] Figure 10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0033] In order to more clearly understand the above objects, features and advantages of the present disclosure, the following will further describe the solutions of the present disclosure. It should be noted that, without conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

[0034] With the development of software technology, key point annotation has become increasingly important. Based on the annotated key points, functions such as surgical navigation can be realized. In related technologies, in medical software, images collected of tissues within the imaging scan range can be superimposed through multi-planar reconstruction technology, and then sectional images of the tissues within the imaging scan range can be reconstructed from any other angular directions such as the coronal plane and the sagittal plane. For example, the reconstructed image can be the three-view sectional image of the tissue. After that, the user manually selects the anatomical key points on this three-view sectional image, and this sectional image is also called a sectional plane image.

[0035] In related technologies, since both the interactive screen space and the views provided by multi-planar reconstruction technology are two-dimensional spaces, and the depth information of the tissue can be reflected by the number of layers in multi-planar reconstruction technology, there is no information loss during the interaction. However, the intuitiveness of this key point selection method based on sectional views is poor, and users need to have relatively high anatomical knowledge. In addition, the femoral ridge is the position where the posterior part of the femoral shaft bulges. The anatomical structures of key points such as the femoral ridge are not obvious and are not easily identifiable on the three anatomical views. Therefore, the accuracy of the finally determined key points is also relatively low.

[0036] To solve the above problems, the embodiments of the present disclosure provide a key point annotation method, which will be introduced below in combination with specific embodiments.

[0037] Figure 1 As shown in the flowchart of a key point annotation method provided by the embodiments of the present disclosure, this key point annotation method can be applied to a key point annotation device. This key point annotation device can be applied to fields such as computer software, medical image processing, and orthopedic surgical robot navigation. This model determination device can be implemented by software and / or hardware, and this model determination device is generally integrated in an electronic device. This electronic device can include devices such as tablet computers, desktop computers, and laptop computers.

[0038] As Figure 1 shown, this key point annotation method can include the following steps.

[0039] Step 101, obtain a three-dimensional tissue model of the target body tissue.

[0040] Among them, the target body tissue can be the body tissue to be annotated with key points. The body tissue can be the tissue that constitutes a biological body. The type of this target body tissue is not limited in this embodiment. For example, the type of this target body tissue can include bones, muscles, etc. Specifically, the target body tissue can be the femur. The three-dimensional tissue model can be the three-dimensional model of the target body tissue.

[0041] In this embodiment, the key point annotation method can be applied to a surgical navigation system. For example, if the surface key point is the femoral crest point, the key point annotation method can be used in orthopedic surgical navigation systems such as anterior cruciate ligament reconstruction. In the embodiments of the present disclosure, the three-dimensional tissue model of the target body tissue can be pre-determined, and the key point annotation device can obtain the three-dimensional tissue model. Alternatively, the key point annotation device can segment the three-dimensional tissue model of the target body tissue from the overall three-dimensional model by means of medical image segmentation.

[0042] In some embodiments of the present disclosure, obtaining the three-dimensional tissue model of the target body tissue includes: obtaining an initial three-dimensional model of the target body part; wherein, the initial three-dimensional model is generated based on the computed tomography (CT) images of the target body part; and inputting the initial three-dimensional model into a preset tissue segmentation model to obtain the three-dimensional tissue model corresponding to the target body tissue in the target body part.

[0043] Wherein, the target body part can be a body part containing the target body tissue. The body part can be a body part of a biological body. The initial three-dimensional model can be the three-dimensional model corresponding to the entire target body part. The preset tissue segmentation model can be a pre-set neural network model with tissue segmentation function, and the preset tissue segmentation model can be used to separate the foreground region of interest based on image processing algorithms, so as to facilitate subsequent processing of the foreground region alone.

[0044] In this embodiment, the initial three-dimensional model of the target body part can be pre-determined, and the key point annotation device can obtain the initial three-dimensional model. Alternatively, by performing computed tomography (CT) scans on different depths of the target body part in advance, a plurality of computed tomography images corresponding to different depths are obtained. The plurality of computed tomography images can be understood as an image sequence, and the key point annotation device can perform three-dimensional imaging on the plurality of computed tomography images to obtain the initial three-dimensional model. Further, the key point annotation device can input the initial three-dimensional model into the trained preset tissue segmentation model, and segment the three-dimensional tissue model representing the target body tissue from the target body part represented by the initial three-dimensional model through the preset tissue segmentation model. Figure 2 A schematic diagram of a three-dimensional tissue model provided by the embodiments of the present disclosure is as Figure 2 shown. If the target body part is the thigh and the target body tissue is the femur, the three-dimensional tissue model is the segmented three-dimensional model of the femur.

[0045] In the above solution, three-dimensional modeling is performed based on the computed tomography (CT) image of the entire target body part, and image segmentation is carried out to obtain a three-dimensional tissue model corresponding to the target body tissue. The three-dimensional models of other tissues that affect key point annotation in the target body part are filtered out, creating a basis for presenting the model corresponding to the surface part of the target body part to the user.

[0046] Step 102: Project the surface of the three-dimensional tissue model onto a preset projection plane to obtain a two-dimensional surface image, and display the two-dimensional surface image.

[0047] Among them, the preset projection plane can be a plane that is pre-set to receive the image of the object to be projected. In this embodiment, the position of the preset projection plane is not limited. The two-dimensional surface image can be a two-dimensional image obtained by projecting the surface of the three-dimensional tissue model onto the preset projection plane. This two-dimensional surface image can be used to depict the situation of the surface of the three-dimensional tissue model in the two-dimensional space corresponding to the preset projection plane.

[0048] In the embodiments of the present disclosure, the key point annotation device can extract the surface of the three-dimensional tissue model, project the surface onto the preset projection plane to obtain a two-dimensional surface image located on the preset projection plane, and display the two-dimensional surface image to the user.

[0049] Figure 3 As shown in the flowchart of another key point annotation method provided in the embodiments of the present disclosure, Figure 3 In some embodiments of the present disclosure, projecting the surface of the three-dimensional tissue model onto a preset projection plane to obtain a two-dimensional surface image includes:

[0050] Step 301: Perform an isosurface extraction process on the three-dimensional tissue model to obtain a three-dimensional isosurface model.

[0051] Among them, the isosurface can be a surface that approximately represents the subdivided surface of the three-dimensional tissue model. The isosurface can be a surface composed of points with equal function values of a pre-set scalar function. The isosurface can be a triangular patch. The isosurface can be used to construct a three-dimensional model for representing the surface of the body tissue. In this embodiment, the extraction method of the isosurface is not limited. The isosurface can be determined based on a pre-set graphics surface rendering algorithm. The graphics surface rendering algorithm can extract the isosurface from the three-dimensional scalar data field according to the set threshold. The three-dimensional isosurface model can be a model corresponding to the surface of the three-dimensional tissue model. The three-dimensional isosurface model can be topological data used to describe the point coordinates of the surface representing the three-dimensional tissue model and the spatial connection relationship between the point coordinates.

[0052] In this embodiment, the key point annotation device can extract the isosurface corresponding to the three-dimensional tissue model in the three-dimensional space according to the pixel values of the three-dimensional tissue model through a graphics surface rendering algorithm, and approximately represent the isosurface with triangular faces to obtain a three-dimensional isosurface model.

[0053] In some embodiments of the present disclosure, the three-dimensional isosurface can be realized based on volume rendering three-dimensional rendering. Specifically, performing isosurface extraction processing on the three-dimensional tissue model to obtain a three-dimensional isosurface model includes:

[0054] Determining the array points located inside the three-dimensional tissue model in the array points as the first array points, and determining the array points located outside the three-dimensional tissue model in the array points as the second array points; wherein, the array points are equally spaced points in the three-dimensional space where the three-dimensional tissue model is located; determining array point pairs; wherein, an array point pair includes an adjacent first array point and a second array point; constructing an isosurface according to the array point pairs, and merging the isosurfaces to obtain a three-dimensional isosurface model.

[0055] Among them, the array points can be multiple points regularly arranged at equal intervals in the three-dimensional space where the three-dimensional tissue model is located. The array points can be the vertices of voxels in the three-dimensional space. A voxel can be the smallest volume unit in the three-dimensional space and is the basic unit voxel that constitutes the model. The voxel is also called a volume pixel. The first array point can be an array point included inside the three-dimensional tissue model. The second array point can be an array point located outside the three-dimensional tissue model. An array point pair can be a point pair composed of an adjacent first array point and a second array point in a voxel.

[0056] In the embodiments of the present disclosure, the key point annotation device can determine each voxel in the data field where the three-dimensional tissue model is located, and obtain the scalar values of the 8 vertices of each voxel. Comparing the scalar value with a preset scalar threshold, and representing the magnitude relationship between the scalar value and the scalar threshold with a binary number. Since a voxel has 8 vertices, an 8-bit binary number can be constructed for each voxel to represent the state of the voxel, and then determine the first array points located inside the three-dimensional tissue model and the second array points located outside the three-dimensional tissue model in each voxel.

[0057] Determining the adjacent first array point and second array point in the voxel as an array point pair, and determining the vertex positions, spatial connection relationships, and the number of triangular patches (i.e., isosurfaces) of each triangular patch according to the array point pair. Further, merging the triangular patches determined by all voxels to form a complete polygon mesh corresponding to the three-dimensional tissue model. The polygon mesh is a three-dimensional isosurface model, and the surface extraction of the three-dimensional tissue model is completed. Taking the target body tissue as the femur as an example, Figure 4 is a schematic diagram of a three-dimensional isosurface model provided by the embodiments of the present disclosure, as Figure 4As shown, it shows the spatial points for characterizing the surface of the three-dimensional tissue model of the femur and the connection relationships between these spatial points. Figure 5 It is a schematic diagram of another three-dimensional isosurface model provided by an embodiment of the present disclosure. As Figure 5 shown, surface treatment is performed on the isosurface formed by the spatial points.

[0058] In the above solution, through the positional relationship between the array points and the three-dimensional tissue model, the three-dimensional isosurface model corresponding to the three-dimensional tissue model is determined, realizing the characterization of the surface of the tissue model.

[0059] Step 302: Project the three-dimensional isosurface model onto a preset projection plane based on a preset observation point to obtain a two-dimensional surface image.

[0060] Among them, the preset observation point can be a pre-set position for observing the three-dimensional isosurface model. Through this preset observation point, the acquisition and presentation angles of the two-dimensional surface image can be determined. This embodiment places no restrictions on the preset observation point. The three-dimensional coordinate system established based on this preset observation point can be an observation coordinate system.

[0061] In this embodiment, the key point annotation device can project the three-dimensional isosurface onto the preset projection plane based on the preset observation point through a ray projection algorithm to form a two-dimensional surface image.

[0062] The construction process of this two-dimensional surface image can include: The key point annotation device can start from the preset observation point. For each pixel point on the preset projection plane, a beam of light is emitted towards the three-dimensional isosurface model along the direction determined by the preset observation point and this pixel point. Equally spaced sampling points are set on the light ray, and the two-dimensional surface image is constructed based on the process of the light ray passing through the three-dimensional isosurface model. Specifically, for each beam of light, in the case where the light ray first intersects the three-dimensional isosurface model, at the intersection point, according to the nature of the three-dimensional isosurface model itself, light source attributes, lighting model, etc., the color of this pixel point on the preset projection plane is calculated. The determination process of this color can include: Obtaining the color data and transparency of the voxels corresponding to each sampling point on the light ray, and at the same time, according to the light absorption model, accumulating the color data and transparency of each sampling point until the light ray leaves the three-dimensional isosurface model or the transparency accumulates to non-transparent, then the sampling ends, and a two-dimensional surface image is obtained. Figure 6 It is a schematic diagram for determining a two-dimensional surface image provided by an embodiment of the present disclosure. As Figure 6 shown, the preset observation point is located at the origin of the observation point coordinates, and the three-dimensional isosurface model in the three-dimensional space is projected onto the preset projection plane to obtain a two-dimensional surface image.

[0063] In the above solution, first, an isosurface extraction is performed on the three-dimensional tissue model to obtain a three-dimensional isosurface model, and then the three-dimensional isosurface model is projected, realizing the projection of the surface of the three-dimensional tissue model, which is convenient for subsequent key point annotation based on the surface.

[0064] In some embodiments of the present disclosure, a two-dimensional surface image is displayed, including: determining a three-dimensional auxiliary model of the auxiliary body tissue in the target body part, projecting the surface of the three-dimensional auxiliary model onto a preset projection plane to obtain a two-dimensional auxiliary image; and jointly displaying the two-dimensional auxiliary image and the two-dimensional surface image.

[0065] Among them, the auxiliary body tissue can be the body tissue that helps to perform key point annotation on the target body tissue. The auxiliary body tissue can be the body tissue that has an associated relationship with the target body tissue, and the associated relationship can be a positional relationship such as mutual contact. For example, if the target body tissue is the femur, the auxiliary body tissue can be the muscle on the femur. The three-dimensional auxiliary model can be a three-dimensional model that helps to perform key point annotation on the surface of the body tissue. The two-dimensional auxiliary image can be a two-dimensional image obtained by projecting the surface of the three-dimensional auxiliary model onto a preset projection plane. The two-dimensional auxiliary image can be used to describe the situation of the three-dimensional auxiliary model in the two-dimensional space corresponding to the preset projection plane.

[0066] In this embodiment, the three-dimensional auxiliary model can be pre-determined, and the key point annotation device can obtain the three-dimensional auxiliary model. Alternatively, the key point annotation device can input the initial three-dimensional model of the target body part into a preset tissue segmentation model that has been trained, and the three-dimensional auxiliary model representing the auxiliary body tissue can be segmented from the target body part represented by the initial three-dimensional model through the preset tissue segmentation model. Further, the key point annotation device can perform isosurface extraction processing on the three-dimensional auxiliary model to obtain the surface of the three-dimensional auxiliary model, project the surface of the three-dimensional auxiliary model onto the preset projection plane based on a preset observation point to obtain a two-dimensional auxiliary image. And perform layer superposition on the two-dimensional auxiliary image and the two-dimensional surface image to obtain a superimposed image. Optionally, the two-dimensional auxiliary image in the superimposed image can be more transparent than the two-dimensional surface image, so that the user can observe the surface of the target body tissue through the auxiliary body tissue. Further, the superimposed image is displayed to the user.

[0067] Optionally, the three-dimensional auxiliary model can be the initial three-dimensional model. The key point identification device can register the initial three-dimensional model and the three-dimensional isosurface model to obtain a comprehensive model including the initial three-dimensional model and the three-dimensional isosurface model. And determine the two-dimensional surface image, two-dimensional coordinates, three-dimensional coordinates, etc. based on the three-dimensional isosurface model in the comprehensive model, determine the two-dimensional auxiliary image based on the three-dimensional isosurface model in the comprehensive model, and jointly display the two-dimensional auxiliary image and the two-dimensional surface image.

[0068] In the above solution, the two-dimensional surface image of the target body tissue in the target body part and the two-dimensional auxiliary image of the auxiliary body tissue are jointly displayed. The information contained in the two-dimensional auxiliary image can assist the user in more accurately determining the positions of the surface key points in the target body tissue.

[0069] Step 103: In response to the user's key point triggering operation on the two-dimensional surface image, determine the two-dimensional coordinates of the surface key point on the two-dimensional surface image.

[0070] Among them, the key point triggering operation can be a triggering operation for determining the key point, and the type of this key point triggering operation is not limited in this embodiment. For example, the key point triggering operation can be a manual selection operation of the user such as mouse clicking or touch screen clicking. The surface key point can be a key point located on the surface of the target body tissue. The surface key point can be a key point with surface features in the target body tissue. The anatomical structure of this kind of key point with surface features is relatively not obvious and is difficult to identify in the anatomical three-view drawings. The surface features can include bulge features, depression features, and so on. For example, if the target body tissue is the femur, the surface key point can include the bone crest point of the femur. The bone crest of the femur is the position where the back of the femoral shaft bulges. The two-dimensional coordinates can be the coordinates of the surface key point in the two-dimensional space where the two-dimensional surface image is located.

[0071] In the embodiments of the present disclosure, the two-dimensional surface image is displayed on the software interface. After the user observes the surface key points on the two-dimensional surface image, the user can perform a key point triggering operation on the two-dimensional surface image to mark the surface key points on the two-dimensional surface image. The key point marking device determines the two-dimensional coordinates of the surface key point on the two-dimensional surface image in response to the user's key point triggering operation.

[0072] Step 104: Convert the two-dimensional coordinates from the two-dimensional surface image to the three-dimensional tissue model to obtain three-dimensional coordinates; among them, the three-dimensional coordinates are the coordinates corresponding to the surface key point on the surface of the three-dimensional tissue model.

[0073] Among them, the three-dimensional coordinates can be the coordinates for characterizing the surface key point in the observation coordinate system. The three-dimensional coordinates can be the coordinates on the three-dimensional tissue model or the coordinates on the three-dimensional isosurface model that approximately represents the surface of the three-dimensional tissue model. This embodiment does not make a limitation.

[0074] In the embodiments of the present disclosure, the two-dimensional coordinates are points on the screen displayed to the user. Through spatial geometric operations, the coordinate system transformation of the two-dimensional coordinates can be realized, and the two-dimensional coordinates are mapped to the observation coordinate system where the three-dimensional tissue model is located to obtain three-dimensional coordinates corresponding to the surface of the three-dimensional tissue model.

[0075] Figure 7The flowchart of another key point annotation method provided by an embodiment of the present disclosure is shown in Figure 7 As shown, in some embodiments of the present disclosure, converting the two-dimensional coordinates from a two-dimensional surface image to a three-dimensional tissue model to obtain three-dimensional coordinates includes:

[0076] Step 701, converting the two-dimensional coordinates from the screen coordinate system to the viewing coordinate system where the three-dimensional tissue model is located to obtain projection coordinates.

[0077] Among them, the screen coordinate system can be the two-dimensional coordinate system corresponding to the screen where the user performs the key point triggering operation. The viewing coordinate system can be a three-dimensional coordinate system established based on a preset viewing point. The projection coordinates can be the coordinate points of the position of the two-dimensional coordinates on the screen in the viewing coordinate system.

[0078] In this embodiment, the two-dimensional coordinates of the surface key points on the two-dimensional surface image are the points selected on the screen, and this two-dimensional coordinate system is based on the screen coordinate system. It is necessary to convert this two-dimensional coordinate from the screen coordinate system to the viewing coordinate system. Specifically, the key point annotation device can convert the two-dimensional coordinates from the screen coordinate system to the viewing coordinate system according to the projection inverse matrix to obtain the projection coordinates. This projection inverse matrix can be the inverse matrix of the projection matrix based on which the surface of the three-dimensional tissue model is projected onto the preset projection plane. Thus, the association between the two-dimensional coordinates determined on the screen and the scene in the three-dimensional space is realized, and a coordinate conversion relationship is established with the model in the three-dimensional space.

[0079] Step 702, determining a projection ray according to the origin of the viewing coordinate system and the projection coordinates.

[0080] Among them, the origin of the viewing coordinate system can be the preset viewing point. One end of the projection ray can be the origin of the viewing coordinate system, and the projection ray points from the viewing coordinate system to the projection coordinates.

[0081] In this embodiment, the projection coordinates obtained through the inverse transformation are in the viewing coordinate system. The key point annotation device can establish a projection ray pointing to the projection coordinates with the origin of the viewing coordinate system as the end point. For example, as Figure 6 shown, the preset viewing point can be Pr, the projection coordinates can be Ps, then the projection ray can be the ray from Pr to Ps.

[0082] Step 703, determining the line-plane intersection point of the projection ray and the three-dimensional isosurface model, and determining the three-dimensional coordinates according to the line-plane intersection point.

[0083] Among them, the line-plane intersection point can be the intersection point of the projection ray and the three-dimensional isosurface model in the viewing coordinate system.

[0084] In an embodiment of the present disclosure, after determining the projection ray and the unit isosurface model in the observation coordinate system, the key point annotation device may determine the intersection point between the projection ray and the three-dimensional isosurface model to obtain the line-plane intersection point. And use this line-plane intersection point as the three-dimensional coordinates of the surface key point, or perform model conversion based on this line-plane intersection point to obtain the three-dimensional coordinates of the surface key point.

[0085] In some embodiments of the present disclosure, determining the line-plane intersection point between the projection ray and the three-dimensional isosurface model includes:

[0086] Determine the circumscribed cube of the three-dimensional isosurface model; use the circumscribed cube as the cube to be processed, decompose the cube to be processed into 8 sub-cubes. If there is an isosurface in the sub-cube, use the sub-cube as the new cube to be processed, and return to determine the new sub-cube until there is no isosurface in the new sub-cube or the number of iterations reaches the preset iteration threshold, to obtain multiple decomposition blocks corresponding to the three-dimensional isosurface model; among the multiple decomposition blocks, determine the decomposition block corresponding to the maximum value of the number of iterations as the leaf decomposition block, and determine the leaf decomposition block intersecting with the projection ray as the target decomposition block; determine the line-plane intersection point according to the intersection point between the isosurface in the target decomposition block and the projection ray.

[0087] Wherein, the circumscribed cube can be a cube surrounding the three-dimensional isosurface model, and each face of the circumscribed cube can be tangent or parallel to the outermost boundary of the three-dimensional isosurface model, so that the three-dimensional isosurface model is completely located inside the circumscribed cube. The cube to be processed can be the cube currently being decomposed. The sub-cube can be multiple cubes obtained by evenly dividing the cube to be processed. The preset iteration threshold can be the maximum value of the number of iterations set in advance. The decomposition block can be a cube obtained after multi-round iterative cycle decomposition of the circumscribed cube into sub-cubes. The leaf decomposition block can be a decomposition block corresponding to the decomposition iteration number reaching the preset iteration number threshold. The target decomposition block can be the leaf decomposition block where the isosurface intersecting with the projection camera is located.

[0088] In this embodiment, after determining the three-dimensional isosurface model and the projection ray, the key point annotation device can determine the position where the projection ray intersects the three-dimensional isosurface model. Since there are a large number of isosurfaces in the three-dimensional isosurface model, if it is determined whether each isosurface intersects the projection ray in turn, the computational load is large and the computational efficiency is low. Especially in the scenario of interacting with the user, the requirement for computational efficiency is relatively high. Therefore, in this embodiment, the key point annotation device first determines an external cube for the three-dimensional isosurface model and uses this external cube as the cube to be processed. The cube to be processed is evenly decomposed into 8 sub-cubes, and the iteration count is incremented by 1. The initial value of the iteration count can be 0. The 8 sub-cubes are numbered from 0 to 7. It is determined whether there are first array points in the sub-cube. If so, it means that there is an isosurface in the sub-cube, and then this sub-cube is used as the new cube to be processed. It returns to determine the new sub-cubes corresponding to the new cube to be processed and numbers the new sub-cubes until there are no first array points in the new sub-cube or the iteration count reaches the preset iteration threshold. If there are no first array points in the sub-cube, the decomposition of this sub-cube is stopped. Finally, a plurality of decomposition blocks corresponding to the three-dimensional isosurface model are obtained. Taking the target body tissue as the femur as an example, Figure 8 is a schematic diagram of a decomposition block provided by an embodiment of the present disclosure, as Figure 8 shown, Figure 8 shows some of the decomposition blocks determined according to the three-dimensional isosurface model corresponding to the femur.

[0089] It can be understood that there is a tree structure among the parent decomposition block, sub-decomposition block, and grandchild decomposition block in the decomposition block according to the decomposition relationship. When determining the line-plane intersection point, starting from the root node of the tree structure, the depth-first search of the tree structure is used to quickly locate the leaf decomposition block in the tree structure where the iteration count reaches the iteration threshold. Further, it is determined whether the leaf decomposition block intersects the projection ray, and the leaf decomposition block that intersects the projection ray is determined as the target decomposition block. For the multiple isosurfaces in the target decomposition block, it is respectively determined whether the isosurface intersects the projection ray. If so, the intersection point of the isosurface and the projection ray is determined to obtain the line-plane intersection point.

[0090] In the above solution, determining whether each isosurface in the three-dimensional isosurface model intersects the projection ray is converted into determining whether the isosurface in the leaf decomposition block that intersects the projection ray intersects the projection ray, reducing the number of isosurfaces for intersection judgment and improving the operation efficiency of the line-plane intersection point.

[0091] In some embodiments of the present disclosure, determining the three-dimensional coordinates according to the line-plane intersection point includes: according to the matching relationship between the three-dimensional isosurface model and the three-dimensional tissue model, converting the line-plane intersection point from the three-dimensional isosurface model to the three-dimensional tissue model to obtain the three-dimensional coordinates.

[0092] Among them, the matching relationship can be the coordinate matching relationship between the three-dimensional isosurface model and the three-dimensional tissue model. This embodiment does not limit the determination method of this matching relationship. For example, this matching relationship can be determined based on medical image fusion technology. Through medical image fusion technology, the three-dimensional tissue model and the three-dimensional isosurface model that describe the same body tissue can be integrated, thereby providing more comprehensive and accurate information.

[0093] In this embodiment, the key point annotation device can register the three-dimensional isosurface model and the three-dimensional tissue model according to a pre-set registration algorithm to obtain the matching relationship between the three-dimensional isosurface model and the three-dimensional tissue model. And based on this matching relationship, the line-plane intersection point is converted from the unit isosurface model to the three-dimensional tissue model to obtain the three-dimensional coordinates representing the surface key points in the three-dimensional tissue model.

[0094] In the above solution, in the scenario where the user observes and interacts, it is based on the three-dimensional isosurface model obtained by volume rendering of the original three-dimensional tissue model. This three-dimensional isosurface model depends on the transfer function of the color and transparency set by the user, and this three-dimensional isosurface model cannot accurately reflect the position information of the surface of the target body tissue. Therefore, in this embodiment, after determining the line-plane intersection point, the corresponding points on the three-dimensional tissue model can be determined according to the matching relationship between the three-dimensional isosurface model and the three-dimensional tissue model, improving the accuracy of the finally determined surface key points.

[0095] A key point annotation method according to an embodiment of the present disclosure includes obtaining a three-dimensional tissue model of a target body tissue; projecting the surface of the three-dimensional tissue model onto a preset projection plane to obtain a two-dimensional surface image and displaying the two-dimensional surface image; in response to a key point trigger operation of the user on the two-dimensional surface image, determining the two-dimensional coordinates of the surface key point on the two-dimensional surface image; converting the two-dimensional coordinates from the two-dimensional surface image to the three-dimensional tissue model to obtain three-dimensional coordinates; where the three-dimensional coordinates are the coordinates corresponding to the surface key point on the surface of the three-dimensional tissue model. Thus, by projecting the surface of the three-dimensional tissue model into a two-dimensional surface image and displaying this two-dimensional surface image, determining the coordinates of the key point on the two-dimensional surface image according to the key point trigger operation of the user, and performing a two-dimensional to three-dimensional conversion of this coordinate to obtain the coordinates corresponding to the surface of the three-dimensional tissue model, the two-dimensional image of the surface of the body tissue is viewed by the user through the projection of the three-dimensional model surface, reducing the requirement for the user's anatomical knowledge, reducing the difficulty of the user in performing key point annotation on the surface of the body tissue, enabling the user to quickly get started with key point annotation, and also improving the accuracy of the three-dimensional coordinates finally determined through coordinate conversion in representing the corresponding key points.

[0096] In some embodiments of the present disclosure, the key point annotation method further includes: converting the three-dimensional coordinates to the screen coordinate system to obtain the key point screen coordinates. It can be understood that the three-dimensional coordinates are in the viewing coordinate system. An inverse transformation is performed on the three-dimensional coordinates to convert the three-dimensional coordinates from the viewing coordinate system to the screen coordinate system, obtaining the key point screen coordinates. Then, based on the key point screen coordinates, the surface key points are displayed to the user on the screen, and subsequent surgical navigation and other processes are performed based on the three-dimensional coordinates.

[0097] Next, a specific example is used to further illustrate the key point annotation method in the embodiments of the present disclosure. In this example, the target body tissue is the femur, and the surface key point is the bone crest point of the femur. In this example, an initial three-dimensional model is determined based on the CT image sequence of the femur. The three-dimensional tissue model corresponding to the femur region is determined by segmenting the initial three-dimensional model. The three-dimensional isosurface model is obtained by extracting the surface of the three-dimensional tissue model. The two-dimensional coordinates of the bone crest point of the femur in the screen coordinate system are determined through an interactive method. The three-dimensional coordinates are obtained by performing a bone surface intersection operation based on the two-dimensional coordinates. The key point screen coordinates after the three-dimensional coordinates are converted to the screen coordinate system are displayed on the screen and the three-dimensional coordinates are saved.

[0098] First, the three-dimensional tissue model representing the femur region in the initial three-dimensional model is segmented, and the surface of the three-dimensional tissue model is extracted to obtain the three-dimensional isosurface model. Specifically, the initial three-dimensional model can be established based on the CT image sequence of the femur. In this embodiment, the femur region can be segmented from the initial three-dimensional model through a preset tissue segmentation model to obtain the three-dimensional tissue model. For subsequent calculations, the three-dimensional tissue model is converted into topological data that describes the points on the surface of the three-dimensional tissue model and the connection relationships between the points. This topological data is the three-dimensional isosurface model. In this embodiment, the key point annotation device can traverse each voxel in the three-dimensional tissue model to obtain the scalar values of the eight vertices of each voxel. The scalar value is compared with a threshold, and the magnitude relationship of the scalar value is represented by a binary number. A voxel has 8 vertices, and an 8-bit binary number can be constructed for each voxel to represent the state of the voxel, thereby determining the vertex positions, quantities, and spatial connection relationships of the isosurface within a voxel. Finally, the isosurfaces generated in all voxels are merged to form a complete polygon mesh, and this polygon mesh is the three-dimensional isosurface model.

[0099] Further, project the three-dimensional isosurface model onto a preset projection plane to obtain a two-dimensional surface image. In this embodiment, the key point annotation device may project the three-dimensional isosurface model in three-dimensional space onto the preset projection plane based on a ray projection algorithm to obtain a two-dimensional surface image. Specifically, the key point annotation device may emit rays with a preset observation point as the emission point, and equidistant sampling points are set on the rays. During the process of the rays passing through the three-dimensional isosurface model, the color data and transparency of the voxels corresponding to each sampling point are obtained. According to the ray absorption model, the color data and transparency of each sampling point on the rays are accumulated until the rays leave the three-dimensional isosurface model or the transparency accumulates to completely opaque, then the sampling ends.

[0100] Further, determine the three-dimensional coordinates of the surface key points in the observation coordinate system according to the key point trigger operation. In this embodiment, the user may select points on the screen with the mouse at the observed bone crest points according to the displayed two-dimensional surface image. Based on this point selection operation, the key point annotation device may determine the two-dimensional coordinates of the surface key points in the screen coordinate system and map the two-dimensional coordinates on the screen to the corresponding position of the bone surface in the three-dimensional space where the observation coordinate system is located. This mapping may be realized by performing coordinate system transformation based on spatial geometric operations.

[0101] Specifically, the two-dimensional coordinates are the coordinates of the points selected on the screen, and these two-dimensional coordinates are based on the screen coordinate system and need to be transformed to the observation coordinate system. The transformation matrix for coordinate system transformation may be the inverse matrix of the projection matrix that projects the three-dimensional isosurface model onto the preset projection plane. The projection coordinates after inverse matrix transformation are in the observation coordinate system. Then, a projection ray is constructed according to the preset observation point and the projection coordinates.

[0102] Due to the large number of isosurfaces in the three-dimensional isosurface model, if it is sequentially determined whether the isosurface intersects the projection ray, the calculation efficiency is relatively low. Especially in an interactive scenario with high efficiency requirements, this disadvantage is more obvious. Therefore, the key point annotation device can determine the circumscribed cube of the three-dimensional isosurface model, use this circumscribed cube as the cube to be processed, evenly decompose the cube to be processed into 8 sub-cubes, numbered 0-7. If there are no first array points in the sub-cube, stop decomposing the sub-cube; otherwise, use this sub-cube as the new cube to be processed, and return to further decompose the new cube to be processed to obtain new sub-cubes and number them. And a preset iteration threshold can be set, and the cube stops decomposing after the number of decomposition times reaches the preset iteration threshold. At the same time, a tree structure is formed among the parent decomposition block, sub-decomposition block, and grandchild decomposition block determined during the decomposition process. When solving and calculating, start from the root node of the tree structure, and use the depth-first search of the tree structure to quickly locate the final leaf decomposition block. Then, only judge whether the isosurfaces inside the leaf decomposition block that intersect the projection ray intersect the projection ray in turn, and finally determine the coordinates of the line-plane intersection point, thereby improving the operation efficiency of determining the line-plane intersection point.

[0103] In addition, in this embodiment, since the model for the user to observe and interact with is a three-dimensional isosurface model determined by volume rendering based on a three-dimensional tissue model and depends on the transfer function of color and transparency set in advance by the user, the three-dimensional isosurface model obtained by volume rendering itself cannot reflect the accurate position information of the femoral surface. Therefore, in this embodiment, the three-dimensional isosurface model and the three-dimensional tissue model can be fused, that is, the three-dimensional isosurface model and the three-dimensional tissue model are aligned in coordinates to form a comprehensive model that simultaneously includes the three-dimensional tissue model and the three-dimensional isosurface model. Project the three-dimensional isosurface model in the comprehensive model onto a preset projection plane to obtain a two-dimensional surface image. In subsequent interactions with the user, use the three-dimensional isosurface model in the comprehensive model to determine the line-plane intersection point, and convert this line-plane intersection point to the three-dimensional tissue model in the comprehensive model to generate three-dimensional coordinates that accurately represent the bone crest points on the bone surface.

[0104] Furthermore, display the surface key points on the screen. Specifically, the three-dimensional coordinates are in the viewing coordinate system. Perform an inverse transformation on this three-dimensional coordinate system to obtain the key point screen coordinates in the screen coordinate system. Based on these key point screen coordinates, display the surface key points on the screen, and save the three-dimensional coordinates for subsequent surgical navigation, etc.

[0105] The key point annotation method provided in the embodiments of the present disclosure intuitively shows the surface of the femur to the user. The user does not need to analyze the cross-sectional view of the body tissue, the operation is relatively intuitive, the requirement for the user's anatomical knowledge reserve is relatively low, and the operation threshold is low and it is easy to get started.

[0106] Figure 9 The figure shows a schematic structural diagram of a key point annotation device provided by an embodiment of the present disclosure.

[0107] As Figure 9 shown, the key point annotation device 900 may include:

[0108] An acquisition module 901, configured to acquire a three-dimensional tissue model of a target body tissue;

[0109] A projection module 902, configured to project the surface of the three-dimensional tissue model onto a preset projection plane to obtain a two-dimensional surface image, and display the two-dimensional surface image;

[0110] A trigger module 903, configured to determine the two-dimensional coordinates of a surface key point on the two-dimensional surface image in response to a key point trigger operation of a user on the two-dimensional surface image;

[0111] A conversion module 904, configured to convert the two-dimensional coordinates from the two-dimensional surface image to the three-dimensional tissue model to obtain three-dimensional coordinates; wherein, the three-dimensional coordinates are the coordinates corresponding to the surface key point on the surface of the three-dimensional tissue model.

[0112] Optionally, the acquisition module 901 is configured to:

[0113] Acquire an initial three-dimensional model of a target body part; wherein, the initial three-dimensional model is generated according to a computed tomography image of the target body part;

[0114] Input the initial three-dimensional model into a preset tissue segmentation model to obtain the three-dimensional tissue model corresponding to the target body tissue in the target body part.

[0115] Optionally, the projecting the surface of the three-dimensional tissue model onto a preset projection plane to obtain a two-dimensional surface image includes:

[0116] Performing an isosurface extraction process on the three-dimensional tissue model to obtain a three-dimensional isosurface model;

[0117] Projecting the three-dimensional isosurface model onto the preset projection plane based on a preset observation point to obtain the two-dimensional surface image.

[0118] Optionally, the performing an isosurface extraction process on the three-dimensional tissue model to obtain a three-dimensional isosurface model includes:

[0119] Determining the array points located inside the three-dimensional tissue model in the array points as first array points, and determining the array points located outside the three-dimensional tissue model in the array points as second array points; wherein, the array points are equally spaced points in the three-dimensional space where the three-dimensional tissue model is located;

[0120] Determine array point pairs; wherein, one of the array point pairs includes an adjacent one of the first array points and the second array points;

[0121] Construct the isosurface according to the array point pairs, and merge the isosurfaces to obtain the three-dimensional isosurface model.

[0122] Optionally, the displaying the two-dimensional surface image includes:

[0123] Determine a three-dimensional auxiliary model of the auxiliary body tissue in the target body part, project the surface of the three-dimensional auxiliary model onto a preset projection plane to obtain a two-dimensional auxiliary image;

[0124] Jointly display the two-dimensional auxiliary image and the two-dimensional surface image.

[0125] Optionally, the conversion module 904 includes:

[0126] Convert the two-dimensional coordinates from the screen coordinate system to the observation coordinate system where the three-dimensional tissue model is located to obtain projection coordinates;

[0127] Determine a projection ray according to the origin of the observation coordinate system and the projection coordinates;

[0128] Determine the line-plane intersection point of the projection ray and the three-dimensional isosurface model, and determine the three-dimensional coordinates according to the line-plane intersection point.

[0129] Optionally, the determining the line-plane intersection point of the projection ray and the three-dimensional isosurface model includes:

[0130] Determine the circumscribed cube of the three-dimensional isosurface model;

[0131] Take the circumscribed cube as the cube to be processed, decompose the cube to be processed into 8 sub-cubes. If there is an isosurface in the sub-cube, take the sub-cube as the new cube to be processed, and return to determine a new sub-cube until there is no isosurface in the new sub-cube or the number of iterations reaches a preset iteration threshold to obtain a plurality of decomposition blocks corresponding to the three-dimensional isosurface model;

[0132] Among the plurality of decomposition blocks, determine the decomposition block corresponding to the maximum value of the number of iterations as the leaf decomposition block, and determine the decomposition block intersecting with the projection ray as the target decomposition block;

[0133] Determine the line-plane intersection point according to the intersection point of the isosurface in the target decomposition block and the projection ray.

[0134] Optionally, the determining the three-dimensional coordinates according to the line-plane intersection point includes:

[0135] According to the matching relationship between the three-dimensional isosurface model and the three-dimensional tissue model, the line-surface intersection points are converted from the three-dimensional isosurface model to the three-dimensional tissue model to obtain the three-dimensional coordinates.

[0136] It should be noted that Figure 9 The key point annotation device 900 shown can execute each step in the above-mentioned key point annotation method embodiment, and achieve each process and effect in the above-mentioned key point annotation method embodiment, which will not be elaborated here.

[0137] Figure 10 The structural schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown.

[0138] As Figure 10 shown, the electronic device may include a processor 1001 and a memory 1002 storing computer program instructions.

[0139] Specifically, the above-mentioned processor 1001 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0140] The memory 1002 may include a mass storage for information or instructions. By way of example and not limitation, the memory 1002 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 1002 may include removable or non-removable (or fixed) media. In a suitable case, the memory 1002 may be internal or external to the integrated gateway device. In a particular embodiment, the memory 1002 is a non-volatile solid-state memory. In a particular embodiment, the memory 1002 includes a read-only memory (ROM). In a suitable case, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0141] The processor 1001 reads and executes the computer program instructions stored in the memory 1002 to perform the steps of the key point annotation method provided by the embodiments of the present disclosure.

[0142] In one example, the electronic device may further include a transceiver 1003 and a bus 1004. Among them, as Figure 10 shown, the processor 1001, the memory 1002, and the transceiver 1003 are connected through the bus 1004 and complete communication with each other.

[0143] The bus 1004 includes hardware, software, or both. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side BUS (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. In a suitable case, the bus 1004 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0144] The following are embodiments of a computer-readable storage medium provided by the embodiments of the present disclosure. The computer-readable storage medium and the key point annotation method of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the computer-readable storage medium may refer to the embodiments of the key point annotation method.

[0145] This embodiment provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute a key point annotation method when executed by a computer processor.

[0146] Of course, for a storage medium containing computer-executable instructions provided by an embodiment of the present disclosure, the computer-executable instructions are not limited to the above method operations, and can also execute related operations in the key point annotation methods provided by any embodiment of the present disclosure.

[0147] From the above description of the embodiments, those skilled in the art can clearly understand that the present disclosure can be implemented by means of software and necessary general-purpose hardware. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as a floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disc of a computer, etc., including several instructions to enable a computer cloud platform (which can be a personal computer, server, or network cloud platform, etc.) to execute the key point annotation methods provided by various embodiments of the present disclosure.

[0148] Note that the above is only a preferred embodiment of the present disclosure and the applied technical principle. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments here, and various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present disclosure. Therefore, although the present disclosure has been described in detail through the above embodiments, the present disclosure is not limited to the above embodiments. Without departing from the concept of the present disclosure, more other equivalent embodiments can be included, and the scope of the present disclosure is determined by the scope of the appended claims.

Claims

1. A key point labeling method, characterized in that: include: Acquire a three-dimensional tissue model of the target body tissue; Projecting the surface of the three-dimensional tissue model onto a preset projection plane to obtain a two-dimensional surface image, and displaying the two-dimensional surface image; In response to a user triggering operation on a key point of the two-dimensional surface image, determining a two-dimensional coordinate of the surface key point on the two-dimensional surface image; The two-dimensional coordinates are converted from the two-dimensional surface image to the three-dimensional tissue model to obtain three-dimensional coordinates; wherein the three-dimensional coordinates are the coordinates corresponding to the surface key points on the surface of the three-dimensional tissue model.

2. The method according to claim 1, characterized in that The step of obtaining a three-dimensional tissue model of the target body tissue comprises: Acquire an initial three-dimensional model of the target body part; wherein the initial three-dimensional model is generated based on a computer tomography image of the target body part; The initial three-dimensional model is input into a preset tissue segmentation model to obtain the three-dimensional tissue model corresponding to the target body tissue in the target body part.

3. The method according to claim 1, characterized in that The step of projecting the surface of the three-dimensional tissue model onto a preset projection plane to obtain a two-dimensional surface image includes: Performing isosurface extraction processing on the three-dimensional tissue model to obtain a three-dimensional isosurface model; The three-dimensional isosurface model is projected onto the preset projection plane based on a preset observation point to obtain the two-dimensional surface image.

4. The method according to claim 3, characterized in that The step of performing isosurface extraction processing on the three-dimensional tissue model to obtain a three-dimensional isosurface model includes: Determine the array points located inside the three-dimensional tissue model as first array points, and determine the array points located outside the three-dimensional tissue model as second array points; wherein the array points are points equidistantly arranged in the three-dimensional space where the three-dimensional tissue model is located; Determine an array point pair; wherein one of the array point pairs includes an adjacent first array point and second array point; The isosurface is constructed according to the array point pairs, and the isosurfaces are merged to obtain the three-dimensional isosurface model.

5. The method according to claim 1, characterized in that The displaying of the two-dimensional surface image comprises: Determine a three-dimensional auxiliary model of auxiliary body tissue in the target body part, and project a surface of the three-dimensional auxiliary model onto a preset projection plane to obtain a two-dimensional auxiliary image; The two-dimensional auxiliary image and the two-dimensional surface image are jointly displayed.

6. The method according to claim 3, characterized in that The step of converting the two-dimensional coordinates from the two-dimensional surface image to the three-dimensional tissue model to obtain the three-dimensional coordinates comprises: Convert the two-dimensional coordinates from the screen coordinate system to the observation coordinate system where the three-dimensional tissue model is located to obtain projection coordinates; Determining a projection ray according to the origin of the observation coordinate system and the projection coordinates; The line-surface intersection point of the projection ray and the three-dimensional isosurface model is determined, and the three-dimensional coordinates are determined according to the line-surface intersection point.

7. The method according to claim 6, characterized in that The determining of the line-surface intersection point between the projection ray and the three-dimensional isosurface model comprises: Determining a circumscribed cube of the three-dimensional isosurface model; The circumscribed cube is used as a cube to be processed, and the cube to be processed is decomposed into 8 sub-cubes. If the isosurface exists in the sub-cube, the sub-cube is used as a new cube to be processed, and a new sub-cube is determined again until the isosurface does not exist in the new sub-cube or the number of iterations reaches a preset iteration threshold, thereby obtaining a plurality of decomposed blocks corresponding to the three-dimensional isosurface model; Among the multiple decomposition blocks, determine the decomposition block corresponding to the maximum value of the iteration number as a leaf decomposition block, and determine the leaf decomposition block intersecting with the projection ray as a target decomposition block; The line-surface intersection point is determined according to the intersection point of the isosurface in the target decomposition block and the projection ray.

8. The method according to claim 6, characterized in that The determining of the three-dimensional coordinates according to the line-plane intersection point comprises: According to the matching relationship between the three-dimensional isosurface model and the three-dimensional tissue model, the line-surface intersection is converted from the three-dimensional isosurface model to the three-dimensional tissue model to obtain the three-dimensional coordinates.

9. A key point marking device, characterized in that: include: An acquisition module, used for acquiring a three-dimensional tissue model of a target body tissue; A projection module, used for projecting the surface of the three-dimensional tissue model onto a preset projection plane to obtain a two-dimensional surface image, and displaying the two-dimensional surface image; A trigger module, configured to determine the two-dimensional coordinates of the surface key point on the two-dimensional surface image in response to a user triggering operation on the key point of the two-dimensional surface image; A conversion module is used to convert the two-dimensional coordinates from the two-dimensional surface image to the three-dimensional tissue model to obtain three-dimensional coordinates; wherein the three-dimensional coordinates are the coordinates corresponding to the surface key points on the surface of the three-dimensional tissue model.

10. An electronic device, characterized in that: include: processor; A memory for storing executable instructions; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the method described in any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the method according to any one of claims 1 to 8.

Images