Decision model modeling method and device for auxiliary planning of open type surgical approach

By constructing a neural network model, using preoperative imaging features and surgical wound features, automated auxiliary surgical planning is realized, solving the problem of low automation in the existing technology, and improving the accuracy and efficiency of surgical planning.

CN118873249APending Publication Date: 2024-11-01TSINGHUA UNIVERSITY
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202410906427.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

In the prior art, it is difficult to realize automated auxiliary surgical planning, rely on the doctor's subjective decision-making, and fail to directly apply expert experience, resulting in a low degree of automation.

Method used

By segmenting the patient's preoperative images, the lesion feature vector and the main contour point feature vector of the surgical wound are obtained, a neural network in the form of a fully connected/multi-layer perceptron is constructed, an expert decision-making model is established, and the patient's preoperative image features are realized to the expert The mapping between surgical decision characteristics results in the final expert decision model used to assist in the planning of open surgical approaches.

Benefits of technology

It realizes automated auxiliary surgical planning, improves the clinical planning skills of novices, enhances the degree of automation of surgical planning, can intuitively compare the differences between planning results and expert decision-making models, and improves training results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118873249A_ABST
    Figure CN118873249A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of computer-aided surgery, in particular to a decision model modeling method and device for auxiliary planning of an open type surgical approach, and the method comprises the steps: carrying out the focus segmentation of a preoperative image of a patient, and obtaining a focus image in a mask form; based on the focus image in the mask form, obtaining a focus feature vector of the patient, and obtaining a main contour point feature vector of the surgical wound of the patient; constructing a neural network in a full-connection / multi-layer perceptron form as an expert decision model ontology, and performing model training by utilizing a focus feature vector and a main contour point feature vector so as to establish mapping between a patient preoperative image feature and an expert operation decision feature; and obtaining a final expert decision model for auxiliary planning of the open surgical approach. Therefore, the problems that in the related technology, expert experience is not directly applied, subjective decisions of doctors are still needed to determine whether the operation planning is available or not, and automatic auxiliary operation planning is difficult to achieve are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of computer-assisted surgery, and in particular to a decision modeling method and device for assisting in planning an open surgical approach. Background Art

[0002] The creation of an open wound on the patient's body surface, skull, or specific bone tissue through surgery is a necessary step to physically remove some lesions in the patient's body that cannot be completely cured by minimally invasive means, or to implement other treatment methods. Usually, the approach planning of such surgery requires the doctor to complete it based on a comprehensive assessment of the lesion information and his or her experience.

[0003] In the related technology, Patent No. CN117576321A can be used to build an auxiliary liver surgery planning modeling system, that is, mapping the designed surgical path into the two-dimensional liver image and combining the expert knowledge base to evaluate the planning effect at this time; Patent No. CN117297771A can also be used to develop a brain tumor surgery planning risk assessment method based on deep learning, that is, based on the twin network and contrast constraints, the input preoperative images are used to perform a risk assessment of the candidate craniotomy area to obtain the optimal craniotomy area, and then determine the optimal surgical path; Experts can also participate in the surgery remotely through remote connection with Publication No. CN110931121A.

[0004] However, in the related art, Patent No. CN117576321A does not directly apply expert experience to model the expert's decision-making experience. Patent No. CN117297771A still requires the doctor's subjective decision to determine whether it is available, and Publication No. CN110931121A only provides a medium for remote consultation or remote surgery. It does not automatically apply the doctor's expert experience, and it is difficult to achieve automated auxiliary surgery planning. The degree of automation is low and urgently needs to be improved. Summary of the invention

[0005] The present application provides a decision modeling method and device for assisting in planning an open surgical approach, in order to solve the problems in the related technology that there is no direct application of expert experience, the doctor's subjective decision is still required to determine whether it is available, and it is difficult to achieve automated assisted surgical planning.

[0006] The first aspect of the present application provides a decision modeling method for assisting in the planning of an open surgical approach, comprising the following steps: performing lesion segmentation on a patient's preoperative image to obtain a lesion image in a masked form; based on the lesion image in the masked form, obtaining a lesion feature vector of the patient, and obtaining a feature vector of the main contour points of the patient's surgical wound; constructing a neural network in the form of a fully connected / multi-layer perceptron as an expert decision model ontology, using the lesion feature vector and the feature vector of the main contour points of the surgical wound to perform model training to establish a mapping between the patient's preoperative image features and the expert surgical decision features, and obtaining a final expert decision model for assisting in the planning of an open surgical approach.

[0007] Optionally, in one embodiment of the present application, obtaining the patient's lesion feature vector includes: constructing an autoencoder neural network, wherein the autoencoder neural network includes a neural network encoder and a neural network decoder, the neural network encoder is used to reduce the dimension of the lesion from its original scale to a specific scale for feature extraction, and the neural network decoder restores the reduced features to the original lesion scale.

[0008] Optionally, in one embodiment of the present application, obtaining the feature vector of the main contour points of the surgical wound of the patient includes: spatially locating the main contour points of the surgical wound and quantitatively characterizing the degree of spatial discreteness to respectively describe the position and size of the wound, and obtaining the feature vector of the main contour points of the surgical wound.

[0009] Optionally, in one embodiment of the present application, the spatial positioning of the main contour points of the surgical incision and the quantitative characterization of the degree of spatial discreteness include: determining the coordinates of the patient's specific anatomical landmark points based on the lesion image, and obtaining the coordinates of the main contour points taught by experts; establishing a coordinate system of specific anatomical landmark points based on the coordinates of the specific anatomical landmark points, and calculating the coordinates of the main contour points in the coordinate system to obtain the coordinates of the main contour points in the coordinate system; obtaining the maximum physical scale in the coordinate system to normalize the coordinates of the main contour points in the coordinate system to obtain the spatial distribution information of the normalized main contour points.

[0010] Optionally, in one embodiment of the present application, the output result of the final expert decision model includes the spatial discreteness and positioning information of the main contour points of the surgical wound, wherein the parameter tuning formula of the final expert decision model may be but is not limited to:

[0011]

[0012] in, is the expected output, mask is the input of AE encoder, E θis the expert decision model, and loss is the loss function.

[0013] The second aspect of the present application provides a decision modeling device for assisting in the planning of an open surgical approach, including: a generation module, used to perform lesion segmentation on a patient's preoperative image to obtain a lesion image in a masked form; an acquisition module, used to obtain the patient's lesion feature vector based on the lesion image in the masked form, and obtain the patient's surgical wound main contour point feature vector; a construction module, used to construct a neural network in the form of a fully connected / multi-layer perceptron as the expert decision model body, and use the lesion feature vector and the surgical wound main contour point feature vector for model training to establish a mapping between the patient's preoperative image features and the expert surgical decision features, so as to obtain the final expert decision model for assisting in the planning of an open surgical approach.

[0014] Optionally, in one embodiment of the present application, the acquisition module includes: a construction unit for constructing an autoencoder neural network, wherein the autoencoder neural network includes a neural network encoder and a neural network decoder, the neural network encoder is used to reduce the dimension of the lesion from its original scale to a specific scale for feature extraction, and the neural network decoder restores the reduced features to the original lesion scale.

[0015] Optionally, in one embodiment of the present application, the acquisition module includes: a generation unit for spatially locating the main contour points of the surgical wound and quantitatively representing the degree of spatial discreteness to respectively describe the position and size of the wound and obtain a feature vector of the main contour points of the surgical wound.

[0016] Optionally, in one embodiment of the present application, the generation unit includes: an acquisition subunit, used to determine the coordinates of the patient's specific anatomical landmark points based on the lesion image, and obtain the coordinates of the main contour points taught by experts; a calculation subunit, used to establish a coordinate system of specific anatomical landmark points based on the coordinates of the specific anatomical landmark points, and calculate the coordinates of the main contour points in the coordinate system to obtain the coordinates of the main contour points in the coordinate system; a generation subunit, used to obtain the maximum physical scale in the coordinate system to normalize the coordinates of the main contour points in the coordinate system to obtain the spatial distribution information of the normalized main contour points.

[0017] Optionally, in one embodiment of the present application, the output result of the final expert decision model includes the spatial discreteness and positioning information of the main contour points of the surgical wound, wherein the parameter tuning formula of the final expert decision model may be but is not limited to:

[0018]

[0019] in, is the expected output, mask is the input of AE encoder, E θ is the expert decision model, and loss is the loss function.

[0020] A third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a decision modeling method for assisting in planning an open surgical approach as described in the above embodiment.

[0021] A fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-mentioned decision modeling method for assisting in planning an open surgical approach.

[0022] A fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, implements the above-mentioned decision modeling method for assisting in planning an open surgical approach.

[0023] The embodiment of the present application can obtain a lesion image in the form of a mask based on the patient's preoperative image segmentation, and then obtain the lesion feature vector and the feature vector of the main contour points of the surgical wound, thereby training the expert decision model, establishing a mapping between the patient's preoperative image features and the expert surgical decision features, and obtaining the final expert decision model for assisting in planning the open surgical approach, which allows novices to intuitively compare the differences in positioning and spatial discreteness between their planning results and the results output by the final expert decision model, thereby improving their clinical planning skills in a targeted manner and achieving the beneficial effect of training novices. It is a general solution for the special unstructured scenario of open surgical planning. As a result, it solves the problems in the related technology that there is no direct application of expert experience, and the subjective decision of the doctor is still required to determine whether it is available, and it is difficult to achieve automated auxiliary surgical planning.

[0024] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0026] Figure 1 A flowchart of a decision modeling method for assisting in planning an open surgical approach provided according to an embodiment of the present application;

[0027] Figure 2A flowchart of an autoencoder neural network training method provided according to an embodiment of the present application;

[0028] Figure 3 A flowchart of a main contour point encoding method provided according to an embodiment of the present application;

[0029] Figure 4 A block diagram of a main contour point encoding method provided according to an embodiment of the present application;

[0030] Figure 5 A schematic diagram of the effect of reconstructing a tumor mask using an autoencoder neural network provided according to an embodiment of the present application;

[0031] Figure 6 A flowchart of constructing and training an expert decision model according to an embodiment of the present application;

[0032] Figure 7 A flowchart of using an expert decision model as a clinical training system according to an embodiment of the present application;

[0033] Figure 8 A schematic block diagram of a decision modeling device for assisting in planning an open surgical approach provided according to an embodiment of the present application;

[0034] Fig. 9 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0036] The following describes the decision modeling method and device for assisting the planning of open surgical approaches in the embodiment of the present application with reference to the accompanying drawings. In view of the problem mentioned in the above background technology that there is no direct application of expert experience, the doctor's subjective decision is still required to determine whether it is available, and it is difficult to achieve automated auxiliary surgical planning, the present application provides a decision modeling method for assisting the planning of open surgical approaches, in which the lesion image in the form of a mask can be obtained according to the patient's preoperative image segmentation, and then the lesion feature vector and the main contour point feature vector of the surgical wound are obtained, thereby training the expert decision model, establishing a mapping between the patient's preoperative image features and the expert surgical decision features, and obtaining the final expert decision model for assisting the planning of open surgical approaches, which allows novices to intuitively compare the differences in positioning and spatial discreteness between their planning results and the results output by the final expert decision model, so as to improve their clinical planning skills in a targeted manner and achieve the beneficial effect of training novices, which is a general solution for the special unstructured scenario of open surgical planning. Thus, the problems that there is no direct application of expert experience in the related technology, the doctor's subjective decision is still required to determine whether it is available, and it is difficult to achieve automated auxiliary surgical planning are solved.

[0037] Specifically, Figure 1 The present invention is a flowchart of a decision modeling method for assisting in planning an open surgical approach according to an embodiment of the present application.

[0038] like Figure 1 As shown, the decision modeling method for assisting in planning an open surgical approach includes the following steps:

[0039] In step S101, lesion segmentation is performed on the patient's preoperative image to obtain a lesion image in the form of a mask.

[0040] In the actual implementation process, the embodiment of the present application can perform lesion analysis on the patient's preoperative image to segment the patient's lesion image, that is, the lesion image in the form of a mask, which can be represented as a mask.

[0041] In step S102, based on the lesion image in the form of a mask, a feature vector of the patient's lesion is obtained, and a feature vector of the main contour points of the patient's surgical wound is obtained.

[0042] In some embodiments, as a direct influencing factor of decision-making, the lesions in the patient's body of the embodiment of the present application must be quantified before they can be used as model input. In addition, in order to cope with the irregular morphology and random location of lesions in the patient's body that are not conducive to conventional measurement and quantitative characterization, the embodiment of the present application can use an autoencoder neural network (Autoencoder, AE) to extract features from the lesion image in mask form, and then obtain a lesion feature vector.

[0043] In some embodiments, the embodiments of the present application can mark the main contour points of the incision required for open surgery based on the experience of human experts, and perform feature extraction on the main contour points of the surgical incision to obtain the feature vector of the main contour points of the patient's surgical incision, wherein the embodiments of the present application perform feature extraction on the main contour points, that is, completely construct a data set for training the expert decision model.

[0044] In some embodiments, the feature vector of the main contour points of the surgical wound in the embodiment of the present application and the corresponding lesion feature vector can be used as the output part and the input part of the data set respectively.

[0045] Optionally, in one embodiment of the present application, obtaining a patient's lesion feature vector includes: constructing an autoencoder neural network, wherein the autoencoder neural network includes a neural network encoder and a neural network decoder, the neural network encoder is used to reduce the lesion from its original scale to a specific scale for feature extraction, and the neural network decoder restores the reduced features to the original lesion scale.

[0046] That is to say, the embodiment of the present application can obtain the patient's lesion feature vector by constructing an autoencoder neural network, wherein the autoencoder neural network may include, but is not limited to, a neural network encoder and a neural network decoder. The former is used to reduce the dimension of the lesion from its original scale to a specific scale to achieve feature extraction, and the latter is responsible for restoring the reduced feature to the original lesion scale. Specifically, the flowchart of the autoencoder neural network training method of the embodiment of the present application is as follows: Figure 2 As shown, it can be:

[0047] Step S201: Construct an AE function body with a symmetrical convolution and transposed convolution network. It can be understood that the initial parameter of the embodiment of the present application is set to θ. In addition, the embodiment of the present application can use three-dimensional convolution and three-dimensional transposed convolution for the three-dimensional mask. The former is used to reduce the dimension of the lesion from its original scale to a specific scale to achieve feature extraction, and the latter is responsible for restoring the reduced dimension features to the original lesion scale. Among them, the whole AE function body can be a U-shaped network, and this application does not make specific restrictions.

[0048] Step S202: prepare a certain number of mask data sets for training AE. It can be understood that the embodiment of the present application can first check and clear the outliers in the data set and perform data normalization.

[0049] Step S203: Perform self-supervisory training on AE. That is, the encoder input of AE in the embodiment of the present application is mask, and the encoder output is the feature vector V of the mask, which can be obtained by formula (1); the decoder input of AE is the feature vector V of the mask, and the output is a reconstructed image M, which can be obtained by formula (2). Self-supervision is achieved by calculating the root mean square error between the mask and the reconstructed M, which can be obtained by formula (3), where N is the number of volumes / pixels. In particular, in the embodiment of the present application, the lesion mask or the volume present is too small in the entire image volume, resulting in training anomalies. A voxel-weighted correction term W is introduced to reward AE for learning to pay attention to the mask area and ignore the background voxels. Wherein, W is a matrix of the same dimension as the mask image. If the mask area takes a larger value and the background area takes a smaller value, the ratio of the two can be set to 100:1, and this application does not make specific restrictions.

[0050] V = encoder θ (mask), (1)

[0051] M=encoder θ (V), (2)

[0052]

[0053] Step S204: Optimize the AE parameters by gradient descent method. That is, the embodiment of the present application can use formula (4) and formula (5) to optimize the AE parameters. When the loss value is stable at a small value, a usable AE is obtained. At this time, its parameter is θ * , as shown in formula (5).

[0054]

[0055] θ * =argmin θ (loss(mask,decoder θ (encoder θ (mask)))), (5)

[0056] Among them, α is the learning rate used to solve the gradient descent, loss θ Indicates that under the current AE parameters (ie AE θ )The loss obtained is To find the gradient operation.

[0057] Optionally, in one embodiment of the present application, a feature vector of the main contour points of the patient's surgical wound is obtained, including: spatially locating the main contour points of the surgical wound and quantitatively characterizing the degree of spatial discreteness to respectively describe the position and size of the wound, and obtaining a feature vector of the main contour points of the surgical wound.

[0058] During the actual implementation process, for the lesion image obtained in step S101, an expert can mark the main contour points of the wound required for open surgery based on experience in the embodiment of the present application. The main contour points will be used to characterize the surgical approach later, such as the intended point of the skull drilling in craniotomy. Furthermore, the embodiment of the present application performs spatial positioning of the main contour points and quantitative characterization of the degree of spatial discreteness to respectively describe the position and size of the wound and thereby indirectly quantify the wound, thereby obtaining a feature vector of the main contour points of the surgical wound.

[0059] Optionally, in one embodiment of the present application, the main contour points of the surgical incision are spatially positioned and the degree of spatial discreteness is quantitatively characterized, including: determining the coordinates of the patient's specific anatomical landmark points based on the lesion image, and obtaining the coordinates of the main contour points taught by experts; establishing a coordinate system of the specific anatomical landmark points based on the coordinates of the specific anatomical landmark points, and calculating the coordinates of the main contour points in the coordinate system to obtain the coordinates of the main contour points in the coordinate system; obtaining the maximum physical scale in the coordinate system to normalize the coordinates of the main contour points in the coordinate system to obtain the spatial distribution information of the normalized main contour points.

[0060] As a possible implementation method, the embodiment of the present application performs spatial positioning of the main contour points of the surgical wound and quantitative characterization of the degree of spatial discreteness, and its construction method is as follows: Figure 3 As shown, the main steps can be:

[0061] Step S301: Determine the coordinates of the patient's specific anatomical landmarks from the patient's lesion image. It can be understood that the embodiment of the present application can determine the coordinates of specific anatomical landmarks according to different types of surgeries, such as obtaining the coordinates of specific anatomical landmarks, such as the centroid coordinates of the brain region, in a craniotomy.

[0062] Step S302: Obtaining the coordinates of the main contour points taught by the expert. It can be understood that the embodiment of the present application can obtain the coordinates of the drilling positions taught by the expert.

[0063] Step S303: Establishing a coordinate system of a specific anatomical landmark based on the coordinates of the specific anatomical landmark. It can be understood that the embodiment of the present application can reconstruct the anatomical coordinate system with the centroid coordinates of the brain region as the origin to obtain the centroid coordinate system.

[0064] Step S304: Calculate the coordinates of the main contour points in the coordinate system to obtain the coordinates of the main contour points in the coordinate system. It can be understood that the embodiment of the present application can calculate the coordinates of the drilling position taught by the expert in the centroid coordinate system, that is, in the centroid coordinate system, calculate the position of the taught drilling position relative to the centroid of the brain area.

[0065] Step S305: Obtaining the maximum physical scale in the coordinate system. It can be understood that the embodiment of the present application can obtain the maximum physical scale in the centroid coordinate system, that is, compare the brain region boundary lengths in the centroid coordinate system in the axial, coronal and sagittal directions and take the maximum value.

[0066] Step S306: Normalize the coordinates of the main contour points in the coordinate system. It can be understood that the embodiment of the present application can normalize the coordinates of the drilling position obtained in S304 with reference to the maximum physical scale to remove the dimension effect and retain the relative information of the expert teaching data.

[0067] Step S307: Obtain the spatial distribution information of the normalized main contour points. It can be understood that the embodiment of the present application can calculate the normalized spatial distribution information of the drilling positions, that is, encode the drilling data to obtain the normalized spatial distribution information of the drilling positions. Specifically, Figure 4 and Figure 5 As shown, in Figure 4 In the figure, the origin C of the coordinate system is the anatomical landmark point, such as the centroid of the brain region in the craniotomy, F1-p1 represents the first contour point of wound 1, F2-p1 represents the first contour point of wound 2, o1 and o2 are the geometric center points of wound 1 and wound 2, respectively. In the embodiment of the present application, wound 1 and wound 2 are only used to distinguish on the schematic diagram and have no obvious actual physical meaning. Figure 5 As mentioned above, the left picture is the original image of the lesion mask, and the right picture is the reconstructed image of the lesion mask. The main contour point coding of the surgical wound includes two aspects: spatial positioning and spatial discreteness. Among them, spatial positioning is defined as the angle between the unit space vector pointing from the centroid to the geometric center of the drilling position and the three positive directions of the centroid coordinate system, namely [α, β, γ] T ; The spatial discreteness is defined as the variance of each drilling position in three anatomical directions, namely As shown in formula (9)-formula (11), it can be expressed as:

[0068]

[0069]

[0070] In step S103, a neural network in the form of a fully connected / multi-layer perceptron is constructed as the expert decision model body, and the lesion feature vector and the feature vector of the main contour points of the surgical incision are used for model training to establish a mapping between the patient's preoperative imaging features and the expert surgical decision features, thereby obtaining the final expert decision model for assisting in planning the open surgical approach.

[0071] In the actual implementation process, the embodiment of the present application can construct a neural network in the form of a fully connected / multi-layer perceptron as the expert decision model body, and train the lesion feature vector as input and the feature vector of the main contour points of the surgical incision as output, so as to ultimately establish a mapping between the patient's preoperative imaging features and the expert surgical decision features, and obtain the final expert decision model for assisting in the planning of open surgical approaches.

[0072] That is to say, the embodiment of the present application can construct and train the expert decision model E θ , and then obtain the final expert decision model. In this embodiment, the nonlinear characteristics of expert decision-making are taken into account. A neural network in the form of a fully connected / multi-layer perceptron can be used as the expert decision model body, and the parameters are optimized through specific expert teaching data training to obtain the final expert decision model. The specific steps are as follows Figure 6 As shown, it can be:

[0073] Step S601: construct a fully connected neural network and set its input layer dimension to be equal to the feature vector dimension of the mask.

[0074] Step S602: Design the output layer dimension of the neural network in S601 to be equal to the dimension of the feature vector of the main contour points, which is the dimension of the craniotomy drilling point in this case.

[0075] Step S603: Use the root mean square error as the loss function to evaluate the performance of the expert decision model. The difference from formula (3) is that there is no need to introduce a weight correction factor, as shown in formula (13).

[0076] y=E θ (V), (12)

[0077]

[0078] in, is the expected output, i.e., the expert teaching data, and y is the actual output of the expert decision model.

[0079] Step 604: Similar to step S204, the parameters of the expert decision model E are tuned by the gradient descent method.

[0080]

[0081] Optionally, in one embodiment of the present application, the output result of the final expert decision model includes the spatial discreteness and positioning information of the main contour points of the surgical wound, wherein the parameter tuning formula of the final expert decision model may be but is not limited to:

[0082]

[0083] in, is the expected output, mask is the input of AE encoder, E θ is the expert decision model, and loss is the loss function.

[0084] From the above, it can be seen that the expert decision model trained in the embodiment of the present application is The corresponding feature vector of the main contour points of the surgical wound that conforms to the expert decision-making experience can be automatically calculated based on the input lesion feature vector. Among them, the output result of the final expert decision model can be the spatial discreteness and positioning information of the main contour points of the surgical wound. The former expresses the size of the surgical wound, and the latter expresses the location of the surgical wound. In addition, the embodiment of the present application can also obtain the parameter tuning formula of the final expert decision model, which can be but not limited to:

[0085]

[0086] in, is the expected output, mask is the input of AE encoder, E θ is the expert decision model, and loss is the loss function.

[0087] The working principle of the decision modeling method for assisting in planning an open surgical approach proposed in the embodiment of the present application is described in detail below in conjunction with a specific embodiment.

[0088] Figure 7 A flowchart of using an expert decision model as a clinical training system according to one embodiment of the present application.

[0089] Step S701: Segment and obtain a lesion image in the form of a mask.

[0090] Step S702: AE performs feature extraction.

[0091] Step S703: Quantitative representation of spatial positioning and spatial discreteness. That is, the embodiment of the present application can obtain [α, β, γ] based on the expert decision model. T and

[0092] Step S704: The novice plans out the main contour points.

[0093] Step S705: encoding of main contour points. That is, the embodiment of the present application can encode the main contour points to obtain [α n ,β n ,γ n ] T and

[0094] Step S706: Calculate the difference. In the embodiment of the present application, the quantitative representations of the spatial positioning and spatial discreteness of the expert and the novice can be calculated respectively to obtain the corresponding difference results, which can be expressed as:

[0095] D d =||[α,β,γ] T -[α n ,β n ,γ n ] T ||,(15)

[0096]

[0097] Step S707: Determine whether it is less than a specific value. That is, the embodiment of the present application can determine whether the difference result is less than a specific value, if it is less than the specific value, execute step S708; if it is not less than the specific value, execute step S704.

[0098] Step S708: End.

[0099] In summary, the embodiment itself can use the expert decision model as a training system. Specifically, after obtaining the lesion image in the form of a mask, the embodiment of the present application can respectively plan a set of drilling positions according to the lesion image in the form of a mask, and the expert can generate a set of coded drilling positions through the final expert decision model. Among them, the planning result of the novice can be the corresponding coding result obtained by the coding method of step S307 above. Furthermore, the embodiment of the present application can perform a difference between the two coding results to obtain a corresponding difference result. If the difference result is within a specific value range, it means that the planning result of the novice meets the expectations of the expert. On the contrary, the planning result of the novice in the embodiment of the present application can be further adjusted according to the coding difference until it meets expectations.

[0100] According to the decision model modeling method for assisting the planning of open surgical approaches proposed in the embodiment of the present application, a lesion image in the form of a mask can be obtained based on the patient's preoperative image segmentation, and then the lesion feature vector and the feature vector of the main contour points of the surgical wound can be obtained. The expert decision model is trained to establish a mapping between the patient's preoperative image features and the expert surgical decision features, and the final expert decision model for assisting the planning of open surgical approaches can be obtained. This allows novices to intuitively compare the differences in positioning and spatial discreteness between their planning results and the results output by the final expert decision model, so as to improve their clinical planning skills in a targeted manner and achieve the beneficial effect of training novices. This is a general solution for the special unstructured scenario of open surgical planning. As a result, the problems in the related technology that there is no direct application of expert experience, the doctor's subjective decision is still required to determine whether it is available, and it is difficult to achieve automated auxiliary surgical planning are solved.

[0101] Next, a decision model building device for assisting in planning an open surgical approach proposed in an embodiment of the present application will be described with reference to the accompanying drawings.

[0102] Figure 8 The present invention is a block diagram of a decision modeling device for assisting in planning an open surgical approach according to an embodiment of the present application.

[0103] like Figure 8 As shown, the decision model building device 10 for assisting in planning an open surgical approach includes: a generation module 100 , an acquisition module 200 and a construction module 300 .

[0104] The generation module 100 is used to perform lesion segmentation on the patient's preoperative image to obtain a lesion image in the form of a mask.

[0105] The acquisition module 200 is used to acquire the lesion feature vector of the patient's lesion based on the lesion image in the form of a mask, and to acquire the feature vector of the main contour points of the patient's surgical wound.

[0106] Construction module 300 is used to construct a neural network in the form of a fully connected / multi-layer perceptron as the expert decision model body, and use the lesion feature vector and the feature vector of the main contour point of the surgical wound to train the model to establish a mapping between the patient's preoperative imaging features and the expert surgical decision features, so as to obtain the final expert decision model for assisting the planning of open surgical approaches.

[0107] Optionally, in one embodiment of the present application, the acquisition module 200 includes: a construction unit.

[0108] Among them, the construction unit is used to construct an autoencoder neural network, wherein the autoencoder neural network includes a neural network encoder and a neural network decoder, the neural network encoder is used to reduce the dimension of the lesion from its original scale to a specific scale for feature extraction, and the neural network decoder restores the reduced features to the original lesion scale.

[0109] Optionally, in one embodiment of the present application, the acquisition module 200 includes: a generation unit.

[0110] The generation unit is used to spatially locate the main contour points of the surgical wound and quantitatively characterize the degree of spatial discreteness, so as to describe the position and size of the wound respectively and obtain the feature vector of the main contour points of the surgical wound.

[0111] Optionally, in one embodiment of the present application, the generating unit includes: an acquiring subunit, a calculating subunit and a generating subunit.

[0112] The acquisition subunit is used to determine the coordinates of the patient's specific anatomical landmark points based on the lesion image, and to acquire the coordinates of the main contour points taught by the expert.

[0113] The calculation subunit is used to establish a coordinate system of the specific anatomical landmark point based on the coordinates of the specific anatomical landmark point, and calculate the coordinates of the main contour point in the coordinate system to obtain the coordinates of the main contour point in the coordinate system.

[0114] The generating subunit is used to obtain the maximum physical scale in the coordinate system, so as to normalize the coordinates of the main contour points in the coordinate system and obtain the spatial distribution information of the normalized main contour points.

[0115] Optionally, in one embodiment of the present application, the output result of the final expert decision model includes the spatial discreteness and positioning information of the main contour points of the surgical serial port, wherein the parameter tuning formula of the final expert decision model may be but is not limited to:

[0116]

[0117] in, is the expected output, mask is the input of AE encoder, E θ is the expert decision model, and loss is the loss function.

[0118] It should be noted that the above explanation of the embodiment of the decision modeling method for assisting planning of an open surgical approach is also applicable to the decision modeling device for assisting planning of an open surgical approach of this embodiment, and will not be repeated here.

[0119] According to the decision modeling device for assisting in planning open surgical approaches proposed in the embodiment of the present application, a lesion image in the form of a mask can be obtained based on the patient's preoperative image segmentation, and then the lesion feature vector and the main contour point feature vector can be obtained, thereby training the expert decision model, establishing a mapping between the patient's preoperative image and the expert surgical decision, and obtaining the final expert decision model for assisting in planning open surgical approaches. This allows novices to intuitively compare the differences in positioning and spatial discreteness between their planning results and the results output by the final expert decision model, thereby improving their clinical planning skills in a targeted manner and achieving the beneficial effect of training novices. This is a general solution for the special unstructured scenario of open surgical planning. As a result, the problems in the related technology that there is no direct application of expert experience, the doctor's subjective decision is still required to determine whether it is available, and it is difficult to achieve automated assisted surgical planning are solved.

[0120] Fig. 9 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. The electronic device may include:

[0121] A memory 901 , a processor 902 , and a computer program stored in the memory 901 and executable on the processor 902 .

[0122] When the processor 902 executes the program, the decision model building method for assisting in planning an open surgical approach provided in the above embodiment is implemented.

[0123] Furthermore, the electronic device further comprises:

[0124] The communication interface 903 is used for communication between the memory 901 and the processor 902 .

[0125] The memory 901 is used to store computer programs that can be executed on the processor 902 .

[0126] The memory 901 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0127] If the memory 901, the processor 902 and the communication interface 903 are implemented independently, the communication interface 903, the memory 901 and the processor 902 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0128] Optionally, in a specific implementation, if the memory 901, the processor 902 and the communication interface 903 are integrated on a chip, the memory 901, the processor 902 and the communication interface 903 can communicate with each other through an internal interface.

[0129] The processor 902 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0130] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned decision modeling method for assisting in planning an open surgical approach.

[0131] An embodiment of the present application further provides a computer program product, including a computer program, which, when executed, implements the above decision modeling method for assisting in planning an open surgical approach.

[0132] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0133] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0134] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0135] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0136] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0137] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0138] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0139] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A decision modeling method for assisting planning of open surgical approaches, characterized in that: The following steps are involved: Perform lesion segmentation on the patient's preoperative images to obtain lesion images in the form of masks; Based on the lesion image in the form of a mask, obtaining a lesion feature vector of the patient, and obtaining a feature vector of main contour points of the surgical wound of the patient; A neural network in the form of a fully connected / multi-layer perceptron is constructed as the expert decision model body, and the lesion feature vector and the main contour point feature vector of the surgical incision are used for model training to establish a mapping between the patient's preoperative imaging features and the expert surgical decision features, thereby obtaining the final expert decision model for assisting in planning open surgical approaches.

2. The method according to claim 1, characterized in that The step of obtaining the lesion feature vector of the patient comprises: An autoencoder neural network is constructed, wherein the autoencoder neural network includes a neural network encoder and a neural network decoder, wherein the neural network encoder is used to reduce the dimension of the lesion from its original scale to a specific scale for feature extraction, and the neural network decoder restores the reduced features to the original lesion scale.

3. The method according to claim 1, characterized in that: The step of obtaining the main contour point feature vector of the surgical wound of the patient comprises: The main contour points of the surgical wound are spatially located and the spatial discreteness is quantitatively characterized to describe the position and size of the wound respectively, and the feature vector of the main contour points of the surgical wound is obtained.

4. The method according to claim 3, characterized in that The spatial positioning of the main contour points of the surgical wound and the quantitative characterization of the spatial discreteness include: Determine the coordinates of the patient's specific anatomical landmarks based on the lesion image, and obtain the coordinates of the main contour points taught by the expert; Establishing a coordinate system of the specific anatomical landmark points based on the coordinates of the specific anatomical landmark points, and calculating the coordinates of the main contour points in the coordinate system to obtain the coordinates of the main contour points in the coordinate system; The maximum physical scale in the coordinate system is obtained to normalize the coordinates of the main contour points in the coordinate system to obtain the spatial distribution information of the normalized main contour points.

5. The method according to claim 1, characterized in that The output result of the final expert decision model includes the spatial discreteness and positioning information of the main contour points of the surgical wound, wherein the parameter tuning formula of the final expert decision model is: in, is the expected output, mask is the input of AE encoder, E θ is the expert decision model, and loss is the loss function.

6. A decision modeling device for assisting in planning an open surgical approach, characterized in that: include: A generation module is used to segment the lesion on the patient's preoperative image to obtain a lesion image in the form of a mask; An acquisition module, used for acquiring a lesion feature vector of the patient based on the lesion image in the form of a mask, and acquiring a feature vector of main contour points of a surgical wound of the patient; A construction module is used to construct a neural network in the form of a fully connected / multi-layer perceptron as the expert decision model body, and use the lesion feature vector and the surgical wound main contour point feature vector to perform model training to establish a mapping between the patient's preoperative imaging features and the expert surgical decision features, so as to obtain the final expert decision model for assisting in planning the open surgical approach.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the decision modeling method for assisting in planning an open surgical approach as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement a decision modeling method for assisting in planning an open surgical approach as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that The computer program is executed to implement the decision modeling method for assisting in planning an open surgical approach according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • Remote operation guiding device based on Holols and operation method

    CN110931121A

  • Brain tumor operation planning risk assessment method and device based on deep learning

    CN117297771A

  • Auxiliary liver operation planning modeling system

    CN117576321A