Esophageal hiatus hernia preoperative decision support method based on multi-modal data

Abstract

The invention provides an esophageal hiatus hernia preoperative decision support method based on multi-modal data, and relates to the technical field of artificial intelligence and the technical field of intelligent medical treatment. The method comprises: obtaining lesion information of a plurality of lesion organs from a target image of a target object, the target image representing respective forms of the plurality of lesion organs and a position relationship among the plurality of lesion organs, and the lesion information comprising lesion form information, lesion diameter parameters and lesion proportion parameters related to the lesion organs, the diseased organ comprises the esophagus of the target object; obtaining health risk indication information of the target object from the treatment information of the target object, wherein the health risk indication information comprises disease indication information and curative effect indication information; the lesion information and the health risk indication information are input into a pre-operation auxiliary decision making model, a pre-operation auxiliary decision making is output, and the pre-operation auxiliary decision making comprises at least one of an emergency operation, a selective operation and conservative treatment.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence technology and smart medical technology, and more specifically, to a preoperative decision support method, system, electronic device, medium, and program product for hiatal hernia surgery based on multimodal data. Background Art

[0002] Hiatal hernia is a common digestive disorder in which the stomach protrudes into the chest cavity through the esophageal hiatus in the diaphragm. Simply put, the diaphragm, a flat muscle between the chest and abdomen, has a natural hole in the middle that allows the esophagus to pass through. When this hole enlarges abnormally, the stomach and even other organs, which were originally located in the abdominal cavity, can be squeezed into the chest cavity.

[0003] Currently, there is a lack of quantitative standards for the severity of hiatal hernia symptoms. Existing diagnosis and treatment decisions rely heavily on the doctor's experience. However, differences in doctors' experience may lead to errors or mistakes in the treatment indications for hiatal hernia, such as delayed treatment, aggravated symptoms, and more serious health problems. Summary of the Invention

[0004] In view of this, the present disclosure provides a method, system, electronic device, medium and program product for preoperative decision support for hiatal hernia surgery based on multimodal data.

[0005] One aspect of the present disclosure provides a preoperative decision support method for hiatal hernia surgery based on multimodal data, comprising:

[0006] Acquiring lesion information of a plurality of lesion organs from a target image of the target object, wherein the target image represents the morphology of each of the plurality of lesion organs and the positional relationship between the plurality of lesion organs, the lesion information including lesion morphology information, lesion diameter parameters, and lesion ratio parameters related to the lesion organs, the lesion organs including the esophagus of the target object;

[0007] Obtaining health risk indication information of the target subject from the medical treatment information of the target subject, wherein the health risk indication information includes symptom indication information and efficacy indication information;

[0008] The above-mentioned lesion information and the above-mentioned health risk indicator information are input into a preoperative decision-making support model, and a preoperative decision-making support model is output, wherein the above-mentioned preoperative decision-making support includes at least one of emergency surgery, elective surgery and conservative treatment.

[0009] Another aspect of the present disclosure provides a preoperative decision support system for hiatal hernia surgery based on multimodal data, comprising:

[0010] a first acquisition module, configured to acquire lesion information of a plurality of lesion organs from a target image of a target object, wherein the target image represents the morphology of each of the plurality of lesion organs and the positional relationship between the plurality of lesion organs, the lesion information including lesion morphology information, lesion diameter parameters, and lesion ratio parameters related to the lesion organs, and the lesion organs include the esophagus of the target object;

[0011] A second acquisition module is configured to acquire health risk indication information of the target subject from the medical treatment information of the target subject, wherein the health risk indication information includes symptom indication information and efficacy indication information; and

[0012] The first output module is used to input the above-mentioned lesion information and the above-mentioned health risk indicator information into a preoperative decision-making support model, and output a preoperative decision-making support model, wherein the above-mentioned preoperative decision-making support includes at least one of emergency surgery, elective surgery and conservative treatment.

[0013] Another aspect of the present disclosure provides an electronic device, comprising:

[0014] one or more processors;

[0015] a memory for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described above.

[0017] Another aspect of the present disclosure provides a computer-readable storage medium having executable instructions stored thereon. When the instructions are executed by a processor, the processor is caused to implement the method described above.

[0018] Another aspect of the present disclosure provides a computer program product, comprising a computer program, which implements the method described above when executed by a processor.

[0019] According to the embodiments of the present disclosure, by obtaining lesion information of multiple diseased organs from the target image of the target object, the uncertainty of judging lesion morphology information, lesion diameter parameters and lesion degree by observing the target image with the naked eye is reduced; by obtaining the health risk indicator information of the target object from the medical information of the target object, the speed of finding information related to esophageal hiatal hernia disease is accelerated, and the diagnostic efficiency of the doctor is improved; the lesion information and health risk indicator information are input into the preoperative decision support model, and the preoperative decision support model is output, which integrates the image information and the medical text information, so that the doctor can better understand the overall health status of the target object, and provides doctors with richer information resources in many aspects, which can not only help experienced doctors to understand the disease more deeply and weigh the pros and cons of different treatment plans, but also assist inexperienced doctors to make systematic treatment decision judgments, making the treatment and diagnosis of esophageal hiatal hernia disease more scientific, and partially avoiding the problem that the treatment indications of esophageal hiatal hernia may have errors or mistakes due to differences in doctors' experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0021] Figure 1 Schematically illustrates an application scenario diagram of a preoperative decision support method for hiatal hernia surgery based on multimodal data according to an embodiment of the present disclosure;

[0022] Figure 2 A flowchart of a method for preoperative decision support for hiatal hernia surgery based on multimodal data according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 3A The present invention schematically illustrates a process of preoperative decision support for hiatal hernia surgery based on multimodal data according to an embodiment of the present disclosure;

[0024] Figure 3B The following schematically illustrates the decision-making process of the repair method according to an embodiment of the present disclosure;

[0025] Figure 4 Schematically shows a flow chart for obtaining lesion information according to an embodiment of the present disclosure;

[0026] Figure 5A A schematic diagram of a medical record map according to an embodiment of the present disclosure is schematically shown;

[0027] Figure 5B A schematic diagram schematically illustrates query results obtained by querying diseases and clinical manifestations suffered by a target subject according to an embodiment of the present disclosure;

[0028] Figure 6Schematically shows a block diagram of a preoperative decision support system for hiatal hernia surgery based on multimodal data according to an embodiment of the present disclosure;

[0029] Figure 7 A block diagram of an electronic device for a preoperative decision support method for hiatal hernia surgery based on multimodal data according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0030] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0031] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0032] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0033] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0034] In the embodiments of this disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of all data involved (including, but not limited to, the personal information of target subjects) comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken with respect to the personal information of target subjects to prevent unauthorized access to user personal information and safeguard the personal information security of target subjects, network security, and national security.

[0035] In the embodiments of the present disclosure, authorization or consent of the target object is obtained before obtaining or collecting the target object's personal information.

[0036] The preoperative auxiliary decision obtained by the method provided in the embodiment of the present disclosure is used to assist qualified relevant personnel in making a diagnosis, and is not directly used for the diagnostic results of the target object. It complies with relevant laws and regulations and is in line with public order and good morals.

[0037] During the process of implementing the concepts disclosed herein, the inventors discovered that the severity of hiatal hernia symptoms is primarily determined by doctors visually based on CT images, which lacks quantitative standards and is highly subjective and uncertain. Furthermore, the disorganized medical records prevent doctors from quickly extracting the necessary information, resulting in low diagnostic efficiency. There is also a lack of standardized intelligent assessment systems for the treatment and diagnosis of hiatal hernia, particularly in areas such as preoperative assessment of the need for surgery, which lacks systematic data support.

[0038] In view of this, the embodiments of the present disclosure provide a method, system, device, medium and program product for preoperative decision support for hiatal hernia based on multimodal data. The method includes: obtaining lesion information of multiple diseased organs from a target image of a target object, wherein the target image characterizes the morphology of each of the multiple diseased organs and the positional relationship between the multiple diseased organs, and the lesion information includes lesion morphology information, lesion diameter parameters and lesion ratio parameters related to the lesion organs, and the lesion organs include the esophagus of the target object; obtaining health risk indication information of the target object from the medical information of the target object, wherein the health risk indication information includes symptom indication information and efficacy indication information; inputting the lesion information and health risk indication information into a preoperative decision support model, and outputting a preoperative decision support model, wherein the preoperative decision support model includes at least one of emergency surgery, elective surgery and conservative treatment.

[0039] Figure 1 The following schematically illustrates an application scenario diagram of a preoperative decision support method for hiatal hernia surgery based on multimodal data according to an embodiment of the present disclosure. Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0040] like Figure 1As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0041] A user may use a first terminal device 101, a second terminal device 102, or a third terminal device 103 to interact with a server 105 via a network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, or the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).

[0042] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0043] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal devices.

[0044] It should be noted that the preoperative decision support method for hiatal hernia based on multimodal data provided by the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the preoperative decision support system for hiatal hernia based on multimodal data provided by the embodiment of the present disclosure can generally be set in the server 105. The preoperative decision support method for hiatal hernia based on multimodal data provided by the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the preoperative decision support system for hiatal hernia based on multimodal data provided by the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the multimodal data-based preoperative decision support method for hiatal hernia provided in the embodiment of the present disclosure may also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or may also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the multimodal data-based preoperative decision support system for hiatal hernia provided in the embodiment of the present disclosure may also be provided in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or may be provided in other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0045] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0046] Figure 2 The flowchart of the preoperative decision support method for hiatal hernia surgery based on multimodal data according to an embodiment of the present disclosure is schematically shown.

[0047] like Figure 2 As shown, the method includes operations S201 to S203.

[0048] In operation S201 , lesion information of a plurality of lesion organs is acquired from a target image of a target object.

[0049] According to an embodiment of the present disclosure, the target image represents the morphology of each of the multiple diseased organs and the positional relationship between the multiple diseased organs. The lesion information includes lesion morphology information, lesion diameter parameters and lesion proportion parameters related to the lesion organs. The lesion organs include the esophagus of the target object.

[0050] In an embodiment of the present disclosure, the target object may be a patient suffering from hiatal hernia; the target image may be an image obtained after a medical examination of the target object, such as a CT image; the lesion organ includes the esophagus of the target object and other organs related to hiatal hernia, such as the stomach. The lesion information includes lesion morphology information of the esophagus and stomach, a lesion diameter parameter, and a lesion ratio parameter. The lesion diameter parameter may be the diameter data of the esophageal hiatus, and the lesion ratio parameter may be the herniation ratio of the stomach. It can be understood that the part above the esophageal hiatus is the herniated part of the stomach, and the part below the esophageal hiatus is the non-herniated part of the stomach. The herniation ratio of the stomach is the ratio of the volume of the herniated part of the stomach to the total volume of the stomach.

[0051] In the disclosed embodiments, a CT scan is performed on a target subject to obtain a CT image. The CT image represents the morphology of the esophagus and stomach, as well as the positional relationship between the esophagus and stomach. Esophageal and gastric lesion information is obtained from the CT image of the target subject, including lesion morphology information, lesion diameter parameters, and lesion ratio parameters.

[0052] In operation S202 , health risk indicator information of the target object is acquired from the medical consultation information of the target object.

[0053] According to an embodiment of the present disclosure, the health risk indication information includes symptom indication information and efficacy indication information.

[0054] In the embodiments of the present disclosure, medical information includes the target subject's personal information, information about the target subject's disease, information about the target subject's clinical manifestations, information about the medical examination items completed by the target subject, information about the target subject's treatment methods, information about the target subject's treatment effects, information about the target subject's treatment duration, etc., which are not listed one by one here. Efficacy indication information can be information about the target subject's treatment effects before preoperative decision-making assistance, and health risk indication information can include information about the target subject's disease indications and information about the target subject's treatment effects before preoperative decision-making assistance.

[0055] In the embodiments of the present disclosure, obtaining the health risk indication information of the target object from the medical information of the target object can be understood as obtaining the disease information suffered by the target object, the clinical manifestation information of the target object, the treatment effect information of the target object, etc. from the medical information of the target object.

[0056] In operation S203, the lesion information and health risk indicator information are input into a preoperative decision support model, and a preoperative decision support model is output.

[0057] According to an embodiment of the present disclosure, preoperative decision support includes at least one of emergency surgery, elective surgery, and conservative treatment.

[0058] In the embodiments of the present disclosure, the preoperative decision-making support model is constructed based on a machine learning algorithm, such as a decision tree model, a random forest model, a support vector machine (SVM), or a logistic regression model. However, this is not limited to these algorithms. Preoperative decision-making support models can also be constructed based on other types of algorithms, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), or Transformer architectures, graph neural networks (GNNs), or gradient boosting trees. Furthermore, a hybrid model combining multiple algorithms can be used to further improve model accuracy.

[0059] In an embodiment of the present disclosure, the preoperative decision support model determines whether the target object needs surgery based on the lesion information and health risk indicator information of the target object, and outputs a preoperative decision support model, namely, at least one of emergency surgery, elective surgery and conservative treatment.

[0060] In an embodiment of the present disclosure, by obtaining lesion information from the target image of the target object, and using a preoperative decision support model to process the lesion information and obtain the target object's health risk indication information from the target patient's medical information, the preoperative decision support model determines whether the target object needs surgery based on the above information, and outputs a preoperative decision support, i.e., at least one of emergency surgery, elective surgery, and conservative treatment.

[0061] According to the embodiments of the present disclosure, by obtaining lesion information of multiple diseased organs from the target image of the target object, the uncertainty of judging lesion morphology information, lesion diameter parameters and lesion degree by observing the target image with the naked eye is reduced; by obtaining the health risk indicator information of the target object from the medical information of the target object, the speed of finding information related to esophageal hiatal hernia disease is accelerated, and the diagnostic efficiency of the doctor is improved; the lesion information and health risk indicator information are input into the preoperative decision support model, and the preoperative decision support model is output, which integrates the image information and the medical text information, so that the doctor can better understand the overall health status of the target object, and provides doctors with richer information resources in many aspects, which can not only help experienced doctors to understand the disease more deeply and weigh the pros and cons of different treatment plans, but also assist inexperienced doctors to make systematic treatment decision judgments, making the treatment and diagnosis of esophageal hiatal hernia disease more scientific, and partially avoiding the problem that the treatment indications of esophageal hiatal hernia may have errors or mistakes due to differences in doctors' experience.

[0062] According to an embodiment of the present disclosure, the above-mentioned disease indication information includes acute disease information, basic disease information and target symptom information related to hiatal hernia disease; the above-mentioned lesion information and the above-mentioned health risk indication information are input into the preoperative decision support model, and the output of the preoperative decision support includes: according to the above-mentioned acute disease information, using the above-mentioned preoperative decision support model to judge whether the above-mentioned patient has acute indications, and obtain a first judgment result. According to an embodiment of the present disclosure, when the above-mentioned first judgment result is yes, the above-mentioned preoperative decision support is output as emergency surgery; when the above-mentioned first judgment result is no, according to the above-mentioned basic disease information, using the above-mentioned preoperative decision support model to judge whether the patient has basic health problems, and obtain a second judgment result. According to an embodiment of the present disclosure, when the above-mentioned second judgment result is yes, the above-mentioned preoperative decision support is output as conservative treatment; when the above-mentioned second judgment result is no, according to the above-mentioned lesion morphology information, using the above-mentioned preoperative decision support model to output the above-mentioned preoperative decision support.

[0063] In an embodiment of the present disclosure, acute indications may indicate acute conditions such as intestinal obstruction, gastric torsion and gastric bleeding. Basic health problems may include pathological changes or dysfunctions of organs such as the heart, liver, and lungs, as well as abnormalities of blood, immune and other systems. Symptom indication information includes acute symptom information, basic symptom information and target symptom information related to hiatal hernia disease. Acute symptom information may indicate acute indications such as intestinal obstruction, gastric torsion and gastric bleeding, and the acute symptom information may be obtained from the medical information of the above-mentioned target object, such as the clinical manifestation information of the target object; basic symptom information may indicate basic health problems such as pathological changes or dysfunctions of organs such as the heart, liver, and lungs, as well as abnormalities of blood, immune and other systems, and the basic symptom information may be obtained from the disease information suffered by the above-mentioned target object.

[0064] In the embodiments of the present disclosure, by judging the acute indications and basic health problems of the target subject, the doctor can better understand the overall health status of the target subject, have a deeper understanding of the condition, and help the doctor weigh the pros and cons of different treatment plans. This can partially avoid the problem of not performing surgery in time due to ignoring the target subject's acute indications such as gastric bleeding, gastric torsion or intestinal obstruction, and can also partially avoid the problem of ignoring the target subject's heart disease, liver disease or lung disease, thereby greatly increasing the risk of surgery.

[0065] According to an embodiment of the present disclosure, the above-mentioned lesion morphology information includes one of the paraesophageal type and the sliding type; the above-mentioned preoperative decision-making support model is used to output the above-mentioned preoperative decision-making support based on the above-mentioned lesion morphology information, including: when the above-mentioned lesion morphology information is the above-mentioned paraesophageal type, according to the above-mentioned efficacy indication information, the above-mentioned preoperative decision-making support model is used to judge whether the treatment is effective, and a third judgment result is obtained. According to an embodiment of the present disclosure, when the above-mentioned third judgment result is yes, the above-mentioned preoperative decision-making support is output as conservative treatment, and when the above-mentioned third judgment result is no, the above-mentioned preoperative decision-making support is output as elective surgery; when the above-mentioned lesion morphology information is the above-mentioned sliding type, according to the above-mentioned target symptom information, the above-mentioned preoperative decision-making support model is used to judge whether the above-mentioned patient has the target symptom, and a fourth judgment result is obtained. According to an embodiment of the present disclosure, when the above-mentioned fourth judgment result is yes, the above-mentioned preoperative decision-making support is output as elective surgery, and when the above-mentioned fourth judgment result is no, the above-mentioned preoperative decision-making support is output as conservative treatment.

[0066] In an embodiment of the present disclosure, the lesion morphology information includes one of the paraesophageal type and the sliding type. The paraesophageal type may be a paraesophageal hiatal hernia, and the sliding type may be a sliding hiatal hernia. The efficacy indication information may indicate whether the treatment of the target object is effective before preoperative auxiliary decision-making. The efficacy indication information may be obtained from the treatment effect information of the target object. The target symptom information may indicate whether the target object has the target symptom. The target symptom information may be obtained from the clinical manifestation information of the target object.

[0067] Figure 3A The diagram schematically illustrates the process of preoperative decision-making assistance for esophageal diseases according to an embodiment of the present disclosure.

[0068] like Figure 3AAs shown, first, a judgment is made as to whether the target subject has an acute indication D301, i.e., the first judgment. If the target subject has an acute indication D301, i.e., the first judgment result is yes, the preoperative auxiliary decision is output as emergency surgery 302. If the target subject does not have an acute indication, i.e., the first judgment result is no, then a judgment is made as to whether the target subject has underlying health problems D303, i.e., the second judgment. If the target subject has underlying health problems D303, i.e., the second judgment result is yes, the preoperative auxiliary decision is output as conservative treatment 305. If the target subject does not have underlying health problems D303, i.e., the second judgment result is no, then a judgment is made based on the target subject's lesion morphology information D304. In the case where the target object's lesion morphology information D304 is of the paraesophageal type, the treatment effect of the target object before the preoperative auxiliary decision is judged, i.e., the third judgment. If the treatment of the target object before the preoperative auxiliary decision is effective, i.e., the result of the third judgment is yes, then the treatment can be continued, and the preoperative auxiliary decision is output as conservative treatment 305; if the treatment of the target object before the preoperative auxiliary decision is ineffective, i.e., the result of the third judgment is no, and the preoperative auxiliary decision is output as elective surgery 308. In the case where the target object's lesion morphology information is of the sliding type, a judgment is made as to whether the target object has the target symptom D307, i.e., the fourth judgment. If the target object has the target symptom D307, i.e., the result of the fourth judgment is yes, and the preoperative auxiliary decision is output as elective surgery 308; if the target object does not have the target symptom D307, i.e., the result of the fourth judgment is no, and the preoperative auxiliary decision is output as conservative treatment 305.

[0069] In the embodiment of the present disclosure, by considering the lesion morphology information of the target object to determine the type of esophageal hiatal hernia of the target object, it can assist inexperienced doctors to make systematic treatment decision judgments, making the treatment and diagnosis of esophageal hiatal hernia disease more scientific.

[0070] According to an embodiment of the present disclosure, when the above-mentioned preoperative decision support includes at least one of emergency surgery and elective surgery, the above-mentioned preoperative decision support model also determines the repair method based on the above-mentioned lesion diameter parameter and the above-mentioned lesion ratio parameter, and the above-mentioned repair method includes at least one of an ordinary repair method and a patch repair method.

[0071] In the embodiments of the present disclosure, a conventional repair method can be understood as conventional suturing, in which tissue edges are directly aligned or closed using sutures. Conventional repair does not involve the use of a patch or other auxiliary materials; tissue closure is achieved solely through sutures. A patch repair method can be understood as patch suturing, in which a biological or synthetic patch is placed in the tissue defect area and secured to the tissue using sutures.

[0072] In the embodiments of the present disclosure, in order to determine the repair method, thresholds can be set for the lesion diameter parameter and the lesion ratio parameter. For example, the lesion diameter parameter threshold can be set to 5 cm. When the diameter of the lesion portion is greater than or equal to 5 cm, a patch repair method can be used; when the diameter of the lesion portion is less than 5 cm, a conventional repair method can be used. For example, the lesion ratio parameter threshold can be set to 1 / 3. When the proportion of the lesion portion volume is greater than or equal to 1 / 3, a patch repair method can be used; when the proportion of the lesion portion volume is less than 1 / 3, a conventional repair method can be used. The threshold settings for the lesion diameter parameter and the lesion ratio parameter can be adjusted according to actual conditions.

[0073] Figure 3B The decision process of the repair method according to the embodiment of the present disclosure is schematically shown.

[0074] like Figure 3B As shown, first, it is judged whether the lesion diameter parameter D309 of the lesion organ is greater than the preset lesion diameter parameter threshold, that is, the fifth judgment. When the preset lesion diameter parameter threshold is 5 cm, it is judged whether the lesion diameter parameter of the lesion organ is greater than or equal to 5 cm. If the lesion diameter parameter D309 of the lesion organ is greater than or equal to 5 cm, that is, the result of the fifth judgment is yes, then it means that the diameter of the lesion part of the lesion organ is too large and cannot be repaired by the ordinary repair method 312, and the preoperative auxiliary decision is output as the patch repair method 310; if the lesion diameter parameter D309 of the lesion organ is less than 5 cm, that is, the result of the fifth judgment is no, then it means that the lesion part of the lesion organ does not need to be repaired by the patch repair method 310, then continue to judge whether the lesion ratio parameter D311 of the lesion organ is greater than the preset lesion ratio parameter threshold, that is, the sixth judgment. When the preset lesion ratio parameter threshold is 1 / 3, it is determined whether the lesion ratio parameter D311 of the lesion organ is greater than or equal to 1 / 3. If the lesion ratio parameter D311 of the lesion organ is greater than or equal to 1 / 3, that is, the sixth judgment is yes, then it means that the volume of the lesion part of the lesion organ is too large and cannot be repaired by the ordinary repair method 312, and the preoperative auxiliary decision output is the patch repair method 403; if the lesion ratio parameter D311 of the lesion organ is less than 1 / 3, that is, the sixth judgment is no, then it means that the volume of the lesion part of the lesion organ is slightly smaller and can be repaired by the ordinary repair method 312, and the preoperative auxiliary decision output is the ordinary repair method 312. Figure 3B The setting of the lesion diameter parameter threshold and the lesion ratio parameter threshold is only an example. The setting of the specific lesion diameter parameter threshold and the lesion ratio parameter threshold needs to be determined according to actual conditions.

[0075] In the embodiment of the present disclosure, by setting the lesion diameter parameter threshold and the lesion ratio parameter threshold respectively, doctors can avoid judging the diameter and volume of the diseased part of the diseased organ by naked eyes, thereby reducing the uncertainty and subjectivity of doctors in the treatment and diagnosis process and making treatment decisions more scientific.

[0076] Figure 4 The flowchart of obtaining lesion information according to an embodiment of the present disclosure is schematically shown.

[0077] like Figure 4 As shown, the method includes operations S401 to S404.

[0078] In operation S401 , image regions of multiple diseased organs in a target image are segmented to obtain multiple organ contours of the multiple diseased organs.

[0079] In the embodiment of the present disclosure, according to the gray value distribution characteristics of multiple diseased organs in the target image, the gray threshold method is used to segment the diseased organs of the target object to obtain multiple organ contours of the multiple diseased organs.

[0080] In the embodiments disclosed herein, the grayscale value of the lower-density tissue in the diseased organ is lower, and the organ contour obtained by segmenting the image region with lower grayscale values may be discontinuous. To avoid discontinuous organ contours, the image regions corresponding to the lower-density tissue in the target image are filled to fill in the blank areas within the organ contour. After filling, pixels outside the organ contour are erased to ensure that only the region corresponding to the diseased organ is retained, while removing background and irrelevant tissue.

[0081] In operation S402 , a plurality of three-dimensional organ models representing a plurality of diseased organs are determined based on a plurality of organ contours.

[0082] In the embodiments of the present disclosure, three-dimensional reconstruction technology is used to reconstruct three-dimensional models of the above-mentioned multiple organ contours to obtain three-dimensional organ models of multiple diseased organs, such as surface rendering methods, volume rendering methods or deep learning algorithms, etc., which are not limited here.

[0083] In an embodiment of the present disclosure, the three-dimensional organ model may be further optimized, for example, by performing surface filling, enveloping, and smoothing processing on the three-dimensional organ model to optimize the appearance and details of the three-dimensional organ model.

[0084] In operation S403 , a target positional relationship between the plurality of three-dimensional organ models is determined based on a positional relationship between the image regions of the plurality of diseased organs in the target image.

[0085] In the embodiments of the present disclosure, a three-dimensional model can be reconstructed for each diseased organ's contour to obtain a three-dimensional organ model of each diseased organ. The target positional relationship between the three-dimensional organ models of the multiple diseased organs can then be determined based on the position of the image region of each diseased organ in the target image and the relationship between the multiple diseased organs. For example, the three-dimensional model of the esophagus and the stomach can be reconstructed for each contour to obtain a three-dimensional organ model of the esophagus and a three-dimensional organ model of the stomach. The target positional relationship between the three-dimensional organ models of the esophagus and the stomach can then be determined based on the positional relationship between the corresponding image regions of the esophagus and the stomach in the target image.

[0086] In the disclosed embodiments, it is also possible to reconstruct a 3D model of the organ contours of multiple diseased organs as a whole. For example, based on the positional relationship between the corresponding image regions of the esophagus and stomach in the target image, the organ contours of the esophagus and stomach are fixed to the same positional relationship. Then, a 3D model of the organ contours of the esophagus and stomach with the fixed positional relationship is reconstructed using 3D reconstruction technology, directly obtaining a 3D organ model of the esophagus and stomach with the fixed positional relationship.

[0087] In operation S404, lesion information is determined based on the relationship between the plurality of three-dimensional organ models and the target position.

[0088] In an embodiment of the present disclosure, lesion morphological information, lesion diameter parameters, and lesion proportion parameters of the lesion organ are determined based on the relationship between multiple three-dimensional organ models and target positions. Lesion morphological information is determined based on the relationship between the morphology and target positions of multiple three-dimensional organ models. For example, if the gastroesophageal junction in the three-dimensional organ model is displaced to the diaphragm, the stomach still maintains its normal longitudinal position, and the fundus is still below the gastroesophageal junction, it can be concluded that the lesion morphological information is a sliding type. For example, if the gastroesophageal junction is in a normal position, but the fundus is displaced to the top of the diaphragm through the enlarged esophageal hiatus and into the chest cavity, it can be concluded that the lesion morphological information is a paraesophageal type.

[0089] In the embodiments disclosed herein, a lesion diameter parameter and a lesion ratio parameter are determined based on the model area corresponding to the lesion portion in the three-dimensional organ model. For example, the lesion diameter parameter can be obtained by measuring the diameter of the esophageal hiatus in the three-dimensional organ model. For example, a curve cutting tool can be used to segment the stomach along the surface where the esophageal hiatus is located. The area above the esophageal hiatus is the herniated portion of the stomach, and the area below the esophageal hiatus is the unherniated portion of the stomach. The ratio of the volume of the herniated portion to the total stomach volume is calculated to obtain the lesion ratio parameter.

[0090] In the embodiment of the present disclosure, the above operations can be completed using image processing software.

[0091] In the embodiments of the present disclosure, lesion morphology information, lesion diameter parameters and lesion proportion parameters are obtained through a three-dimensional organ model, which reduces the uncertainty of judging lesion morphology information, lesion diameter parameters and lesion extent by observing the target image with the naked eye, making the treatment and diagnosis of hiatal hernia more scientific, and partially avoiding the problem of errors or mistakes in the treatment indications for hiatal hernia due to differences in doctors' experience.

[0092] According to an embodiment of the present disclosure, the above-mentioned medical information includes a medical record map related to the above-mentioned target object; the above-mentioned obtaining of the health risk indication information of the above-mentioned target object from the medical information of the above-mentioned target object includes: querying the first target entity with an acute indication attribute from the above-mentioned medical record map, and determining the above-mentioned acute disease information based on the above-mentioned first target entity; querying the second target entity with a basic health problem attribute from the above-mentioned medical record map, and determining the above-mentioned basic disease information based on the above-mentioned second target entity; querying the third target entity with a target symptom attribute from the above-mentioned medical record map, and determining the above-mentioned target symptom information related to hiatal hernia disease based on the above-mentioned third target entity; querying the fourth target entity with a therapeutic effect attribute from the above-mentioned medical record map, and determining the above-mentioned therapeutic effect indication information based on the above-mentioned fourth target entity.

[0093] In an embodiment of the present disclosure, a medical record map can be constructed by data processing software, and the key information required to construct the medical record map can be extracted from the medical records through an extraction model. For example, entities can be defined as six categories, namely patient information, disease, clinical manifestation, examination, time, and treatment; entity relationships can be defined as eight categories, namely, suffering from, existing, accepted, discovered, caused, used for, treatment effect, and duration of action. The extraction model is used to jointly extract the entities and entity relationships in the medical records of the target object. The entity definition and entity relationship definition in this disclosure are only an example, and the specific definition of entities and entity relationships can be adjusted according to actual conditions.

[0094] In the embodiments of the present disclosure, the entity and entity relationship extraction model can be a PFN network model, but is not limited to this. It can also be other deep learning models such as RNN, LSTM, traditional machine learning models such as SVM, or a hybrid model.

[0095] In an embodiment of the present disclosure, the first target entity may include one or more of a disease and a clinical manifestation, and an acute indication indicates an acute condition of the target subject, such as gastric bleeding, gastric volvulus, or intestinal obstruction. One or more of the disease and clinical manifestation of the target subject is queried from the medical record map, and based on the query result, it is determined whether the target subject has an acute condition such as gastric bleeding, gastric volvulus, or intestinal obstruction.

[0096] In an embodiment of the present disclosure, the second target entity may be one or more of a disease and a clinical manifestation. The basic health question indicates that the target subject has underlying conditions related to organs such as the heart, liver, and lungs. One or more of the target subject's diseases and clinical manifestations are queried from the medical record map, and based on the query results, it is determined whether the target subject has underlying conditions related to organs such as the heart, liver, and lungs.

[0097] In an embodiment of the present disclosure, the third target entity may be one or more of a disease and a clinical manifestation, and the target symptom information may indicate that the target subject has typical symptoms such as acid reflux and heartburn. One or more of the target subject's diseases and / or clinical manifestations are queried from the medical record map, and based on the query results, it is determined whether the target subject has typical symptoms such as acid reflux and heartburn.

[0098] In an embodiment of the present disclosure, the fourth target entity may be one or more of a disease and a clinical manifestation, and the efficacy indicator information indicates whether the treatment of the target subject is effective. One or more of the disease and clinical manifestation of the target subject is queried from the medical record map, and whether the treatment of the target subject is effective is determined based on the query result.

[0099] In an embodiment of the present disclosure, efficacy indicator information can be directly obtained through clinical manifestations, for example, the medical record map directly includes information on the improvement of esophageal hiatal hernia; it can also be obtained indirectly through diseases and clinical manifestations, for example, the medical record map includes the frequency of vomiting of the target subject two months ago, the treatment method and the current frequency of vomiting. Based on the query results, it is found that after treatment, the frequency of vomiting of the target subject now is significantly less than the frequency of vomiting two months ago, then it can be determined that the treatment of the target subject is effective.

[0100] Figure 5A A schematic diagram of a medical record map according to an embodiment of the present disclosure is schematically shown.

[0101] like Figure 5AAs shown, the medical record graph, with target object P506 as the core node, connects to other nodes via edges, indicating target object P506's diseases, clinical manifestations, treatments, and examinations. Target object P506 is connected to nodes Hypertension D502, Hiatal Hernia D505, and Heart Disease D508 via edges, forming a Suffering relationship, indicating that target object P506 suffers from Hypertension D502, Hiatal Hernia D505, and Heart Disease D508. Target object P506 is connected to nodes Gastric Bleeding D509, Choking D507, and Acid Regurgitation D503 via edges, forming an Exists relationship, indicating that target object P506 suffers from symptoms of Gastric Bleeding D509, Choking D507, and Acid Regurgitation D503. The acid suppression drug node M501 is connected to the acid reflux node D503 via an edge, indicating that acid suppression drug M501 treats the acid reflux symptom D503 of target subject P506. The acid suppression drug node M501 has a therapeutic effect relationship with the acid reflux node D503, indicating that acid suppression drug M501 has a therapeutic effect on the acid reflux symptom D503 of target subject P506. The target object node P506 is connected to the CT examination node 504 via an edge, having an accept relationship, indicating that target subject P506 received CT examination 504.

[0102] In the embodiment of the present disclosure, the target subject P506 suffers from hypertension D502, hiatal hernia D505, and heart disease D508, and has symptoms of gastric bleeding D509, choking D507, and acid reflux D503. The target subject P506 has undergone a CT scan 504 and has been treated with acid-suppressing drug M501.

[0103] In an embodiment of the present disclosure, a doctor can query key information in a targeted manner based on the medical record map. For example, a doctor can query the disease information and clinical manifestation information of the target subject P506.

[0104] Figure 5B The diagram schematically shows query results obtained by querying diseases and clinical manifestations suffered by a target object according to an embodiment of the present disclosure.

[0105] like Figure 5BAs shown, the query results show the target object P506 as the core node, connected to other nodes via edges, indicating the target object P506's diseases and clinical manifestations. Target object P506 is connected to Hypertension D502, Hiatal Hernia D505, and Heart Disease D508 via edges, forming a Suffering relationship, indicating that target object P506 suffers from Hypertension D502, Hiatal Hernia D505, and Heart Disease D508. Target object P506 is connected to Gastric Bleeding D509, Choking D507, and Acid Regurgitation D503 via edges, forming an Exists relationship, indicating that target object P506 suffers from Gastric Bleeding D509, Choking D507, and Acid Regurgitation D503. According to the query results, the doctor can obtain the following information: the target subject P506 suffers from hypertension D502, hiatal hernia D505 and heart disease D508, and has symptoms of gastric bleeding D509, choking D507 and acid reflux D503.

[0106] In the embodiment of the present disclosure, acute disease information, basic disease information, target symptom information and efficacy indication information are obtained based on the medical record map, which speeds up the search for information related to esophageal hiatal hernia disease and improves the doctor's diagnostic efficiency.

[0107] Figure 6 Schematic diagram of a preoperative decision support system for hiatal hernia surgery based on multimodal data according to an embodiment of the present disclosure

[0108] like Figure 6 As shown, the multimodal data-based preoperative decision support system 600 for hiatal hernia includes a first acquisition module 610 , a second acquisition module 620 and a first output module 630 .

[0109] The first acquisition module 610 is used to obtain lesion information of multiple diseased organs from a target image of the target object, wherein the above-mentioned target image represents the morphology of each of the multiple diseased organs and the positional relationship between the multiple diseased organs. The above-mentioned lesion information includes lesion morphology information, lesion diameter parameters and lesion ratio parameters related to the above-mentioned diseased organs, and the above-mentioned diseased organs include the esophagus of the above-mentioned target object.

[0110] The second acquisition module 620 is used to acquire the health risk indication information of the target object from the medical information of the target object, wherein the health risk indication information includes symptom indication information and efficacy indication information.

[0111] The first output module 630 inputs the above-mentioned lesion information and the above-mentioned health risk indicator information into a preoperative decision support model, and outputs a preoperative decision support model, wherein the above-mentioned preoperative decision support model includes at least one of emergency surgery, elective surgery and conservative treatment.

[0112] According to the embodiments of the present disclosure, by obtaining lesion information of multiple diseased organs from the target image of the target object, the uncertainty of judging lesion morphology information, lesion diameter parameters and lesion degree by observing the target image with the naked eye is reduced; by obtaining the health risk indicator information of the target object from the medical information of the target object, the speed of finding information related to esophageal hiatal hernia disease is accelerated, and the diagnostic efficiency of the doctor is improved; the lesion information and health risk indicator information are input into the preoperative decision support model, and the preoperative decision support model is output, which integrates the image information and the medical text information, so that the doctor can better understand the overall health status of the target object, and provides doctors with richer information resources in many aspects, which can not only help experienced doctors to understand the disease more deeply and weigh the pros and cons of different treatment plans, but also assist inexperienced doctors to make systematic treatment decision judgments, making the treatment and diagnosis of esophageal hiatal hernia disease more scientific, and partially avoiding the problem that the treatment indications of esophageal hiatal hernia may have errors or mistakes due to differences in doctors' experience.

[0113] According to an embodiment of the present disclosure, the above-mentioned disease indication information includes acute disease information, basic disease information and target symptom information related to hiatal hernia disease.

[0114] According to an embodiment of the present disclosure, the first output module 630 includes a first output submodule, a second output submodule, and a third output submodule.

[0115] The first output submodule is used to determine whether the patient has acute indications based on the acute disease information and the preoperative decision support model, and obtain a first judgment result. When the first judgment result is yes, the preoperative decision support is output as emergency surgery.

[0116] The second output submodule is used to determine whether the patient has underlying health problems based on the above-mentioned basic disease information and the above-mentioned preoperative decision-making support model when the above-mentioned first judgment result is no, and obtain a second judgment result, wherein when the above-mentioned second judgment result is yes, the above-mentioned preoperative decision-making support is output as conservative treatment.

[0117] The third output submodule is used to output the above-mentioned preoperative auxiliary decision using the above-mentioned preoperative auxiliary decision model based on the above-mentioned lesion morphology information when the above-mentioned second judgment result is no.

[0118] According to an embodiment of the present disclosure, the above-mentioned lesion morphology information includes one of the paraesophageal type and the sliding type.

[0119] According to an embodiment of the present disclosure, the third output submodule includes a first output unit and a second output unit.

[0120] The first output unit is used to determine whether the treatment is effective by using the above-mentioned preoperative decision-making support model according to the above-mentioned efficacy indication information when the above-mentioned lesion morphology information is the above-mentioned paraesophageal type, and obtain a third judgment result, wherein when the above-mentioned third judgment result is yes, the above-mentioned preoperative decision-making support is output as conservative treatment, and when the above-mentioned third judgment result is no, the above-mentioned preoperative decision-making support is output as elective surgery.

[0121] The second output unit is used to determine whether the patient has the target symptoms using the preoperative decision support model when the lesion morphology information is of the sliding type, based on the target symptom information, to obtain a fourth judgment result, wherein when the fourth judgment result is yes, the preoperative decision support is output as elective surgery; and when the fourth judgment result is no, the preoperative decision support is output as conservative treatment.

[0122] According to an embodiment of the present disclosure, the multimodal data-based preoperative decision support system 600 for hiatal hernia further includes a second output module.

[0123] The second output module is used for determining the repair method according to the lesion diameter parameter and the lesion ratio parameter when the preoperative decision support model includes at least one of an emergency surgery and an elective surgery, and the repair method includes at least one of an ordinary repair method and a patch repair method.

[0124] According to an embodiment of the present disclosure, the first acquisition module 610 includes a segmentation submodule, a first determination submodule, a second determination submodule, and a third determination submodule.

[0125] The segmentation submodule is used to segment the image regions of the plurality of diseased organs in the target image to obtain a plurality of organ contours of the plurality of diseased organs.

[0126] The first determination submodule is used to determine a plurality of three-dimensional organ models representing the plurality of diseased organs based on the plurality of organ contours.

[0127] The second determining submodule is configured to determine a target positional relationship between the plurality of three-dimensional organ models based on a positional relationship between the image regions of the plurality of diseased organs in the target image.

[0128] The third determining submodule is used to determine the lesion information according to the relationship between the plurality of three-dimensional organ models and the target position.

[0129] According to an embodiment of the present disclosure, the second acquisition module 620 includes a fourth determination submodule, a fifth determination submodule, a sixth determination submodule, and a seventh determination submodule.

[0130] The fourth determination submodule is used to query the first target entity with an acute indication attribute from the above-mentioned medical record map, and determine the above-mentioned acute disease information based on the above-mentioned first target entity.

[0131] The fifth determination submodule is used to query the second target entity with basic health problem attributes from the above-mentioned medical record map, and determine the above-mentioned basic disease information based on the above-mentioned second target entity.

[0132] The sixth determination submodule is used to query the third target entity with the target symptom attribute from the above-mentioned medical record map, and determine the above-mentioned target symptom information related to the hiatal hernia disease based on the above-mentioned third target entity.

[0133] The seventh determination submodule is used to query the fourth target entity with the treatment effect attribute from the above-mentioned medical record map, and determine the above-mentioned treatment effect indication information based on the above-mentioned fourth target entity.

[0134] According to the embodiments of the present invention, any number of modules, sub-modules, units, and sub-units, or at least part of the functions of any number of them, can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, sub-modules, units, and sub-units can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, sub-modules, units, and sub-units can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, sub-modules, units, and sub-units can be at least partially implemented as a computer program module, which can perform the corresponding functions when the computer program module is executed.

[0135] For example, any multiple of the first acquisition module 610, the second acquisition module 620, and the first output module 630 can be combined into a single module / unit / sub-unit, or any one of these modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functionality of one or more of these modules / units / sub-units can be combined with at least part of the functionality of other modules / units / sub-units and implemented in a single module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the first acquisition module 610, the second acquisition module 620, and the first output module 630 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented in hardware or firmware by any other reasonable means of circuit integration or packaging, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or any appropriate combination of any of these. Alternatively, at least one of the first acquisition module 610 , the second acquisition module 620 , and the first output module 630 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.

[0136] It should be noted that the preoperative decision support system for hiatal hernia based on multimodal data in the embodiments of the present application corresponds to the preoperative decision support method for hiatal hernia based on multimodal data in the embodiments of the present application. The description of the preoperative decision support system for hiatal hernia based on multimodal data specifically refers to the preoperative decision support method for hiatal hernia based on multimodal data, which will not be repeated here.

[0137] Figure 7 A block diagram of an electronic device for a preoperative decision support method for hiatal hernia surgery based on multimodal data according to an embodiment of the present disclosure is schematically shown. Figure 7 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0138] like Figure 7As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a ROM (Read Only Memory) 702 or a program loaded from a storage unit 708 into a RAM (Random Access Memory) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0139] Various programs and data required for the operation of the electronic device 700 are stored in the RAM 703. The processor 701, ROM 702, and RAM 703 are connected to each other via a bus 704. The processor 701 executes the various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than the ROM 702 and RAM 703. The processor 701 may also execute the various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0140] According to an embodiment of the present disclosure, electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to bus 704. Electronic device 700 may also include one or more of the following components connected to I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 708 including a hard disk; and a communication section 709 including a network interface card such as a LAN card or modem. Communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to I / O interface 705 as needed. Removable media 711, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 710 as needed, so that computer programs read from the removable media can be installed into storage section 708 as needed.

[0141] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0142] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0143] According to embodiments of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0144] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703 .

[0145] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the preoperative decision support method for hiatal hernia based on multimodal data provided by the embodiment of the present disclosure.

[0146] When the computer program is executed by the processor 701, the above functions defined in the system / device of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0147] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 709, and / or installed from a removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0148] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, languages such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems and methods according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or coupled in various ways, and all of these combinations and / or couplings fall within the scope of the present disclosure.

[0150] The above describes the embodiments of the present disclosure. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A preoperative decision support method for hiatal hernia surgery based on multimodal data, comprising: Acquiring lesion information of a plurality of lesion organs from a target image of a target object, wherein the target image represents the morphology of each of the plurality of lesion organs and the positional relationship between the plurality of lesion organs, the lesion information including lesion morphology information, lesion diameter parameters, and lesion ratio parameters related to the lesion organs, and the lesion organs include the esophagus of the target object; Acquiring health risk indication information of the target subject from the medical information of the target subject, wherein the health risk indication information includes symptom indication information and efficacy indication information; The lesion information and the health risk indicator information are input into a preoperative decision support model, and a preoperative decision support model is output, wherein the preoperative decision support model includes at least one of emergency surgery, elective surgery and conservative treatment.

2. The method according to claim 1, wherein The disease indication information includes acute disease information, basic disease information and target symptom information related to hiatal hernia disease; Inputting the lesion information and the health risk indicator information into a preoperative decision support model and outputting a preoperative decision support model includes: Based on the acute condition information, using the preoperative decision support model to determine whether the patient has an acute indication, obtaining a first determination result, wherein if the first determination result is yes, outputting the preoperative decision support as emergency surgery; If the first judgment result is negative, the preoperative decision support model is used to determine whether the patient has any underlying health problems based on the underlying disease information, thereby obtaining a second judgment result. If the second judgment result is positive, the preoperative decision support model is output as conservative treatment. When the second judgment result is negative, the preoperative auxiliary decision is outputted using the preoperative auxiliary decision model according to the lesion morphology information.

3. The method according to claim 2, wherein the lesion morphology information includes one of a paraesophageal type and a sliding type; Outputting the preoperative auxiliary decision by using the preoperative auxiliary decision model according to the lesion morphology information includes: When the lesion morphology information is the paraesophageal type, the preoperative decision support model is used to determine whether the treatment is effective based on the efficacy indication information, to obtain a third judgment result, wherein if the third judgment result is yes, the preoperative decision support is output as conservative treatment; if the third judgment result is no, the preoperative decision support is output as elective surgery; In the case where the lesion morphology information is of the sliding type, the preoperative decision support model is used to determine whether the patient has the target symptoms based on the target symptom information to obtain a fourth judgment result, wherein when the fourth judgment result is yes, the preoperative decision support is output as elective surgery, and when the fourth judgment result is no, the preoperative decision support is output as conservative treatment.

4. The method according to claim 1, wherein In the case where the preoperative decision support includes at least one of emergency surgery and elective surgery, the preoperative decision support model also determines the repair method based on the lesion diameter parameter and the lesion proportion parameter, and the repair method includes at least one of an ordinary repair method and a patch repair method.

5. The method according to claim 1, wherein The step of acquiring lesion information of multiple lesion organs from a target image of a target object includes: Segmenting the image regions of the plurality of diseased organs in the target image to obtain a plurality of organ contours of the plurality of diseased organs; determining a plurality of three-dimensional organ models representing the plurality of diseased organs based on the plurality of organ contours; determining a target positional relationship between the plurality of three-dimensional organ models based on a positional relationship between the image regions of the plurality of diseased organs in the target image; The lesion information is determined based on the relationship between the plurality of three-dimensional organ models and the target position.

6. The method according to claim 1, wherein The medical information includes a medical record map related to the target object; The step of obtaining the health risk indicator information of the target subject from the medical treatment information of the target subject includes: Querying a first target entity having an acute indication attribute from the medical record graph, and determining the acute disease information based on the first target entity; Querying a second target entity having a basic health problem attribute from the medical record graph, and determining the basic disease information based on the second target entity; Querying a third target entity having a target symptom attribute from the medical record map, and determining the target symptom information related to hiatal hernia disease based on the third target entity; A fourth target entity having a treatment effect attribute is queried from the medical record map, and the treatment effect indication information is determined based on the fourth target entity.

7. A preoperative decision support system for hiatal hernia surgery based on multimodal data, comprising: a first acquisition module, configured to acquire lesion information of a plurality of lesion organs from a target image of a target object, wherein the target image represents the morphology of each of the plurality of lesion organs and the positional relationship between the plurality of lesion organs, the lesion information including lesion morphology information, lesion diameter parameters, and lesion ratio parameters related to the lesion organs, and the lesion organs include the esophagus of the target object; A second acquisition module is configured to acquire health risk indication information of the target subject from the medical treatment information of the target subject, wherein the health risk indication information includes symptom indication information and efficacy indication information; and The first output module is used to input the lesion information and the health risk indicator information into a preoperative decision support model, and output a preoperative decision support model, wherein the preoperative decision support model includes at least one of emergency surgery, elective surgery and conservative treatment.

8. An electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to implement the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images