A guidance auxiliary system for minimally invasive thoracoscopic surgery of mediastinal tumors

Abstract

The present invention relates to the field of biometric recognition technology, and specifically to a minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors. It includes an acquisition module for acquiring electronic medical record information and acquiring chest CT images and ultrasonic waveforms in real time through an ultrasonic detector; a medical record analysis module for determining the complexity of the medical record based on keywords in the electronic medical record information; a CT analysis module for determining the degree of movement and deflection of the thoracoscope's viewing angle at the current moment; a waveform analysis module for determining the similarity between the ultrasonic waveform and the preset structural waveforms of different structures, and then determining the structural expression; a reminder module for determining the sensitivity of the viewing angle of the thoracoscope at the current moment based on the complexity of the medical record, the degree of movement and deflection, and the structural expression, and performing surgical operation assistance reminders based on the sensitivity of the viewing angle. The present invention can effectively improve the reliability of surgical assistance.

Description

Technical Field

[0001] The present invention relates to the technical field of biometric identification, and in particular to a guidance auxiliary system for minimally invasive thoracoscopic surgery of mediastinal tumors. Background Art

[0002] Mediastinal tumors refer to tumors that occur in the thoracic mediastinum (the area between the two lungs in the chest cavity), usually including teratomas, lymphomas, thymomas, neurogenic tumors and metastatic tumors. Minimally invasive thoracoscopic surgery is a technology that uses thoracoscopy to diagnose and treat mediastinal tumors. Compared with traditional open chest surgery, minimally invasive surgery is less traumatic, has a shorter recovery period after surgery, and has less bleeding. In minimally invasive thoracoscopic surgery for mediastinal tumors, a thoracoscope is used to enter through a small incision in the patient to provide high-definition endoscopic images. The doctor can use the display screen to locate the tumor for resection. During the actual operation, some guidance assistance functions will be added to improve surgical accuracy.

[0003] In related technologies, image processing is usually used for identification and auxiliary guidance. In this way, the visual information that a minimally invasive thoracoscope can provide inside the chest cavity has certain limitations due to the field of view angle, tissue thickness, etc., which leads to low reliability of surgical assistance. Summary of the Invention

[0004] In order to solve the technical problem that the visual information provided by minimally invasive thoracoscopic surgery inside the chest cavity is limited by certain factors such as the field of view and tissue thickness, thereby resulting in low reliability of surgical assistance, the present invention provides a minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors. The technical solutions adopted are as follows:

[0005] The present invention proposes a minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors, comprising:

[0006] An acquisition module is used to obtain electronic medical record information and obtain chest CT images and ultrasound waveforms in real time through an ultrasound detector;

[0007] A medical record analysis module is used to determine the complexity of the medical records of mediastinal tumor locations based on keywords in the electronic medical record information;

[0008] The CT analysis module is used to determine the center point of the chest CT image and determine the degree of movement and deflection of the thoracoscope viewing angle at the current moment based on the image distance between the pixel points at the same image position in the chest CT image at the current moment and the previous moment and the center point, as well as the grayscale difference of the pixel points;

[0009] a waveform analysis module, configured to determine the similarity between the ultrasound waveform at the current moment and the preset structural waveforms of different structures within the thoracic cavity by comparing the ultrasound waveform at the current moment and the preset structural waveforms of the different structures, and to determine the structural representation of the thoracic view at the current moment based on the similarity, wherein the different structures within the thoracic cavity include at least the heart, great blood vessels, and spine;

[0010] The reminder module is used to determine the visual sensitivity of the thoracoscope at the current moment based on the complexity of the medical record, the degree of movement and deflection, and the structural expression, and to provide auxiliary reminders for surgical operations based on the visual sensitivity.

[0011] Furthermore, the electronic medical record information is an electronic medical record sheet, and determining the complexity of the medical record of the mediastinal tumor location based on keywords in the electronic medical record information includes:

[0012] Extracting keywords from the text in the electronic medical record to determine a medical record keyword group;

[0013] The intersection-and-union ratio of the number of identical keywords in the medical record keyword group and the preset mediastinal tumor keyword group is calculated as the complexity of the medical record of the mediastinal tumor site.

[0014] Furthermore, the method for obtaining the medical record keyword group includes:

[0015] Perform word segmentation processing on the text in the electronic medical record based on the Jieba word segmentation algorithm to obtain medical record word segmentation;

[0016] Mediastinum keywords in medical record segmentation are extracted based on preset characteristic nouns of the oncology department, wherein the mediastinum keywords constitute a medical record keyword group.

[0017] Furthermore, determining the degree of movement and deflection of the thoracoscope viewing angle at the current moment based on the image distance between the pixel point at the same image position in the chest CT image at the current moment and the previous moment and the center point, and the grayscale difference of the pixel point, includes:

[0018] Determine the Euclidean distance between the pixel point and the center point at any image position as the center distance of the corresponding image position;

[0019] The reciprocal of the center distance is normalized as the distance influence degree, wherein the distance influence degree of the center point is 1;

[0020] The degree of movement and deflection of the thoracoscope viewing angle at the current moment is determined based on the grayscale difference and distance influence of the pixel point at each image position between the current moment and the previous moment.

[0021] Furthermore, determining the degree of movement and deflection of the thoracoscope viewing angle at the current moment based on the grayscale difference and distance influence degree of the pixel point at each image position at the current moment and the previous moment includes:

[0022] Calculate the product of the grayscale difference of the pixel points at the same image position and the distance influence degree to obtain the position influence index;

[0023] The mean values ​​of all position influence indicators are normalized to obtain the degree of movement deflection.

[0024] Furthermore, the determining of the similarity between the ultrasonic waveform at the current moment and the preset structural waveforms of different structures in the chest cavity based on the comparison thereof includes:

[0025] Performing dynamic time warping on the ultrasonic waveform and each preset structure waveform respectively to obtain DTW values ​​of the ultrasonic waveform and each preset structure waveform;

[0026] The similarity between the ultrasonic waveform and the preset structure waveforms of different structures is determined based on the comparison of the DTW values ​​between the ultrasonic waveform and each preset structure waveform.

[0027] Furthermore, the determining of the similarity between the ultrasonic waveform and the preset structural waveforms of different structures based on the comparison of the DTW values ​​of the ultrasonic waveform and each preset structural waveform includes:

[0028] The reciprocals of the DTW values ​​of the ultrasonic waveform and any preset structure waveform are normalized to obtain the similarity between the ultrasonic waveform and the corresponding preset structure waveform.

[0029] Furthermore, determining the structural representation of the thoracoscopy view at the current moment according to the similarity includes:

[0030] Taking a preset number of preset structure waveforms with the largest similarity values ​​as approaching waveforms, setting the additional attention of the approaching waveforms to 1, and setting the additional attention of other preset structure waveforms to 0;

[0031] Calculating the product of the similarity between the ultrasonic waveform and each preset structure waveform and the additional attention degree to obtain a proximity index between the ultrasonic waveform and each preset structure waveform;

[0032] The mean of the proximity index between the ultrasonic waveform and all preset structural waveforms is calculated, and the maximum and minimum values ​​are normalized as the structural expression of the thoracoscopic perspective at the current moment.

[0033] Furthermore, the visual sensitivity of the thoracoscope at the current moment is determined based on the complexity of the medical history, the degree of movement and deflection, and the structural expression, including:

[0034] The product of the complexity of the medical history, the degree of movement and deflection, and the structural expression is calculated, and the maximum and minimum values ​​are normalized to obtain the viewing angle sensitivity.

[0035] Furthermore, performing surgical operation auxiliary reminders according to the visual angle sensitivity includes:

[0036] When the viewing angle sensitivity is greater than a preset sensitivity threshold, a surgical operation sensitivity reminder is performed.

[0037] The present invention has the following beneficial effects:

[0038] The embodiment of the present invention achieves the guidance and assistance effect of minimally invasive thoracoscopic surgery by combining real-time analysis of three dimensions: electronic medical record information, CT images, and ultrasonic waveforms. Among them, the complexity of the medical record is analyzed from the electronic medical record information, and the complexity of the overall operation is determined before the operation, which helps to analyze the complexity of the location of the patient's mediastinal tumor. From the real-time CT image changes, the degree of movement and deflection of the thoracoscope perspective at the current moment is analyzed, and then the perspective changes at the current moment are analyzed, and the invasiveness of the surgical operation is determined from the perspective changes, that is, the greater the perspective deflection, the more likely it is to invade other structures during the operation. Therefore, the degree of movement and deflection can be analyzed accurately and quickly, thereby improving the timeliness and reliability of the analysis. The processing of ultrasonic waveforms can effectively compare the similarities of important structures, thereby accurately locating the performance of important structures. Therefore, the visual sensitivity of the thoracoscope at the current moment is determined based on the complexity of the medical history, the degree of movement and deflection, and the degree of structural expression. Surgical operation auxiliary reminders are given according to the visual sensitivity. In this way, the complex situation of the location of the mediastinal tumor before surgery and the important structures that may be touched during real-time surgery are comprehensively considered to make guided auxiliary reminders and improve the reliability of surgical guidance assistance. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 A structural diagram of a guidance assistance system for minimally invasive thoracoscopic surgery of mediastinal tumors provided by one embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the clinical division of the mediastinum provided by one embodiment of the present invention;

[0042] Figure 3 A CT schematic diagram of a tumor lesion area provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0043] To further illustrate the technical means and effectiveness of the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors proposed by the present invention. In the following description, different references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0044] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0045] The mediastinum is located in the center of the human chest cavity, involving important structures such as the heart, large blood vessels, trachea, and esophagus. Based on the common four-part division method of the mediastinum, it can be roughly divided into four parts (i.e., upper mediastinum, anterior mediastinum, middle mediastinum, and posterior mediastinum). Mediastinal tumors may be located in any part, and different important structures will be involved in different parts. For example, the posterior mediastinum contains the trachea, esophagus, etc., while the middle mediastinum contains the heart, large blood vessels, etc.

[0046] Regardless of the tumor's location, its size and invasiveness significantly impact the difficulty of surgery. This is especially true when mediastinal tumors are located at the intersection of different sites. During surgery, it is necessary to simultaneously consider and address the complex structures of both regions to avoid damaging any important structures (e.g., a mediastinal tumor may simultaneously invade the thymus (located in the anterior mediastinum) and large blood vessels (located in the superior mediastinum), which greatly impacts the complexity of surgery.

[0047] The specific scheme of the minimally invasive thoracoscopic surgery guidance auxiliary system for mediastinal tumors provided by the present invention is described in detail below with reference to the accompanying drawings.

[0048] See also Figure 1 , which shows a structural diagram of a minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors provided by an embodiment of the present invention. The system includes: an acquisition module 101, a medical record analysis module 102, a CT analysis module 103, a waveform analysis module 104, and a reminder module 105. The embodiment of the present invention mainly provides auxiliary reminders for minimally invasive thoracoscopic surgery for mediastinal tumors at the current moment through analysis of three dimensions: medical records, CT, and waveforms. The specific introduction of each module includes:

[0049] An acquisition module 101 is used to acquire electronic medical record information and obtain chest CT images and ultrasound waveforms in real time through an ultrasound detector;

[0050] The medical record analysis module 102 is used to determine the complexity of the medical records of the mediastinal tumor location based on keywords in the electronic medical record information;

[0051] The CT analysis module 103 is used to determine the center point of the chest CT image and determine the degree of movement and deflection of the thoracoscope viewing angle at the current moment based on the image distance between the pixel points at the same image position in the chest CT image at the current moment and the previous moment and the center point, as well as the grayscale difference of the pixel points;

[0052] The waveform analysis module 104 is configured to determine the similarity between the ultrasound waveform at the current moment and the preset structural waveforms of different structures within the thoracic cavity by comparing the ultrasound waveform at the current moment and the preset structural waveforms of the different structures, and to determine the structural representation of the thoracic view at the current moment based on the similarity, wherein the different structures within the thoracic cavity include at least the heart, great blood vessels, and spine;

[0053] The reminder module 105 is used to determine the visual sensitivity of the thoracoscope at the current moment based on the complexity of the medical history, the degree of movement and deflection, and the degree of structural expression, and to provide auxiliary reminders for surgical operations based on the visual sensitivity.

[0054] The embodiments of the present invention are described in detail:

[0055] Mediastinal tumors are located in the center of the chest cavity, involving important structures such as the heart, great blood vessels, trachea, and esophagus. The mediastinum is usually divided into four parts, with the line connecting the sternal angle and the lower edge of the fourth thoracic vertebra as the boundary. The lower mediastinum is divided into three areas: anterior, middle, and posterior. Figure 2 , Figure 2 A schematic diagram of the clinical divisions of the mediastinum provided in one embodiment of the present invention.

[0056] The medical record analysis module 102 is a module for implementing specific analysis of electronic medical record information, and can determine the complexity of the medical record of the mediastinal tumor site based on keywords in the electronic medical record information.

[0057] The patient's electronic medical record information contains the doctor's preoperative analysis of mediastinal tumors using relevant equipment, including specific information such as location and severity description, which can help analyze the complex location of the patient's mediastinal tumor.

[0058] Before minimally invasive thoracoscopic surgery (VATS) for patients with mediastinal tumors, doctors use CT or MRI to determine tumor size and location. This preoperative tumor description is also included in the electronic medical record. Therefore, by extracting relevant preoperative tumor information from the patient's electronic medical record, we can determine the patient's tumor status before surgery.

[0059] Among them, the electronic medical record information is an electronic medical record form, and the specific method for obtaining the complexity of the medical record further includes: extracting keywords from the text in the electronic medical record form to determine the medical record keyword group; calculating the intersection and union ratio of the number of identical keywords in the medical record keyword group and the preset mediastinal tumor keyword group as the medical record complexity of the mediastinal tumor location.

[0060] Among them, keyword extraction can include multiple extraction methods. Preferably, in the embodiment of the present invention, the text in the electronic medical record is segmented based on the jieba segmentation algorithm to obtain medical record segmentation; the mediastinum keywords in the medical record segmentation are extracted based on the preset characteristic nouns of the oncology department, wherein the mediastinum keywords constitute the medical record keyword group.

[0061] It should be noted that the jieba word segmentation algorithm is a word segmentation method well known in the art. Of course, in other embodiments of the present invention, other word segmentation methods may also be used, such as hidden Markov models, n-gram models, etc. Of course, recurrent neural networks (RNNs), long short-term memory networks (LSTMs) combined with CRFs, or Transformer models may also be used to automatically learn word segmentation rules through large amounts of data training, without limitation.

[0062] After word segmentation processing, the medical record segmentation of the electronic medical record is obtained. The case segmentation contains segmentations related to specific mediastinal tumors and irrelevant segmentations. For specific analysis, in an embodiment of the present invention, the mediastinum keywords in the medical record segmentation can be extracted from the preset feature nouns of the oncology department.

[0063] Among them, the preset characteristic nouns of the oncology department are characteristic nouns related to mediastinal tumors preset by the oncology department. The phrases of the preset characteristic nouns of the oncology department can be pre-set, and the medical record segmentation words and the phrases can be searched. If the search is consistent, the medical record segmentation words are used as mediastinum keywords, and all the mediastinum keywords in an electronic medical record are combined into a medical record keyword group.

[0064] Among them, the preset mediastinal tumor keyword group can be specifically a phrase composed of preset characteristic nouns of the oncology department. That is to say, through the intersection and union ratio of the number of identical keywords in the medical record keyword group and the preset mediastinal tumor keyword group, the complexity of the medical record of the mediastinal tumor location can be specifically calculated.

[0065] Among them, the intersection of quantity and union ratio is the ratio of the number of repeated keywords in the two phrases to the number of all keywords. The larger the value, the more repeated keywords there are and the larger the proportion of repeated keywords in the overall preset mediastinal tumor keyword group.

[0066] The intersection-of-numbers-and-union ratio (IoU) indicates the similarity between the two. The higher the similarity, the more tumor keywords are mentioned in the electronic medical record description, indicating that the patient's mediastinal tumor is more likely to be located near the junction of different mediastinal regions. This indicates that the patient's mediastinal tumor distribution is more complex. This further suggests that the more complex the situation, the more relevant text is required to describe it in the medical record, directly determining the complexity of the patient's medical record.

[0067] The CT analysis module 103 is used to determine the center point of the chest CT image and determine the degree of movement and deflection of the thoracoscope viewing angle at the current moment based on the image distance between the pixel points at the same image position in the chest CT image at the current moment and the previous moment and the center point, as well as the grayscale difference of the pixel points.

[0068] During the surgery for mediastinal tumors, the thoracoscope will display the relevant image information of the patient's mediastinal tumor in real time. Doctors usually adjust the viewing angle of the thoracoscope to observe the size of the mediastinal tumor from different angles, the distribution of surrounding tissues, and perform different surgical incision operations to ensure the accuracy and safety of the surgery (the greater the viewing angle deviation, the more likely it is to invade other structures during the operation). See Figure 3 , Figure 3 A CT schematic diagram of a tumor lesion area provided by one embodiment of the present invention.

[0069] Adjustment of the viewing angle of the thoracoscope will cause a change in the grayscale of the image, so the current movement and deflection degree of the thoracoscope can be determined by the grayscale change of adjacent frame images.

[0070] Furthermore, the degree of movement and deflection of the thoracoscope viewing angle at the current moment is determined based on the image distance between the pixel point and the center point at the same image position in the chest CT image at the current moment and the previous moment, as well as the grayscale difference of the pixel point, including: determining the Euclidean distance between the pixel point and the center point at any image position as the center distance of the corresponding image position; normalizing the inverse of the center distance as the distance influence degree, wherein the distance influence degree of the center point is 1; and determining the degree of movement and deflection of the thoracoscope viewing angle at the current moment based on the grayscale difference and distance influence degree of the pixel point at each image position at the current moment and the previous moment.

[0071] In the embodiment of the present invention, the image center is used as the specific offset analysis center. During the actual offset process, various offset modes such as translation and rotation may occur. During the actual surgery, the important area will be subconsciously placed in the center. Therefore, the weight passing through the center is higher, and the farther the distance, the lower the corresponding weight. Therefore, the reciprocal of the center distance is normalized as the distance influence degree, and the distance influence degree of the center point is set to 1.

[0072] Therefore, the degree of movement deflection of the thoracoscope viewing angle at the current moment can be calculated by combining the grayscale change and the distance influence degree. Furthermore, in some embodiments of the present invention, the product of the grayscale difference of the pixel point at the same image position and the distance influence degree is calculated to obtain the position influence index; the mean value of all position influence indicators is normalized to obtain the degree of movement deflection.

[0073] Among them, because the grayscale changes of the overall CT image in the same structural area are small, but changes will occur due to offset in the cross-structural area, therefore, the greater the grayscale difference of the pixel points at the same image position, the greater the possibility of image offset at the corresponding position, and the more cross-region textures there are, the greater the degree of movement deflection. Because the center position is usually the area with more obvious attention, the distance influence degree is further strengthened and qualified to obtain the position influence index, and the mean of all position influence indicators is statistically normalized to obtain the degree of movement deflection.

[0074] In an embodiment of the present invention, there are multiple ways to analyze the motion offset of CT images, such as key point matching based on image analysis, etc. The grayscale change analysis method is more convenient and quick. During minimally invasive thoracoscopic surgery with high real-time requirements, the results analyzed by the method of the embodiment of the present invention can meet the basic auxiliary needs while being able to perform data processing more quickly and quickly obtain accurate and reliable specific parameters of the degree of motion offset.

[0075] The waveform analysis module 104 is used to determine the similarity between the ultrasonic waveform at the current moment and the preset structural waveforms of different structures in the chest cavity by comparing them, and to determine the structural representation of the thoracoscopy view at the current moment based on the similarity, wherein the different structures in the chest cavity include at least the heart, great blood vessels, and spine.

[0076] As the thoracoscope moves and deflects within the body, the surgeon will perform corresponding surgical procedures on the patient's mediastinal tumor. To ensure that the surgeon's surgical procedures avoid damaging other important internal structures, the surgeon's proximity to critical structures within the thoracic cavity must be carefully considered. Therefore, the thoracoscope and ultrasound detector are synchronized to obtain real-time ultrasound waveform data.

[0077] Because important structures inside the chest cavity usually have specific ultrasonic waveforms, the ultrasonic waveforms obtained in real time can be compared with the ultrasonic waveforms of various important structures. The more similar the current ultrasonic waveform data is to the ultrasonic waveform of an important structure in the chest cavity, the closer the current position of the thoracoscope is to an important structure in the current chest cavity.

[0078] Furthermore, in some embodiments of the present invention, the similarity between the ultrasonic waveform and the preset structural waveforms of different structures in the chest cavity is determined based on the comparison between the ultrasonic waveform at the current moment and the preset structural waveforms of different structures, including: performing dynamic time warping processing on the ultrasonic waveform and each preset structural waveform respectively to obtain the DTW values ​​of the ultrasonic waveform and each preset structural waveform; and determining the similarity between the ultrasonic waveform and the preset structural waveforms of different structures based on the comparison between the DTW values ​​of the ultrasonic waveform and each preset structural waveform.

[0079] The preset structural fluctuation is a waveform obtained by performing ultrasonic analysis on the normal structure in advance, which represents the fluctuation information of the normal structure. That is, the ultrasonic waveform at the current moment is compared with the preset structural waveform to determine the similarity.

[0080] The embodiment of the present invention mainly uses the dynamic time warping algorithm as a specific fluctuation comparison and analysis method. Therefore, the ultrasonic waveform and each preset structure waveform are subjected to dynamic time warping processing respectively to obtain the DTW values ​​of the ultrasonic waveform and each preset structure waveform. It should be noted that the larger the DTW value, the greater the difference between the two fluctuations. Therefore, in the embodiment of the present invention, the similarity between the ultrasonic waveform and the preset structure waveforms of different structures is determined based on the comparison of the DTW values ​​of the ultrasonic waveform and each preset structure waveform, including: normalizing the reciprocal of the DTW values ​​of the ultrasonic waveform and any preset structure waveform to the maximum and minimum values ​​to obtain the similarity between the ultrasonic waveform and the corresponding preset structure waveform.

[0081] That is, the similarity is obtained by calculating the inverse and normalizing the maximum and minimum values. The higher the similarity, the greater the similarity between the ultrasonic waveform and the preset structural waveform of the corresponding structure.

[0082] Then, the structural expression can be calculated based on the similarity, and the structural expression of the thoracoscopic view at the current moment can be determined based on the similarity, including: taking a preset number of preset structural waveforms with the largest similarity values ​​as approaching waveforms, setting the additional attention of the approaching waveform to 1, and setting the additional attention of other preset structural waveforms to 0; calculating the product of the similarity and additional attention between the ultrasonic waveform and each preset structural waveform to obtain the proximity index of the ultrasonic waveform and each preset structural waveform; calculating the mean of the proximity index of the ultrasonic waveform and all preset structural waveforms, and normalizing the maximum and minimum values ​​as the structural expression of the thoracoscopic view at the current moment.

[0083] Since each structure has a similarity, numerical analysis is required. In an embodiment of the present invention, the preset number can be set to 3, that is, the three preset structure waveforms with the largest similarity values ​​are used as approaching waveforms, and the additional attention of the approaching waveform is set to 1, and the additional attention of other preset structure waveforms is set to 0. Weighting is performed by unit values ​​1 and 0 to avoid the influence of unrelated structures.

[0084] Afterwards, the product of the similarity between the ultrasonic waveform and each preset structural waveform and the additional attention is calculated to obtain the proximity index between the ultrasonic waveform and each preset structural waveform. The proximity index represents the index information of the current thoracoscope position being closer to a certain structure in the current chest cavity. Therefore, the mean of the proximity index between the ultrasonic waveform and all preset structural waveforms is calculated, and the maximum and minimum values ​​are normalized as the structural expression of the thoracoscope perspective at the current moment.

[0085] The reminder module 105 is used to determine the visual sensitivity of the thoracoscope at the current moment based on the complexity of the medical history, the degree of movement and deflection, and the degree of structural expression, and to provide auxiliary reminders for surgical operations based on the visual sensitivity.

[0086] In the embodiment of the present invention, by analyzing the three dimensions of electronic medical record information, view angle change and ultrasonic waveform, the view angle sensitivity of the thoracoscope at the current moment can be determined by combining the three-dimensional information.

[0087] Furthermore, in some embodiments of the present invention, the visual sensitivity of the thoracoscope at the current moment is determined in combination with the complexity of the medical record, the degree of movement deflection, and the structural expression, including: calculating the product of the complexity of the medical record, the degree of movement deflection, and the structural expression, and normalizing the maximum and minimum values ​​to obtain the visual sensitivity.

[0088] The greater the values ​​of medical record complexity, movement deflection, and structural expression, the more complex and sensitive the scene information is. For example, the more complex the medical record, the greater the amplitude of visual angle deflection, and the higher the similarity with important structures, the more sensitive the visual angle expression is, and the greater the need for surgical assistance reminders.

[0089] Therefore, the product of the three-dimensional data is directly calculated, and the maximum and minimum values ​​are normalized to obtain the visual angle sensitivity. Furthermore, surgical operation auxiliary reminders are provided based on the visual angle sensitivity, including: when the visual angle sensitivity exceeds a preset sensitivity threshold, a surgical operation sensitivity reminder is provided.

[0090] The preset sensitivity threshold is a threshold value of the viewing angle sensitivity. Optionally, the preset sensitivity threshold may be, for example, 0.8. That is to say, when the viewing angle sensitivity is greater than 0.8, a sensitive reminder for surgical operation is issued.

[0091] The surgical operation sensitive reminder in the embodiment of the present invention can be specifically, for example, a light reminder, that is, when a sensitive situation is detected, the light is changed to attract the attention of relevant personnel. Of course, other reminder methods such as vibration can also be used, and there is no limitation to this.

[0092] The embodiment of the present invention achieves the guidance and assistance effect of minimally invasive thoracoscopic surgery by combining real-time analysis of three dimensions: electronic medical record information, CT images, and ultrasonic waveforms. Among them, the complexity of the medical record is analyzed from the electronic medical record information, and the complexity of the overall operation is determined before the operation, which helps to analyze the complexity of the location of the patient's mediastinal tumor. From the real-time CT image changes, the degree of movement and deflection of the thoracoscope perspective at the current moment is analyzed, and then the perspective changes at the current moment are analyzed, and the invasiveness of the surgical operation is determined from the perspective changes, that is, the greater the perspective deflection, the more likely it is to invade other structures during the operation. Therefore, the degree of movement and deflection can be analyzed accurately and quickly, thereby improving the timeliness and reliability of the analysis. The processing of ultrasonic waveforms can effectively compare the similarities of important structures, thereby accurately locating the performance of important structures. Therefore, the visual sensitivity of the thoracoscope at the current moment is determined based on the complexity of the medical history, the degree of movement and deflection, and the degree of structural expression. Surgical operation auxiliary reminders are given according to the visual sensitivity. In this way, the complex situation of the location of the mediastinal tumor before surgery and the important structures that may be touched during real-time surgery are comprehensively considered to make guided auxiliary reminders and improve the reliability of surgical guidance assistance.

[0093] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0094] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors, characterized by: include: An acquisition module is used to obtain electronic medical record information and obtain chest CT images and ultrasound waveforms in real time through an ultrasound detector; A medical record analysis module is used to determine the complexity of the medical records of mediastinal tumor locations based on keywords in the electronic medical record information; The CT analysis module is used to determine the center point of the chest CT image and determine the degree of movement and deflection of the thoracoscope viewing angle at the current moment based on the image distance between the pixel points at the same image position in the chest CT image at the current moment and the previous moment and the center point, as well as the grayscale difference of the pixel points; a waveform analysis module, configured to determine the similarity between the ultrasound waveform at the current moment and the preset structural waveforms of different structures within the thoracic cavity by comparing the ultrasound waveform at the current moment and the preset structural waveforms of the different structures, and to determine the structural representation of the thoracic view at the current moment based on the similarity, wherein the different structures within the thoracic cavity include at least the heart, great blood vessels, and spine; The reminder module is used to determine the visual sensitivity of the thoracoscope at the current moment based on the complexity of the medical record, the degree of movement deflection and the structural expression. The visual sensitivity is obtained by normalizing the maximum and minimum values ​​of the product of the complexity of the medical record, the degree of movement deflection and the structural expression; when the visual sensitivity is greater than the preset sensitivity threshold, a sensitive reminder for surgical operation is performed.

2. The minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors according to claim 1, characterized in that: The electronic medical record information is an electronic medical record sheet, and determining the complexity of the medical record of the mediastinal tumor location based on keywords in the electronic medical record information includes: Extracting keywords from the text in the electronic medical record to determine a medical record keyword group; The intersection-and-union ratio of the number of identical keywords in the medical record keyword group and the preset mediastinal tumor keyword group is calculated as the complexity of the medical record of the mediastinal tumor site.

3. The minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors according to claim 2, characterized in that: The method for obtaining the medical record keyword group includes: Perform word segmentation processing on the text in the electronic medical record based on the Jieba word segmentation algorithm to obtain medical record word segmentation; Mediastinum keywords in medical record segmentation are extracted based on preset characteristic nouns of the oncology department, wherein the mediastinum keywords constitute a medical record keyword group.

4. The minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors according to claim 1, characterized in that: The method of determining the degree of movement and deflection of the thoracoscope viewing angle at the current moment based on the image distance between the pixel point at the same image position in the chest CT image at the current moment and the previous moment and the center point, and the grayscale difference of the pixel point, includes: Determine the Euclidean distance between the pixel point and the center point at any image position as the center distance of the corresponding image position; The reciprocal of the center distance is normalized as the distance influence degree, wherein the distance influence degree of the center point is 1; The degree of movement and deflection of the thoracoscope viewing angle at the current moment is determined based on the grayscale difference and distance influence of the pixel point at each image position between the current moment and the previous moment.

5. The minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors according to claim 4, characterized in that: Determining the degree of movement and deflection of the thoracoscope viewing angle at the current moment based on the grayscale difference and distance influence degree of the pixel point at each image position at the current moment and the previous moment includes: Calculate the product of the grayscale difference of the pixel points at the same image position and the distance influence degree to obtain the position influence index; The mean values ​​of all position influence indicators are normalized to obtain the degree of movement deflection.

6. The minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors according to claim 1, characterized in that: The step of comparing the ultrasonic waveform at the current moment with the preset structural waveforms of different structures in the chest cavity to determine the similarity between the ultrasonic waveform and the preset structural waveforms of different structures includes: Performing dynamic time warping on the ultrasonic waveform and each preset structure waveform respectively to obtain DTW values ​​of the ultrasonic waveform and each preset structure waveform; The similarity between the ultrasonic waveform and the preset structure waveforms of different structures is determined based on the comparison of the DTW values ​​between the ultrasonic waveform and each preset structure waveform.

7. The minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors according to claim 6, characterized in that: The step of comparing the DTW value of the ultrasonic waveform with each preset structure waveform to determine the similarity between the ultrasonic waveform and the preset structure waveforms of different structures includes: The reciprocals of the DTW values ​​of the ultrasonic waveform and any preset structure waveform are normalized to obtain the similarity between the ultrasonic waveform and the corresponding preset structure waveform.

8. The minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors according to claim 1, characterized in that: Determining the structural expressiveness of the thoracoscopy view at the current moment according to the similarity includes: Taking a preset number of preset structure waveforms with the largest similarity values ​​as approaching waveforms, setting the additional attention of the approaching waveforms to 1, and setting the additional attention of other preset structure waveforms to 0; Calculating the product of the similarity between the ultrasonic waveform and each preset structure waveform and the additional attention degree to obtain a proximity index between the ultrasonic waveform and each preset structure waveform; The mean of the proximity index between the ultrasonic waveform and all preset structural waveforms is calculated, and the maximum and minimum values ​​are normalized as the structural expression of the thoracoscopic perspective at the current moment.

Images