Longitudinal tumor minimally invasive thoracoscopic surgery guiding auxiliary system

Abstract

The invention relates to the technical field of biological feature recognition, in particular to a mediastinal tumor minimally invasive thoracoscopic surgery guiding auxiliary system. Comprising an obtaining module used for obtaining electronic medical record information and obtaining a chest CT image and an ultrasonic waveform in real time through an ultrasonic detector; the medical record analysis module is used for determining the complexity of the medical record according to the keyword of the electronic medical record information; the CT analysis module is used for determining the moving deflection degree of the visual angle of the thoracoscope at the current moment; the waveform analysis module is used for determining the similarity between the ultrasonic waveform and preset structure waveforms of different structures so as to determine the structure representation degree; and the reminding module is used for determining the visual angle sensitive degree of the thoracoscope at the current moment in combination with the medical record complexity degree, the movement deflection degree and the structural expression degree, and performing surgical operation auxiliary reminding according to the visual angle sensitive degree. The reliability of operation assistance can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of biometric recognition, and particularly relates to a minimally invasive thoracoscopic surgery guidance and assistance system for mediastinal tumors. Background Art

[0002] Mediastinal tumors refer to tumors that occur in the mediastinum of the chest cavity (the area between the two lungs in the chest cavity), and usually include teratomas, lymphomas, thymomas, neurogenic tumors, metastatic tumors, etc. Minimally invasive thoracoscopic surgery is a technique for diagnosing and treating mediastinal tumors using a thoracoscope. Compared with traditional open-chest surgery, minimally invasive surgery has less trauma, a shorter recovery period after surgery, and less bleeding. In minimally invasive thoracoscopic surgery for mediastinal tumors, a thoracoscope is inserted through a small incision in the patient to provide high-definition endoscopic images, and doctors can locate and remove tumors through a display screen. During the actual surgical process, some guidance and assistance functions are added to improve the surgical accuracy.

[0003] In related technologies, image processing is usually used for recognition and auxiliary guidance. In this way, due to certain limitations in the visual information provided by the minimally invasive thoracoscope inside the chest cavity, such as the viewing angle and tissue thickness, the reliability of surgical assistance is relatively low. Summary of the Invention

[0004] In order to solve the technical problem that due to certain limitations in the visual information provided by the minimally invasive thoracoscope inside the chest cavity, such as the viewing angle and tissue thickness, the reliability of surgical assistance is relatively low, the present invention provides a minimally invasive thoracoscopic surgery guidance and assistance system for mediastinal tumors. The specific technical solution adopted is as follows:

[0005] The present invention provides a minimally invasive thoracoscopic surgery guidance and assistance system for mediastinal tumors, including:

[0006] An acquisition module, configured to acquire electronic medical record information, and acquire chest CT images and ultrasonic waveforms in real time through an ultrasonic detector;

[0007] A medical record analysis module, configured to determine the complexity of the medical record at the location of the mediastinal tumor according to the keywords in the electronic medical record information;

[0008] A CT analysis module, configured to determine the central point position of the chest CT image, and determine the movement deflection degree of the thoracoscope viewing angle at the current moment according to the image distance between the pixel points at the same image position of the chest CT image at the current moment and the previous moment and the central point position, and the gray difference of the pixel points.

[0009] A waveform analysis module, which is used to determine the similarity between the ultrasonic waveform at the current moment and the preset structure waveforms of different structures in the thoracic cavity according to the comparison between the ultrasonic waveform at the current moment and the preset structure waveforms of different structures in the thoracic cavity, and determine the structural expression degree of the thoracoscope view at the current moment according to the similarity, wherein the different structures in the thoracic cavity at least include the heart, large blood vessels and spine;

[0010] A reminder module, which is used to determine the view sensitivity of the thoracoscope at the current moment in combination with the complexity of the medical record, the degree of movement and deflection, and the structural expression degree, and perform an auxiliary reminder for surgical operations according to the view sensitivity.

[0011] Further, the electronic medical record information is an electronic medical record form, and determining the complexity of the medical record of the mediastinal tumor location according to the keywords of the electronic medical record information includes:

[0012] Extract keywords from the text in the electronic medical record form to determine a medical record keyword group;

[0013] Calculate the intersection ratio of the number of identical keywords between the medical record keyword group and the preset mediastinal tumor keyword group as the complexity of the medical record of the mediastinal tumor location.

[0014] Further, the method for obtaining the medical record keyword group includes:

[0015] Perform word segmentation processing on the text in the electronic medical record form based on the jieba word segmentation algorithm to obtain medical record word segments;

[0016] Extract mediastinal keywords from the medical record word segments based on the preset characteristic nouns in the oncology department, wherein the mediastinal keywords form a medical record keyword group.

[0017] Further, determining the degree of movement and deflection of the thoracoscope view at the current moment according to the image distance between the pixel points at the same image position of the thoracic CT images at the current moment and the previous moment and the central point position, and the gray level difference of the pixel points includes:

[0018] Determine the Euclidean distance between the pixel points at any image position and the central point position as the central distance corresponding to the image position;

[0019] Normalize the reciprocal of the central distance as the distance influence degree, wherein the distance influence degree of the central point position is 1;

[0020] Determine the degree of movement and deflection of the thoracoscope view at the current moment according to the gray level difference and the distance influence degree of the pixel points at each image position at the current moment and the previous moment.

[0021] Further, determining the movement and deflection degree of the thoracoscope view at the current moment according to the gray difference and distance influence degree of the pixel points at each image position between the current moment and the previous moment includes:

[0022] Calculating the product of the gray difference and the distance influence degree of the pixel points at the same image position to obtain a position influence index;

[0023] Normalizing the mean value of all the position influence indexes to obtain the movement and deflection degree.

[0024] Further, determining the similarity between the ultrasonic waveform at the current moment and the preset structure waveforms of different structures in the thoracic cavity according to the comparison between the ultrasonic waveform at the current moment and the preset structure waveforms of different structures includes:

[0025] Performing dynamic time warping processing on the ultrasonic waveform and each preset structure waveform respectively to obtain the DTW values of the ultrasonic waveform and each preset structure waveform;

[0026] Determining the similarity between the ultrasonic waveform and the preset structure waveforms of different structures according to the comparison of the DTW values of the ultrasonic waveform and each preset structure waveform.

[0027] Further, determining the similarity between the ultrasonic waveform and the preset structure waveforms of different structures according to the comparison of the DTW values of the ultrasonic waveform and each preset structure waveform includes:

[0028] Performing maximum-minimum normalization processing on the reciprocal of the DTW value of the ultrasonic waveform and any one preset structure waveform to obtain the similarity between the ultrasonic waveform and the corresponding preset structure waveform.

[0029] Further, determining the structural representation degree of the thoracoscope view at the current moment according to the similarity includes:

[0030] Regarding the preset number of preset structure waveforms with the largest similarity values as the approaching waveforms, setting the additional attention degree of the approaching waveforms to 1, and setting the additional attention degree of other preset structure waveforms to 0;

[0031] Calculating the product of the similarity between the ultrasonic waveform and each preset structure waveform and the additional attention degree to obtain the proximity index between the ultrasonic waveform and each preset structure waveform;

[0032] Calculating the mean value of the proximity indexes between the ultrasonic waveform and all the preset structure waveforms, and performing maximum-minimum normalization processing as the structural representation degree of the thoracoscope view at the current moment.

[0033] Further, determining the view sensitivity degree of the thoracoscope at the current moment by combining the case complexity, the movement and deflection degree, and the structural representation degree includes:

[0034] Calculate the product of the complexity of the medical record, the degree of movement and deflection, and the degree of structural manifestation, and perform maximum-minimum normalization to obtain the perspective sensitivity degree.

[0035] Further, perform surgical operation assistance reminder according to the perspective sensitivity degree, including:

[0036] When the perspective sensitivity degree is greater than a preset sensitivity threshold, perform a surgical operation sensitivity reminder.

[0037] The present invention has the following beneficial effects:

[0038] By combining the real-time analysis of three dimensions of electronic medical record information, CT images, and ultrasonic waveforms, the embodiment of the present invention realizes the guiding and assisting effect of minimally invasive thoracoscopic surgery; among them, the complexity of the medical record is analyzed from the electronic medical record information, and the complexity of the overall surgery is determined before the operation, which helps to analyze the complex situation of the location of the patient's mediastinal tumor; from the real-time change of the CT image, the degree of movement and deflection of the thoracoscopic perspective at the current moment is analyzed, and then the perspective change at the current moment is analyzed. From the perspective change, the invasion situation of the surgical operation is determined, that is, the larger the perspective deflection, the more likely it is to invade other structures during the operation. Therefore, through the degree of movement and deflection, accurate and rapid analysis can be carried out, improving the timeliness and reliability of the analysis; the processing of ultrasonic waveforms can effectively perform similarity comparison on important structures, so as to accurately locate the manifestation of important structures. Therefore, by combining the complexity of the medical record, the degree of movement and deflection, and the degree of structural manifestation, the perspective sensitivity degree of the thoracoscope at the current moment is determined, and a surgical operation assistance reminder is performed according to the perspective sensitivity degree, so as to comprehensively consider the complex condition of the location of the mediastinal tumor before the operation and the important structures that may be touched during the real-time operation, and make a guiding assistance reminder, improving the reliability of surgical guidance and assistance. Description of the Drawings

[0039] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0040] Figure 1 It is a structural diagram of a guiding and assisting system for minimally invasive thoracoscopic surgery of mediastinal tumors provided by an embodiment of the present invention;

[0041] Figure 2 It is a schematic diagram of mediastinal clinical zoning provided by an embodiment of the present invention;

[0042] Figure 3 It is a CT schematic diagram of a tumor lesion area provided by an embodiment of the present invention. Detailed implementation manners

[0043] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following, in combination with the accompanying drawings and preferred embodiments, details the specific implementation manners, structures, features and effects of a minimally invasive thoracoscopic surgery guidance and assistance system for mediastinal tumors proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0045] The mediastinum is located in the center of the human chest cavity, which involves important structures such as the heart, large blood vessels, trachea, esophagus, etc. Based on the common four-part division method of the mediastinum, it can be roughly divided into four parts (i.e., the superior mediastinum, anterior mediastinum, middle mediastinum, and posterior mediastinum), and mediastinal tumors may be located in any part. For different parts, different important structures will be involved. For example, the posterior mediastinum contains the trachea, esophagus, etc., and the middle mediastinum contains the heart, large blood vessels, etc.

[0046] Regardless of which area the tumor is located in, its size and invasiveness will also significantly affect the difficulty of the surgery. Especially when mediastinal tumors are distributed at the intersection of different parts, during the surgery, it is necessary to simultaneously consider and handle the complex structures from these two areas to avoid damaging any important structure (for example, mediastinal tumors may simultaneously invade the thymus (located in the anterior mediastinum) and large blood vessels (located in the superior mediastinum), which will greatly affect the complexity of the surgery.

[0047] The following specifically describes the specific solution of a minimally invasive thoracoscopic surgery guidance and assistance system for mediastinal tumors provided by the present invention in combination with the accompanying drawings.

[0048] Please refer to Figure 1 , which shows the structure diagram of a minimally invasive thoracoscopic surgery guidance and assistance system for mediastinal tumors provided by an embodiment of the present invention. The system includes: an acquisition module 101, a medical record analysis module 102, a CT analysis module 103, a waveform analysis module 104, and a reminder module 105. The embodiments of the present invention mainly assist in reminding the minimally invasive thoracoscopic surgery for mediastinal tumors at the current moment through the analysis of three dimensions: medical records, CT, and waveforms. The specific introduction of each module includes:

[0049] The acquisition module 101 is used to acquire electronic medical record information and real-time acquire chest CT images and ultrasonic waveforms through an ultrasonic detector;

[0050] The medical record analysis module 102 is used to determine the complexity of the medical record of the mediastinal tumor location according to the keywords in the electronic medical record information;

[0051] The CT analysis module 103 is used to determine the central point position of the thoracic cavity CT image, and determine the movement and deflection degree of the thoracoscope view at the current moment according to the image distance between the pixel points at the same image position of the thoracic cavity CT image at the current moment and the previous moment and the central point position, as well as the gray level difference of the pixel points;

[0052] The waveform analysis module 104 is used to determine the similarity between the ultrasonic waveform at the current moment and the preset structure waveforms of different structures in the thoracic cavity according to the comparison between the ultrasonic waveform at the current moment and the preset structure waveforms of different structures in the thoracic cavity, and determine the structural expression degree of the thoracoscope view at the current moment according to the similarity. Among them, different structures in the thoracic cavity at least include the heart, large blood vessels and spine;

[0053] The reminder module 105 is used to combine the medical record complexity, movement and deflection degree, and structural expression degree to determine the view sensitivity degree of the thoracoscope at the current moment, and perform surgical operation assistance reminder according to the view sensitivity degree.

[0054] A specific introduction to the content of the embodiments of the present invention is as follows:

[0055] For mediastinal tumors, they are located in the center of the thoracic cavity and involve important structures such as the heart, large blood vessels, trachea, and esophagus. Generally, the mediastinum is divided into four regions by the four-point method. The four-point method takes the connection line between the sternal angle and the lower edge of the fourth thoracic vertebra as the boundary and is divided into the upper and lower mediastinum. Among them, the lower mediastinum is divided into the anterior, middle, and posterior regions with the pericardium as the boundary. See Figure 2 , Figure 2 which is a schematic diagram of mediastinal clinical division provided by an embodiment of the present invention.

[0056] The medical record analysis module 102 is a module used to implement the specific analysis of electronic medical record information, and can determine the complexity of the medical record of the mediastinal tumor location according to the keywords in the electronic medical record information.

[0057] The electronic medical record information of the patient contains the relevant analysis of the mediastinal tumor by the doctor through relevant equipment before the operation, including specific information such as positional and severity descriptions, which can help analyze the complexity of the location of the patient's mediastinal tumor.

[0058] Before a patient with a mediastinal tumor undergoes minimally invasive thoracoscopic surgery, the doctor will use equipment such as CT or magnetic resonance to determine the size of the tumor and analyze the location of the tumor, and the preoperative tumor information description will be reflected in the electronic medical record information. Therefore, the relevant preoperative tumor information description in the patient's electronic medical record information can be extracted to determine the relevant conditions of the patient's tumor before the operation.

[0059] Among them, the electronic medical record information is an electronic medical record form. The specific method for obtaining the complexity of the medical record further includes: extracting keywords from the text in the electronic medical record form to determine a medical record keyword group; calculating the intersection ratio of the number of identical keywords between the medical record keyword group and a preset keyword group for mediastinal tumors as the complexity of the medical record for the mediastinal tumor location.

[0060] Among them, keyword extraction can include various extraction methods. Preferably, in the embodiments of the present invention, word segmentation processing is performed on the text in the electronic medical record form based on the jieba word segmentation algorithm to obtain medical record word segments; mediastinal keywords are extracted from the medical record word segments based on preset characteristic nouns in the oncology department, where the mediastinal keywords form a medical record keyword group.

[0061] It should be noted that the jieba word segmentation algorithm is a well-known word segmentation method in the art. Of course, in some other embodiments of the present invention, other word segmentation methods can also be used, such as the hidden Markov model, n-gram model, etc. Of course, a recurrent neural network (RNN), long short-term memory network (LSTM) combined with CRF, or Transformer model can also be used to automatically learn word segmentation rules through a large amount of data training, and this is not limited.

[0062] After word segmentation processing, medical record word segments of the electronic medical record form are obtained. The medical record word segments contain word segments related to specific mediastinal tumors and also irrelevant word segments. For specific analysis, in the embodiments of the present invention, mediastinal keywords can be extracted from the medical record word segments based on preset characteristic nouns in the oncology department.

[0063] Among them, the preset characteristic nouns in the oncology department are characteristic nouns related to mediastinal tumors preset by the oncology department. A phrase of the preset characteristic nouns in the oncology department can be set in advance, and the medical record word segments are retrieved with this phrase. If the retrieval is consistent, the medical record word segments are used as mediastinal keywords, and all the mediastinal keywords in an electronic medical record form form a medical record keyword group.

[0064] Among them, the preset keyword group for mediastinal tumors can specifically be a phrase composed of preset characteristic nouns in the oncology department. That is to say, the complexity of the medical record for the mediastinal tumor location can be specifically calculated through the intersection ratio of the number of identical keywords between the medical record keyword group and the preset keyword group for mediastinal tumors.

[0065] Among them, the intersection ratio is the ratio of the number of repeated keywords in two phrases to the number of all keywords. The larger this value is, the more repeated keyword numbers there are and the larger the proportion in the overall preset keyword group for mediastinal tumors.

[0066] The Intersection over Union (IoU) represents the similarity between the two. When the similarity is higher, more tumor keywords are involved in the description of the electronic medical record, indicating that the location of the patient's mediastinal tumor is more likely to be close to the junction of different mediastinal regions. Thus, the corresponding distribution of the patient's mediastinal tumor will be more complex. That is to say, the more complex the situation is, the more relevant text is required to describe it in the medical record. Therefore, the complexity of the medical record can be directly determined.

[0067] The CT analysis module 103 is used to determine the central point of the chest CT image, and based on the image distance between the pixel points at the same image position of the chest CT image at the current moment and the previous moment and the central point, as well as the gray-scale difference of the pixel points, determine the degree of movement and deflection of the thoracoscope view at the current moment.

[0068] During the process of the patient undergoing mediastinal tumor surgery, the thoracoscope will display real-time relevant image information of the patient's mediastinal tumor. Usually, doctors will adjust the view of the thoracoscope to observe the size of the mediastinal tumor from different angles, the distribution of the surrounding tissues, and perform different surgical cutting operations, etc., to ensure the accuracy and safety of the surgery (the larger the view deflection, the more likely it is to invade other structures during the operation). See Figure 3 , Figure 3 It is a schematic diagram of the CT of the tumor lesion area provided by an embodiment of the present invention.

[0069] However, the adjustment of the view of the thoracoscope will cause changes in the image gray scale. Therefore, the degree of movement and deflection of the current thoracoscope can be determined through the gray-scale change performance of adjacent frame images.

[0070] Furthermore, based on the image distance between the pixel points at the same image position of the chest CT image at the current moment and the previous moment and the central point, as well as the gray-scale difference of the pixel points, determining the degree of movement and deflection of the thoracoscope view at the current moment includes: determining the Euclidean distance between the pixel points at any image position and the central point as the central distance corresponding to the image position; normalizing the reciprocal of the central distance as the distance influence degree, where the distance influence degree of the central point is 1; determining the degree of movement and deflection of the thoracoscope view at the current moment according to the gray-scale difference and the distance influence degree of the pixel points at each image position at the current moment and the previous moment.

[0071] In the embodiment of the present invention, the image center is used as the specific offset analysis center. During the actual offset process, various different offset methods such as translation and rotation may occur. In the actual surgical process, the important area will be subconsciously placed at the center position. Therefore, the weight of the center is higher, and the farther the distance, the lower the corresponding weight. Thus, the reciprocal of the central distance is normalized as the distance influence degree, and the distance influence degree of the central point is set to 1.

[0072] Thus, the movement and deflection degree of the thoracoscope view at the current moment can be calculated by combining the gray-scale change and the distance influence degree. Further, in some embodiments of the present invention, the product of the gray-scale difference and the distance influence degree of pixel points at the same image position is calculated to obtain a position influence index; the mean value of all position influence indexes is normalized to obtain the movement and deflection degree.

[0073] Among them, since the gray-scale change of the overall CT image is small in the same structural area, and changes will occur due to offset in the cross-structural area. Therefore, the greater the gray-scale difference of pixel points at the same image position, the greater the possibility of image offset at the corresponding position, and the more cross-regional textures, the greater the movement and deflection degree. And because the central position is usually an area with more obvious attention, further weighted determination is performed through the distance influence degree to obtain a position influence index, and the mean value of all position influence indexes is statistically calculated and normalized to obtain the movement and deflection degree.

[0074] In the embodiments of the present invention, there are various ways to analyze the movement and offset of CT images, such as key point matching based on image analysis, etc. Using the method of gray-scale change analysis is more convenient and fast. During the minimally invasive thoracoscopic surgery with high real-time requirements, the results analyzed by the method of the embodiments of the present invention can, while meeting the requirements of basic auxiliary needs, process data more quickly and rapidly obtain specific parameters of the accurate and reliable movement and offset degree.

[0075] The waveform analysis module 104 is used to determine the similarity between the ultrasonic waveform at the current moment and the preset structure waveforms of different structures in the chest cavity according to the comparison between the ultrasonic waveform at the current moment and the preset structure waveforms of different structures in the chest cavity, and determine the structural representation degree of the thoracoscope view at the current moment according to the similarity. Among them, different structures in the chest cavity at least include the heart, large blood vessels, and spine.

[0076] As the thoracoscope moves and deflects in the body, the doctor will perform corresponding operations on the patient's mediastinal tumor surgery. To ensure that the doctor's surgical treatment avoids damaging other important internal structures, the proximity to important structures in the chest cavity should be fully considered. Therefore, by synchronously moving the thoracoscope and the ultrasonic detector, ultrasonic waveform data can be obtained in real time.

[0077] Because there are usually specific ultrasonic waveforms for important structures in the chest cavity, the ultrasonic waveform obtained in real time can be compared with the ultrasonic waveforms of each important structure. When the current ultrasonic waveform data is more similar to the ultrasonic waveform data of an important structure in the chest cavity, it means that the current position of the thoracoscope is closer to an important structure in the current chest cavity.

[0078] Further, in some embodiments of the present invention, the similarity between the ultrasonic waveform at the current moment and the preset structure waveforms of different structures in the thoracic cavity is determined by comparing the ultrasonic waveform at the current moment with the preset structure waveforms of different structures, including: performing dynamic time warping processing on the ultrasonic waveform and each preset structure waveform respectively to obtain the DTW values of the ultrasonic waveform and each preset structure waveform; determining the similarity between the ultrasonic waveform and the preset structure waveforms of different structures according to the comparison of the DTW values of the ultrasonic waveform and each preset structure waveform.

[0079] Among them, the preset structure fluctuation is the waveform obtained by performing ultrasonic analysis on the normal structure in advance, which represents the fluctuation information of the normal structure, that is, the ultrasonic waveform at the current moment is compared with the preset structure waveform to determine the similarity.

[0080] In the embodiments of the present invention, the dynamic time warping algorithm is mainly used as the specific fluctuation comparison analysis method. Therefore, dynamic time warping processing is performed on the ultrasonic waveform and each preset structure waveform respectively to obtain the DTW values of the ultrasonic waveform and each preset structure waveform. It should be noted that the larger the value of the DTW value, the greater the difference between the two fluctuations. Therefore, in the embodiments of the present invention, the similarity between the ultrasonic waveform and the preset structure waveforms of different structures is determined according to the comparison of the DTW values of the ultrasonic waveform and each preset structure waveform, including: performing maximum-minimum normalization processing on the reciprocal of the DTW value of the ultrasonic waveform and any preset structure waveform to obtain the similarity between the ultrasonic waveform and the corresponding preset structure waveform.

[0081] That is, the similarity is obtained by calculating the reciprocal and performing maximum-minimum normalization processing. The higher the similarity, the greater the similarity between the ultrasonic waveform and the preset structure waveform of the corresponding structure.

[0082] Then, the structural representation degree can be calculated according to the similarity. The structural representation degree of the thoracoscope view at the current moment is determined according to the similarity, including: taking the preset number of preset structure waveforms with the largest similarity values as the approaching waveforms, setting the additional attention degree of the approaching waveforms to 1, and setting the additional attention degree of other preset structure waveforms to 0; calculating the product of the similarity between the ultrasonic waveform and each preset structure waveform and the additional attention degree to obtain the proximity index between the ultrasonic waveform and each preset structure waveform; calculating the mean value of the proximity indexes between the ultrasonic waveform and all preset structure waveforms, and performing maximum-minimum normalization processing as the structural representation degree of the thoracoscope view at the current moment.

[0083] Since each structure has a similarity value, numerical analysis is required. In the embodiments of the present invention, the preset number can be set to 3, that is, the 3 preset structure waveforms with the largest similarity values are taken as the approaching waveforms, and the additional attention degree of the approaching waveforms is set to 1, and the additional attention degree of other preset structure waveforms is set to 0, and weighted by the unit values 1 and 0 to avoid the influence of irrelevant structures.

[0084] After that, calculate the product of the similarity and the extra attention degree between the ultrasonic waveform and each preset structure waveform to obtain the proximity index between the ultrasonic waveform and each preset structure waveform. Then, this proximity index represents the index information indicating that the position of the current thoracoscope is getting closer to a certain structure in the current thoracic cavity. Thus, calculate the mean value of the proximity indexes between the ultrasonic waveform and all preset structure waveforms, and perform maximum-minimum normalization processing as the structural representation degree of the thoracoscope view at the current moment.

[0085] The reminder module 105 is used to determine the view sensitivity of the thoracoscope at the current moment by combining the complexity of the medical record, the degree of movement and deflection, and the structural representation degree, and perform surgical operation assistance reminder according to the view sensitivity.

[0086] In the embodiment of the present invention, by analyzing three dimensions of electronic medical record information, view change, and ultrasonic waveform, the view sensitivity of the thoracoscope at the current moment can be combined with the information of the three dimensions.

[0087] Further, in some embodiments of the present invention, determining the view sensitivity of the thoracoscope at the current moment by combining the complexity of the medical record, the degree of movement and deflection, and the structural representation degree includes: calculating the product of the complexity of the medical record, the degree of movement and deflection, and the structural representation degree, and performing maximum-minimum normalization to obtain the view sensitivity.

[0088] Since the larger the values of the complexity of the medical record, the degree of movement and deflection, and the structural representation degree are, the more complex and sensitive the scene information is shown, such as the more complex the medical record, the larger the view deflection amplitude, and the higher the similarity to the important structure. Thus, the more sensitive the view performance is, the more surgical assistance reminder is needed.

[0089] Therefore, directly calculate the product value of the data obtained from the three dimensions, and perform maximum-minimum normalization processing to obtain the view sensitivity. Further, performing surgical operation assistance reminder according to the view sensitivity includes: when the view sensitivity is greater than the preset sensitivity threshold, performing surgical operation sensitivity reminder.

[0090] Among them, the preset sensitivity threshold is the threshold value of the view sensitivity. Optionally, the preset sensitivity threshold can be specifically, for example, 0.8, that is to say, when the view sensitivity is greater than 0.8, perform surgical operation sensitivity reminder.

[0091] The surgical operation sensitivity reminder in the embodiment of the present invention can be specifically, for example, performing a lighting reminder, that is, when a sensitive situation is detected, attract the attention of relevant personnel by changing the lighting. Of course, it can also be other reminder methods such as vibration, and there is no limitation on this.

[0092] Embodiments of the present invention achieve the guiding and assisting effect of minimally invasive thoracoscopic surgery by combining real-time analysis of three dimensions: electronic medical record information, CT images, and ultrasonic waveforms. Among them, the complexity of the medical record is analyzed from the electronic medical record information, and the complexity of the overall surgery is determined before the operation, which helps to analyze the complexity of the location of the patient's mediastinal tumor. From the real-time changes in CT images, the degree of movement and deflection of the thoracoscopic view at the current moment is analyzed, and then the view change at the current moment is analyzed. From the view change, the invasion situation of the surgical operation is determined. That is, the larger the view deflection, the more likely it is to invade other structures during the operation. Therefore, through the degree of movement and deflection, accurate and rapid analysis can be carried out, improving the timeliness and reliability of the analysis. The processing of ultrasonic waveforms can effectively compare the similarities of important structures, so as to accurately locate the performance of important structures. Thus, by combining the complexity of the medical record, the degree of movement and deflection, and the degree of structural manifestation, the view sensitivity of the thoracoscope at the current moment is determined, and surgical operation assistance reminders are made according to the view sensitivity, so as to comprehensively consider the complex conditions of the location of the mediastinal tumor before the operation and the important structures that may be touched during the real-time operation, and make guiding assistance reminders to improve the reliability of surgical guidance and assistance.

[0093] It should be noted that the above sequence of embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0094] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the differences between each embodiment and other embodiments are emphasized.

Claims

1. A minimally invasive thoracoscopic surgery guidance auxiliary system for mediastinal tumors, characterized in that: include: An acquisition module is used to acquire electronic medical record information and obtain chest CT images and ultrasound waveforms in real time through an ultrasound detector; A medical record analysis module is used to determine the complexity of the medical records of the mediastinal tumor site based on the keywords of the electronic medical record information; The CT analysis module is used to determine the center point of the chest CT image, and determine the degree of movement and deflection of the thoracoscope viewing angle at the current moment according to the image distance between the pixel point at the same image position in the chest CT image at the current moment and the previous moment and the center point, as well as the grayscale difference of the pixel point; A waveform analysis module, configured to determine the similarity between the ultrasonic waveform at the current moment and the preset structural waveforms of different structures in the chest cavity by comparing the ultrasonic waveform at the current moment with the preset structural waveforms of different structures, and determine the structural expression of the thoracoscopy viewing angle at the current moment according to the similarity, wherein the different structures in the chest cavity include at least the heart, great blood vessels and spine; The reminder module is used to determine the visual sensitivity of the thoracoscope at the current moment based on the complexity of the medical record, the degree of movement and deflection, and the degree of structural expression, and to provide auxiliary reminders for surgical operations based on the visual sensitivity.

2. A minimally invasive thoracoscopic surgery guidance assistance system for mediastinal tumors as claimed in claim 1, characterized in that: The electronic medical record information is an electronic medical record sheet, and determining the complexity of the medical record of the mediastinal tumor location based on keywords in the electronic medical record information includes: Extracting keywords from the text in the electronic medical record to determine a medical record keyword group; The intersection and union ratio of the number of identical keywords in the medical record keyword group and the preset mediastinal tumor keyword group is calculated as the complexity of the medical record of the mediastinal tumor site.

3. A minimally invasive thoracoscopic surgery guidance auxiliary system for mediastinal tumors as claimed in claim 2, characterized in that: The method for obtaining the medical record keyword group includes: Perform word segmentation processing on the text in the electronic medical record based on the Jieba word segmentation algorithm to obtain medical record word segmentation; The mediastinum keywords in the medical record segmentation are extracted based on the preset characteristic nouns of the oncology department, wherein the mediastinum keywords constitute the medical record keyword group.

4. The minimally invasive thoracoscopic surgery guidance auxiliary system for mediastinal tumors according to claim 1, characterized in that: Determining the degree of movement and deflection of the thoracoscope viewing angle at the current moment according to the image distance between the pixel point at the same image position of the chest CT image at the current moment and the previous moment and the center point, and the grayscale difference of the pixel point, includes: Determine the Euclidean distance between the pixel point and the center point at any image position as the center distance of the corresponding image position; The reciprocal of the center distance is normalized as the distance influence degree, wherein the distance influence degree of the center point is 1; The movement and deflection degree of the thoracoscope viewing angle at the current moment is determined according to the grayscale difference of the pixel point at each image position between the current moment and the previous moment and the influence degree of the distance.

5. A minimally invasive thoracoscopic surgery guidance auxiliary system for mediastinal tumors as claimed in claim 4, characterized in that: Determining the degree of movement and deflection of the thoracoscope viewing angle at the current moment according to the grayscale difference and distance influence degree of the pixel point at each image position at the current moment and the previous moment includes: Calculate the product of the grayscale difference of the pixel points at the same image position and the distance influence degree to obtain the position influence index; The mean values ​​of all position influence indicators are normalized to obtain the degree of movement deflection.

6. The minimally invasive thoracoscopic surgery guidance auxiliary system for mediastinal tumors according to claim 1, characterized in that: The step of comparing the ultrasonic waveform at the current moment with the preset structural waveforms of different structures in the chest cavity to determine the similarity between the ultrasonic waveform and the preset structural waveforms of different structures includes: Performing dynamic time warping processing on the ultrasonic waveform and each preset structure waveform respectively to obtain DTW values ​​of the ultrasonic waveform and each preset structure waveform; The similarity between the ultrasonic waveform and the preset structural waveforms of different structures is determined based on the comparison of the DTW values ​​between the ultrasonic waveform and each preset structural waveform.

7. A minimally invasive thoracoscopic surgery guidance auxiliary system for mediastinal tumors as claimed in claim 6, characterized in that: The step of comparing the DTW value of the ultrasonic waveform with each preset structure waveform to determine the similarity between the ultrasonic waveform and the preset structure waveforms of different structures includes: The reciprocals of the DTW values ​​of the ultrasonic waveform and any preset structure waveform are normalized to the maximum and minimum values ​​to obtain the similarity between the ultrasonic waveform and the corresponding preset structure waveform.

8. The minimally invasive thoracoscopic surgery guidance auxiliary system for mediastinal tumors according to claim 1, characterized in that: Determining the structural representation of the thoracoscopy view at the current moment according to the similarity includes: Taking a preset number of preset structure waveforms with the largest similarity values ​​as approaching waveforms, setting the additional attention of the approaching waveforms to 1, and setting the additional attention of other preset structure waveforms to 0; Calculate the product of the similarity between the ultrasonic waveform and each preset structure waveform and the additional attention degree to obtain a proximity index between the ultrasonic waveform and each preset structure waveform; The mean of the proximity index between the ultrasonic waveform and all preset structural waveforms is calculated, and the maximum and minimum values ​​are normalized as the structural expression of the thoracoscopic perspective at the current moment.

9. The minimally invasive thoracoscopic surgery guidance auxiliary system for mediastinal tumors according to claim 1, characterized in that: Combining the complexity of the medical record, the degree of movement and deflection, and the degree of structural expression, determining the visual sensitivity of the thoracoscope at the current moment includes: The product of the complexity of the medical history, the degree of movement and deflection, and the degree of structural expression is calculated, and the maximum and minimum values ​​are normalized to obtain the viewing angle sensitivity.

10. The minimally invasive thoracoscopic surgery guidance auxiliary system for mediastinal tumors according to claim 1, characterized in that: According to the visual angle sensitivity, surgical operation auxiliary reminders are provided, including: When the viewing angle sensitivity is greater than a preset sensitivity threshold, a surgical operation sensitive reminder is performed.

Images