Surgical data management methods, devices, equipment, media and products

Abstract

The present application discloses a surgical data management method, apparatus, equipment, medium, and product, relating to the medical field. The method includes obtaining surgical data, the surgical data including audio data and endoscopic video data; performing entity recognition on the audio data to obtain a first keyword; the first keyword including the name of a first lesion; performing image recognition on the endoscopic video data to obtain a second keyword; the second keyword including the name of a second lesion; generating a lesion analysis conclusion based on the first lesion name and the second lesion name; obtaining a management tag based on the first keyword and the second keyword, and storing the surgical data and the lesion analysis conclusion based on the management tag. The present application can combine the endoscopic video data and audio data during surgery to quickly identify the surgical data and generate tags to store the surgical data, thereby facilitating the management and utilization of the surgical data.

Description

Technical Field

[0001] The present application relates to the medical field, and in particular to a surgical data management method, device, equipment, medium and product. Background Art

[0002] An endoscope typically consists of a camera and light source and is inserted into the body through natural orifices (such as the digestive tract or respiratory tract) or small incisions. Endoscopes are commonly used to observe internal structures. In many cases, endoscopes are also equipped with small tools to perform specific tasks, such as biopsy, polyp removal, or hemostasis, enabling surgical procedures. Endoscopic surgery has become a common minimally invasive surgical procedure in modern medicine.

[0003] Data management for endoscopic surgery generally involves video recording and storage management of the endoscope. Endoscopic surgery videos are of great significance for subsequent surgical review, medical dispute resolution, and teaching management.

[0004] In the prior art, surgical video data is usually stored and managed manually, making it difficult to quickly retrieve and utilize. Summary of the Invention

[0005] The purpose of this application is to provide a surgical data management method, device, equipment, medium and product that can combine endoscopic video data and audio data during surgery to quickly identify surgical data and generate tags to store surgical data, so as to facilitate the management and utilization of surgical data.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a surgical data management method, comprising:

[0008] Acquiring surgical data, wherein the surgical data includes audio data and endoscopic video data;

[0009] Performing entity recognition on the audio data to obtain a first keyword; the first keyword includes a name of a first lesion;

[0010] Performing image recognition on the endoscopic video data to obtain a second keyword; the second keyword includes a second lesion name;

[0011] generating a lesion analysis conclusion according to the first lesion name and the second lesion name;

[0012] A management tag is acquired according to the first keyword and the second keyword, and the surgical data and the lesion analysis conclusion are stored according to the management tag.

[0013] Optionally, the second keyword further includes a second surgical procedure name and a second instrument name; and performing image recognition on the endoscopic video data to obtain the second keyword includes:

[0014] Sampling the endoscopic video data to obtain a first endoscopic image;

[0015] Preprocessing the first endoscopic image to obtain a second endoscopic image;

[0016] Performing instrument recognition on the second endoscope image according to the pre-trained preset first recognition model to obtain the second instrument name; and obtaining the corresponding second surgical procedure name according to the second instrument name;

[0017] The second endoscopic image is subjected to lesion recognition based on the pre-trained preset second recognition model to obtain the second lesion name.

[0018] Optionally, performing entity recognition on the audio data to obtain the first keyword includes:

[0019] Perform speech recognition on audio data to obtain target text;

[0020] Entity recognition is performed on the target text according to a pre-trained entity recognition model to obtain a first keyword and a corresponding entity category; wherein the first keyword also includes a first instrument name, a first surgical procedure name, and a lesion site name.

[0021] Optionally, the entity categories include "instrument", "procedure", "site" and "lesion", and the acquiring of management tags according to the first keyword and the second keyword, and the storing of the surgical data and the lesion analysis conclusion according to the management tags include:

[0022] Acquiring the corresponding entity category according to a mapping relationship between the second keyword and the entity category, wherein the mapping relationship includes: the second instrument name corresponds to the “instrument”, the second procedure name corresponds to the “procedure”, and the second lesion name corresponds to the “lesion”;

[0023] Deduplication operation is performed on the first keyword and the second keyword to obtain a management tag;

[0024] Storing the surgical data and the lesion analysis conclusion according to the management tag;

[0025] A classification index is generated according to the entity category corresponding to the management category.

[0026] Optionally, generating a lesion analysis conclusion according to the first lesion name and the second lesion name includes:

[0027] Determining whether there is an entity in the second lesion name that does not belong to the first lesion name;

[0028] If so, the entity that does not belong to the first lesion and the name of the first lesion are used as the lesion analysis conclusion; wherein the name of the first surgical lesion also includes a corresponding prompt mark;

[0029] If not, the first lesion name is used as the lesion analysis conclusion.

[0030] Optionally, the surgical data further includes a condition analysis and a surgical statement. After performing a deduplication operation on the first keyword and the second keyword to obtain a management tag, the surgical data management method further includes:

[0031] Obtaining condition analysis and surgical content description through a preset large language model according to the management label;

[0032] After storing the surgical data and the lesion analysis conclusion according to the management tag, the surgical data management method further includes:

[0033] Obtaining a search tag, and displaying a surgical data storage link and corresponding basic information that matches the management tag;

[0034] If target surgical data and teaching instructions are obtained, the endoscopic video data and audio data corresponding to the target surgical data, as well as the lesion analysis conclusion, the condition analysis, and the surgical content statement are played;

[0035] If target surgical data and teaching assessment instructions are obtained, the endoscopic video data corresponding to the target surgical data is played, an assessment recording is recorded, and the assessment recording is analyzed.

[0036] In a second aspect, the present application provides a surgical data management device, comprising:

[0037] an acquisition module, configured to acquire surgical data, wherein the surgical data includes audio data and endoscopic video data;

[0038] an identification module, configured to perform entity recognition on the audio data to obtain a first keyword; the first keyword includes a name of a first lesion;

[0039] Also used for performing image recognition on the endoscopic video data to obtain a second keyword; the second keyword includes a second lesion name;

[0040] an analysis module, configured to generate a lesion analysis conclusion based on the first lesion name and the second lesion name;

[0041] A storage module is used to obtain a management tag according to the first keyword and the second keyword, and store the surgical data and the lesion analysis conclusion according to the management tag.

[0042] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the surgical data management methods described above.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the surgical data management methods described above.

[0044] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the surgical data management methods described above.

[0045] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0046] The present application provides a surgical data management method, apparatus, equipment, medium and product. By combining endoscopic video data and audio data during surgery, entity recognition is performed on the audio data to obtain a first keyword; image recognition is performed on the endoscopic video data to obtain a second keyword; and management tags are obtained based on the first keyword and the second keyword. The management tags can quickly identify and store surgical data. The management tags are used to manage and store surgical videos. The management tags can serve as a supplement to basic information. When searching for surgical videos, surgical data can be retrieved by entering one or more management tags, making storage and use more flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 This is a flowchart of a surgical data management method in one embodiment of the present application;

[0049] Figure 2 for Figure 1 Schematic diagram of the refinement method flow chart of step 103;

[0050] Figure 3 for Figure 1 Schematic diagram of the refinement method flow of step 102;

[0051] Figure 4 for Figure 1 Schematic diagram of the refinement method flow chart of step 105;

[0052] Figure 5 for Figure 1 Schematic diagram of the refinement method flow of step 104;

[0053] Figure 6 A flowchart of a surgical data management method provided in another embodiment of the present application;

[0054] Figure 7 A schematic diagram of the functional modules of a surgical data management device provided in one embodiment of the present application;

[0055] Figure 8 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0057] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0058] In an exemplary embodiment, Figure 1 As shown, a surgical data management method is provided, including the following steps 101 to 105. In which:

[0059] Step 101: Acquire surgical data, which includes audio data and endoscopic video data;

[0060] Specifically, the surgical data here include endoscopic surgical data and endoscopic examination data.

[0061] Furthermore, the surgical data also includes its basic information, such as surgery number, basic surgical information, patient information, etc.

[0062] Step 102: Perform entity recognition on the audio data to obtain a first keyword; the first keyword includes the name of the first lesion;

[0063] Specifically, the audio data is first converted into text, and then entity recognition is performed to obtain the first keyword.

[0064] Step 103: Perform image recognition on the endoscope video data to obtain a second keyword; the second keyword includes the name of the second lesion;

[0065] Specifically, image recognition is performed on pictures in the endoscope video data to obtain the second keyword.

[0066] Specifically, the first keyword and the second keyword both include keywords of different categories, for example, the name of the lesion is one of the categories.

[0067] Step 104, generating a lesion analysis conclusion based on the first lesion name and the second lesion name;

[0068] Specifically, the lesion analysis conclusion is obtained by combining the first lesion name and the second lesion name. The first lesion name is the lesion manually judged in the operating room, and the second lesion name is the lesion name obtained based on image recognition technology. The first lesion name may be the same as or different from the second lesion name. When the first lesion name is different from the second lesion name, the second lesion name is used as a supplement to the first lesion name to generate the lesion analysis conclusion.

[0069] Step 105 : Obtain a management tag according to the first keyword and the second keyword, and store the surgical data and lesion analysis conclusion according to the management tag.

[0070] Specifically, a management tag is obtained based on the first keyword and the second keyword, and the data and lesion analysis conclusions are stored; the management tag is used as the basis for retrieval, and when extracting surgical data, the retrieval of the target surgical data can be completed by entering or selecting one or more management tags.

[0071] Specifically, the management tags obtained according to this method include multiple tags of the same or different categories. Compared with the hierarchical classified list data management, the method provided by the embodiment of the present application can be more flexible in storage and retrieval.

[0072] Implement the above steps 101 to 105, by combining the endoscopic video data and audio data during the operation, perform entity recognition on the audio data to obtain the first keyword; perform image recognition on the endoscopic video data to obtain the second keyword; obtain a management tag based on the first keyword and the second keyword, the management tag can quickly identify and store the surgical data, the management tag is used to manage and store the surgical video, the management tag can serve as a supplement to the basic information, when searching the surgical video, the surgical data can be retrieved by entering one or more management tags, and the storage and use are more flexible. In addition, the present application can also obtain a second lesion name by performing lesion analysis on the video image by identifying the audio data, and use the second lesion name as a supplement to the first lesion name to generate a lesion analysis conclusion, which can better improve the lesion analysis results and avoid missed detections and false detections.

[0073] In another exemplary embodiment of the present application, further, the second keyword includes the second instrument name, the second procedure name and the second lesion name. In order to more accurately perform image recognition on the endoscopic video to obtain the second keyword, such as Figure 2 As shown, step 103 can be replaced by the following steps 201 to 204:

[0074] Step 201: sampling endoscopic video data to obtain a first endoscopic image;

[0075] Specifically, the endoscope video data is sampled at a preset time interval to obtain a first endoscope image.

[0076] Step 202: pre-process the first endoscopic image to obtain a second endoscopic image;

[0077] Exemplarily, the first endoscopic image is subjected to image denoising, image enhancement and normalization processing in sequence to obtain the second endoscopic image.

[0078] Further optionally, a homomorphic filtering algorithm is used for image enhancement, which can perform illumination correction on the endoscopic image and effectively enhance the image features.

[0079] Step 203: Perform instrument recognition on the second endoscope image based on the pre-trained preset first recognition model to obtain a second instrument name; and obtain a corresponding second surgical procedure name based on the second instrument name.

[0080] For example, if this step is used in gastroenterology, the second instrument names include biopsy forceps, snare, electrosurgery, hemostatic clip, and duodenoscope; the second procedure name corresponding to biopsy forceps is polyp sampling, the second procedure name corresponding to snare is polypectomy, the second procedure name corresponding to electrosurgery is ESD (Endoscopic Submucosal Dissection), the second procedure name corresponding to hemostatic clip is gastrointestinal bleeding, and the second procedure name corresponding to duodenoscope is ERCP (Endoscopic Retrograde Cholangio-Pancreatography).

[0081] As an exemplary embodiment, a first feature extraction method is performed on the second endoscope image to obtain a first feature, and the first feature is input into a preset first recognition model to obtain an output second instrument name;

[0082] As an optional embodiment, the first feature extraction method is: grayscale conversion and edge detection are performed on the second endoscopic image in sequence to obtain the first feature.

[0083] Furthermore, the pre-trained preset first recognition model is obtained according to the following steps:

[0084] Step 2031: Acquire endoscopic images including and excluding surgical instruments, and perform image denoising, image enhancement, normalization, and the first feature extraction method on the endoscopic images in sequence according to the above-mentioned method to obtain a second feature;

[0085] Step 2032: Mark the second feature, where the mark includes the second instrument name and the absence of surgical instruments. The second feature and the corresponding mark are used as a first data set. The first data set is used to obtain a first training set and a first test set according to a preset ratio. The preset first recognition model is trained using the first training set.

[0086] Step 2033: Test and evaluate the trained preset first recognition model based on the first test set. If the evaluation meets the criteria, the model is used as the pre-trained preset first recognition model.

[0087] As another implementation, the Mask R-CNN model (Mask Region-based Convolutional Neural Network) is used as the preset first recognition model. Accordingly, when acquiring the first dataset, professional image annotation tools (such as LabelMe and CVAT) are used to label the images using bounding boxes and pixel set masks.

[0088] Specifically, Mask R-CNN is a deep learning model for object detection and instance segmentation that can simultaneously identify object locations, classify them, and generate pixel-level masks.

[0089] Step 204: performing lesion recognition on the second endoscopic image according to the pre-trained preset second recognition model to obtain a second lesion name, wherein the preset second recognition model is the pre-trained preset first recognition model;

[0090] Specifically, the second lesion names include polyp, ulcer, tumor and no lesion.

[0091] As an exemplary embodiment, a second feature extraction method is performed on the second endoscopic image to obtain a second feature, and the second feature is input into a preset first recognition model to obtain an output second instrument name;

[0092] Specifically, the second feature extraction method is:

[0093] Step 2041 , performing threshold segmentation on the second endoscopic image to obtain a target area;

[0094] Step 2042: convert the second endoscope image into an HSV color space image, and obtain the color features of the target area based on the HSV color space image;

[0095] Step 2043: extracting the energy, homogeneity, and correlation of the target area as texture features;

[0096] Step 2044 , extracting one or more of the contour, circularity, and convex hull defects of the target area as morphological features;

[0097] Step 2045: performing feature fusion or feature splicing on the color feature, texture feature, and morphological feature to obtain a third feature;

[0098] Specifically, feature-level weighted fusion can be performed, or the weights of each feature can be learned through adaptive weight learning using a neural network.

[0099] Specifically, the preset second recognition model is obtained according to the following steps:

[0100] Endoscopic images including and excluding lesions are collected, a third feature is obtained from the endoscopic images according to the above steps 2041 to 2045, the third feature is marked, the mark includes the name of the second lesion, and the third feature and the corresponding mark are used as the second data set. A second training set and a second test set are obtained from the second data set according to a preset ratio, and the preset first recognition model is trained with the second training set; the trained preset first recognition model is tested and evaluated based on the second test set. If the evaluation meets the standards, it is used as the final pre-trained preset second recognition model.

[0101] Specifically, the preset second recognition model includes a neural network model, a logistic regression model, a SVM model, a decision tree model, and a random forest model.

[0102] As another embodiment, a pre-trained Mask R-CNN model is used as the preset second recognition model. Accordingly, when acquiring the second dataset, a professional image annotation tool (such as LabelMe or CVAT) is used to label the endoscopic image using a bounding box and a pixel set mask. The pre-trained Mask R-CNN model is trained based on the second dataset to obtain a second recognition model. The second endoscopic image is then input into the second recognition model to obtain a lesion recognition result.

[0103] In another exemplary embodiment of the present application, in order to obtain the first keyword more accurately, such as Figure 3 As shown, step 102 can be replaced by steps 301 to 305, and steps 301 to 305 specifically include:

[0104] Step 301: Perform speech recognition on the audio data to obtain the target text;

[0105] Specifically, you can directly use existing open source models for speech recognition to obtain the target text, such as Whisper, DeepSpeech, and Wav2Vec speech recognition systems.

[0106] Step 302 , performing entity recognition on the target text according to the pre-trained entity recognition model to obtain a first keyword and a corresponding entity category; wherein the first keyword includes a first instrument name, a first procedure name, a lesion site name, and a first lesion name.

[0107] The pre-trained entity recognition model is obtained through the following steps:

[0108] Collect literature related to the department's surgery, and use the department's materials as a text set. Perform word segmentation preprocessing and other preprocessing steps on the text set to obtain a first text, and mark the first text. The mark includes entities and non-entities, and the entities also include corresponding entity types; the marked first text is used as a third data set, and the third training set and the third test set are obtained from the third data set according to a preset ratio. The preset entity recognition model is trained with the third training set; the preset entity recognition model is tested and evaluated according to the third test set to obtain a pre-trained entity recognition model.

[0109] Specifically, the first text is tagged using the BIO tagging method, wherein one or more of B, I, and O are tagged in a sentence tag. B- indicates the beginning of an entity, for example, B-Person indicates the beginning of a person's name. I- indicates that it is within an entity but not the beginning, for example, I-Person indicates the subsequent part of a person's name. O indicates that it does not belong to any entity.

[0110] Specifically, the first keyword includes the name of the first instrument, the name of the first surgical procedure, the name of the lesion site, and the name of the first lesion;

[0111] Specifically, the first instrument name, the first surgical procedure name, the lesion site name, and the first lesion name are entity categories, and may be labeled as "instrument," "procedure," "site," and "lesion," for example, respectively;

[0112] Furthermore, for example, in the first text related to gastroenterology, the entity categories of "polyp", "ulcer", "tumor", "xx tumor", etc. are marked as "lesions", and the entity categories of "esophagus", "stomach", "duodenum", "colon", "rectum", "bile tract" and "pancreatic duct" are marked as "sites"; the entity categories of "polyp sampling", "polypectomy", "ESD surgery", "gastrointestinal blood" and "ERCP surgery" are marked as "procedures".

[0113] In another exemplary embodiment of the present application, in order to further set the management tag and improve the index mechanism, such as Figure 4 As shown, based on the above steps, step 105 can also be replaced by the following steps:

[0114] Step 401: Obtain a corresponding entity category based on a mapping relationship between a second keyword and an entity category, wherein the mapping relationship includes: the second instrument name corresponds to "instrument", the second procedure name corresponds to "procedure", and the second lesion name corresponds to "lesion";

[0115] Specifically, the mapping relationship between the second keyword and the entity category is matched with the entity category in the first keyword. The second instrument name obtained in step 203 corresponds to "instrument", and the second procedure name obtained corresponds to "procedure"; the second lesion name obtained in step 204 corresponds to "lesion".

[0116] Step 402: De-duplicate the first keyword and the second keyword to obtain a management tag;

[0117] Furthermore, as a method, the surgical data also includes basic information, such as the patient's name, department, surgery name, surgical site, surgeon, etc. One or more of the above basic information can be pre-selected as the third keyword, and the first keyword, second keyword, and third keyword are deduplicated to obtain a management tag;

[0118] Furthermore, the third keyword also includes a mapping relationship of the corresponding entity category, which is used to generate a classification index in step 404; specifically, the entity category corresponding to the third keyword can match the entity category in the first keyword, or it can complement the entity category in the first keyword. For example, if the surgeon is included as one of the entity categories, then a new entity category needs to be added to the existing entity category.

[0119] Step 403: storing the surgical data and lesion analysis conclusions according to the management tags;

[0120] Specifically, the management tag serves as a search tag for the database, and the surgical data can be searched by inputting one or more search tags.

[0121] Step 404: Generate a classification index based on the entity category corresponding to the management category.

[0122] Furthermore, based on the entity categories generated by the entity categories, the surgical data can be classified, sorted and arranged according to the entity categories, making information search more convenient.

[0123] In another exemplary embodiment of the present application, in order to provide a method for accurately obtaining lesion analysis conclusions, such as Figure 5 As shown, the above step 104 can also be replaced by the following steps:

[0124] Step 501, determining whether there is an entity in the second lesion name that does not belong to the first lesion name;

[0125] Step 502: If it exists, the entity that does not belong to the first lesion and the name of the first lesion are used as the lesion analysis conclusion; wherein, the name of the first surgical lesion also includes a corresponding prompt mark.

[0126] As an embodiment, entities belonging to the first lesion name but not to the second lesion name are marked as a reference for the doctor's diagnosis.

[0127] For example, directly labeled as reference: tumor (reference), polyp.

[0128] Step 503: If the first lesion name does not exist, the first lesion name is used as the lesion analysis conclusion.

[0129] Specifically, the first lesion name is obtained through entity recognition of the audio content and is based on the surgeon's diagnosis. The second lesion name is obtained through lesion analysis of the video image and is based on the judgment of the machine learning model. To avoid missed or false detections, the second lesion name is used as a supplement to the first lesion name to generate the lesion analysis conclusion, which can better improve the lesion analysis results.

[0130] As another embodiment, the surgical data management method provided by this application can also be used in a school or hospital intern environment to complete teaching tasks and teaching assessment goals, such as Figure 6 The surgical data management method includes steps 601 to 608, specifically including:

[0131] Step 601: Acquire surgical data, which includes audio data and endoscopic video data;

[0132] Specifically, this is similar to step 101 and will not be described in detail here.

[0133] Step 602: Perform entity recognition on the audio data to obtain a first keyword; the first keyword includes the name of the first lesion;

[0134] Specifically, this is similar to step 102 and will not be described in detail here.

[0135] Step 603: Perform image recognition on the endoscope video data to obtain a second keyword; the second keyword includes the name of the second lesion;

[0136] Specifically, this is similar to step 103 and will not be described in detail here.

[0137] Step 604: generating a lesion analysis conclusion based on the first lesion name and the second lesion name;

[0138] Specifically, this is similar to step 104 and will not be described in detail here.

[0139] Step 605: Obtain a management tag based on the first keyword and the second keyword, and obtain a condition analysis and surgical content statement based on the management tag using a preset large language model;

[0140] Specifically, the first keyword and the second keyword are deduplicated to obtain a management label, and the management label is input into a preset large language model for context filling to obtain a condition analysis and a surgical content statement, and the surgical data and lesion analysis conclusions are stored according to the management label, wherein the surgical data includes a surgical condition analysis and a surgical content statement.

[0141] Specifically, the preset large language model needs to be equipped with a professional surgical literature database and medical field knowledge graph to improve the accuracy of context filling.

[0142] Step 606 , obtaining a search tag, and displaying and managing surgical data storage links and corresponding basic information that match the tag;

[0143] Specifically, the above steps 601 to 605 are used to store the surgical data, and step 606 and the following steps are used to perform user interaction based on the stored data;

[0144] Specifically, the search tag is a management tag that the user searches or selects, and the management tag is matched according to the search tag to obtain one or more matching surgical data storage links and corresponding basic information.

[0145] Step 607: If the target surgical data and teaching instructions are obtained, the endoscopic video data and audio data corresponding to the target surgical data, as well as the lesion analysis conclusion, condition analysis, and surgical content description are played;

[0146] Specifically, the user selects one of the surgical data storage links based on the matching one or more surgical data storage links and the corresponding basic information. The target surgical data can be determined based on the user's selection, and the teaching instruction is the teaching function selected by the user.

[0147] Specifically, the condition analysis and surgical content description are played or displayed as explanations of the surgery, and the lesion analysis conclusion is the conclusion of the surgical data.

[0148] Step 608: If the target surgical data and the teaching assessment instructions are obtained, the endoscopic video data corresponding to the target surgical data is played, the assessment recording is recorded, and the assessment recording is analyzed.

[0149] Furthermore, the teaching instruction is an examination function selected by the user, and the endoscopic video data is played to the examinee's terminal. The endoscopic video data refers to silent video data. At the same time, the examinee's defense audio is recorded and the audio is analyzed.

[0150] Furthermore, the assessment recording can be transcribed into text to obtain a second text, and the second text can be identified to determine whether the second text contains a management tag corresponding to the target surgical data. If so, additional points can be awarded to the examinee.

[0151] Furthermore, different scores can be set for management tags, and points can be added through corresponding scores.

[0152] The present application also provides an application scenario, which applies the above-mentioned surgical data management method.

[0153] Specifically, the surgical data management method provided in this embodiment can be applied to the surgical data management of a hospital or hospital department. By obtaining a management tag based on the first keyword and the second keyword, the surgical data can be quickly identified and stored to facilitate its management and utilization. In addition, the present application can also obtain a second lesion name by performing lesion analysis on a video image by recognizing audio data, and use the second lesion name as a supplement to the first lesion name to generate a lesion analysis conclusion, which can better improve the lesion analysis results and avoid missed detections and false detections.

[0154] Specifically, the surgical data management method provided in this embodiment can also be applied to an intern teaching system in a hospital or hospital department to complete teaching tasks and teaching assessment goals in a school or hospital intern environment.

[0155] Based on the same inventive concept, embodiments of the present application also provide a surgical data management device for implementing the aforementioned surgical data management method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more surgical data management device embodiments provided below can be found in the aforementioned limitations of the surgical data management method and will not be further elaborated here.

[0156] In an exemplary embodiment, Figure 7 As shown, a surgical data management device is provided, comprising:

[0157] An acquisition module is used to acquire surgical data, including audio data and endoscopic video data;

[0158] A recognition module, configured to perform entity recognition on the audio data to obtain a first keyword; the first keyword includes a name of a first lesion;

[0159] Also used for performing image recognition on the endoscope video data to obtain a second keyword; the second keyword includes a second lesion name;

[0160] An analysis module, configured to generate a lesion analysis conclusion based on the first lesion name and the second lesion name;

[0161] The storage module is used to obtain a management tag according to the first keyword and the second keyword, and store the surgical data and lesion analysis conclusion according to the management tag.

[0162] As an optional implementation, the second keyword also includes a second procedure name and a second instrument name; the identification module is further used to:

[0163] Sampling the endoscope video data to obtain a first endoscope image;

[0164] Preprocessing the first endoscopic image to obtain a second endoscopic image;

[0165] Performing instrument recognition on the second endoscope image according to the pre-trained preset first recognition model to obtain a second instrument name; and obtaining a corresponding second surgical procedure name according to the second instrument name;

[0166] The second lesion name is obtained by performing lesion recognition on the second endoscopic image according to the pre-trained preset second recognition model.

[0167] As an optional implementation manner, the identification module is further configured to:

[0168] Perform speech recognition on audio data to obtain target text;

[0169] Entity recognition is performed on the target text according to a pre-trained entity recognition model to obtain a first keyword and a corresponding entity category; wherein the first keyword also includes a first instrument name, a first surgical procedure name, and a lesion site name.

[0170] As an optional implementation, the entity categories include "instrument", "procedure", "site" and "lesion", and the storage module is further used to:

[0171] Obtaining a corresponding entity category according to a mapping relationship between the second keyword and the entity category, wherein the mapping relationship includes: the second instrument name corresponds to "instrument", the second procedure name corresponds to "procedure", and the second lesion name corresponds to "lesion";

[0172] De-duplicate the first keyword and the second keyword to obtain a management tag;

[0173] Store surgical data and lesion analysis conclusions according to management tags;

[0174] Generate a classification index based on the entity category corresponding to the management category.

[0175] As an optional implementation, the analysis module is further configured to:

[0176] Determining whether there is an entity in the second lesion name that does not belong to the first lesion name;

[0177] If it exists, the entity that does not belong to the first lesion and the name of the first lesion are used as the lesion analysis conclusion; wherein, the name of the first surgical lesion also includes a corresponding prompt mark;

[0178] If it does not exist, the name of the first lesion will be used as the lesion analysis conclusion.

[0179] As an optional implementation manner, the storage module is further configured to:

[0180] Obtain condition analysis and surgical content description based on management labels through a preset large language model;

[0181] As an optional embodiment, the surgical data management device further includes an interaction module for:

[0182] Obtain search tags, display and manage surgical data storage links and corresponding basic information that match the tags;

[0183] If the target surgical data and teaching instructions are obtained, the endoscopic video data and audio data corresponding to the target surgical data, as well as the lesion analysis conclusion, condition analysis and surgical content description are played;

[0184] If target surgical data and teaching assessment instructions are obtained, the endoscopic video data corresponding to the target surgical data is played, an assessment recording is recorded, and the assessment recording is analyzed.

[0185] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a surgical data management method is implemented.

[0186] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0187] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0188] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0189] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0190] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0191] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0192] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0193] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0194] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of this application. In summary, the content of this specification should not be construed as limiting this application.

Claims

1. A surgical data management method, characterized in that: The surgical data management method comprises: Acquiring surgical data, wherein the surgical data includes audio data and endoscopic video data; Performing speech recognition on the audio data to obtain a target text; performing entity recognition on the target text according to a pre-trained entity recognition model to obtain a first keyword and a corresponding entity category; the first keyword includes a first lesion name, a first instrument name, a first surgical procedure name, and a lesion site name; Performing image recognition on the endoscopic video data to obtain a second keyword; the second keyword includes a second lesion name, a second surgical procedure name, and a second instrument name; generating a lesion analysis conclusion according to the first lesion name and the second lesion name; Acquire the corresponding entity category according to a mapping relationship between the second keyword and the entity category, wherein the entity category includes instruments, procedures, parts, and lesions, and the mapping relationship includes: the second instrument name corresponds to the instrument, the second procedure name corresponds to the procedure, and the second lesion name corresponds to the lesion; Deduplication operation is performed on the first keyword and the second keyword to obtain a management tag; Storing the surgical data and the lesion analysis conclusion according to the management tag; A classification index is generated according to the entity category corresponding to the management tag.

2. The surgical data management method according to claim 1, wherein: The performing image recognition on the endoscope video data to obtain the second keyword includes: Sampling the endoscopic video data to obtain a first endoscopic image; Preprocessing the first endoscopic image to obtain a second endoscopic image; Performing instrument recognition on the second endoscope image according to the pre-trained preset first recognition model to obtain the second instrument name; and obtaining the corresponding second surgical procedure name according to the second instrument name; The second endoscopic image is subjected to lesion recognition based on the pre-trained preset second recognition model to obtain the second lesion name.

3. The surgical data management method according to any one of claims 1 to 2, characterized in that: Generating a lesion analysis conclusion according to the first lesion name and the second lesion name includes: Determining whether there is an entity in the second lesion name that does not belong to the first lesion name; If so, the entity that does not belong to the first lesion and the name of the first lesion are used as the lesion analysis conclusion; wherein the name of the first surgical lesion also includes a corresponding prompt mark; If not, the first lesion name is used as the lesion analysis conclusion.

4. The surgical data management method according to claim 1, wherein: The surgical data also includes a condition analysis and a surgical statement. After performing a deduplication operation on the first keyword and the second keyword to obtain a management tag, the surgical data management method further includes: Obtaining condition analysis and surgical content description through a preset large language model according to the management label; After storing the surgical data and the lesion analysis conclusion according to the management tag, the surgical data management method further includes: Obtain a search tag, where the search tag is a management tag searched or selected by the user; Displaying the surgical data storage link and corresponding basic information that matches the management tag; if target surgical data and teaching instructions are obtained, playing the endoscopic video data and audio data corresponding to the target surgical data, as well as the lesion analysis conclusion, the condition analysis, and the surgical content statement; If target surgical data and teaching assessment instructions are obtained, the endoscopic video data corresponding to the target surgical data is played, an assessment recording is recorded, and the assessment recording is analyzed.

5. A surgical data management device, characterized in that: The surgical data management device comprises: an acquisition module, configured to acquire surgical data, wherein the surgical data includes audio data and endoscopic video data; a recognition module configured to perform speech recognition on the audio data to obtain a target text; perform entity recognition on the target text according to a pre-trained entity recognition model to obtain a first keyword and a corresponding entity category; the first keyword includes a first lesion name, a first instrument name, a first surgical procedure name, and a lesion site name; Also used for performing image recognition on the endoscopic video data to obtain a second keyword; the second keyword includes a second lesion name, a second surgical procedure name, and a second instrument name; an analysis module, configured to generate a lesion analysis conclusion based on the first lesion name and the second lesion name; Storage modules for: Acquire the corresponding entity category according to a mapping relationship between the second keyword and the entity category, wherein the entity category includes instruments, procedures, parts, and lesions, and the mapping relationship includes: the second instrument name corresponds to the instrument, the second procedure name corresponds to the procedure, and the second lesion name corresponds to the lesion; Deduplication operation is performed on the first keyword and the second keyword to obtain a management tag; Storing the surgical data and the lesion analysis conclusion according to the management tag; A classification index is generated according to the entity category corresponding to the management tag.

6. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the surgical data management method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the surgical data management method according to any one of claims 1 to 4 are implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the surgical data management method according to any one of claims 1 to 4 are implemented.

Images