Arthroscopic surgery data processing method and system based on cloud platform

Abstract

The invention discloses an arthroscopic surgery data processing method and system based on a cloud platform, and the method comprises the steps: obtaining arthroscopic surgery data, creating a database, and obtaining an arthroscopic surgery database; carrying out key frame extraction operation according to the arthroscopic surgery database to obtain a key frame image; inputting the key frame image into a pre-trained image semantic segmentation model to obtain an effective frame image; performing lesion area information extraction operation according to the effective frame image to obtain lesion area information; performing matching operation according to the lesion area information, the effective frame image and the arthroscopic surgery database to obtain a structured text; performing semantic structure tree generation operation according to the structured text to obtain an arthroscopic surgery semantic structure tree; and according to the arthroscopic surgery semantic structure tree, performing visual processing operation for query and data mining. The method can realize automatic extraction and classification of arthroscopic surgery data.

Description

Technical Field

[0001] The present invention relates to the technical field of surgical data processing, and in particular, to an arthroscopic surgical data processing method and system based on a cloud platform. Background Art

[0002] At present, arthroscopic surgery, as a minimally invasive surgical technique, has been widely used in recent years. By integrating observation and operation, it can directly observe the internal structure of a patient's joint. With the increase in the number of surgeries, hospitals have accumulated a large number of surgical video resources, which are often not fully utilized and lack unified management and timely processing. Although existing surgical teaching systems can provide support for medical education through virtual reality technology and real-time image transmission, there are still deficiencies in the integration of surgical videos, preoperative examinations, intraoperative registration forms, and postoperative pathology reports. Manual operations are still required when recording relevant information, which is inefficient and prone to errors. Moreover, the correct understanding and analysis of arthroscopic surgical videos are often limited to surgeons with a certain level of seniority. These problems limit the application value of surgical video resources in medical research, teaching, and clinical practice. It is urgent to achieve automatic extraction, classification, and structured storage of surgical videos through technical means to improve the utilization efficiency of surgical data.

[0003] As a giant company in the orthopedic joint medical field, Smith & Nephew has proposed a method for arthroscopic video analysis and a device therefor (WO2021 / 168408A1). However, its purpose is to provide automatic segmentation of videos to facilitate the guidance of surgeons in the finally recorded videos. It is limited to the real-time feed of arthroscopic surgical videos and tends to only analyze and utilize the video data of the current surgical object, lacking macro guidance significance.

[0004] It can be seen that although surgical videos have important value in medical education and clinical practice, the current management methods often fail to effectively integrate videos with relevant medical records. Surgical videos exist in an independent form and lack direct connection with information such as preoperative examinations, intraoperative registration forms, and postoperative pathology reports. This information island phenomenon leads to the inability to comprehensively understand the patient's condition and surgical process when making surgical decisions, thus affecting the surgical effect and the patient's prognosis. In addition, even with a large number of surgical videos as references and guidance, there are difficulties in storing and retrieving surgical videos. Due to the lack of unified management and classification standards, it often takes a lot of time and effort to find specific surgical videos.

[0005] In summary, there is a problem in the prior art that automatic extraction and classification of arthroscopic surgical data cannot be achieved. Summary of the Invention

[0006] The present invention provides an arthroscopic surgery data processing method and system based on a cloud platform to achieve automatic extraction and classification of arthroscopic surgery data.

[0007] In a first aspect, to solve the above technical problems, the present invention provides an arthroscopic surgery data processing method based on a cloud platform, including:

[0008] Obtain arthroscopic surgery data, create a database, and obtain an arthroscopic surgery database;

[0009] According to the arthroscopic surgery database, perform key frame extraction operation on the surgery videos therein to obtain key frame images;

[0010] Input the key frame images into a pre-trained image semantic segmentation model to obtain effective frame images;

[0011] According to the effective frame images, perform lesion area information extraction operation to obtain lesion area information;

[0012] According to the lesion area information, the effective frame images, and the arthroscopic surgery database, perform a matching operation to obtain structured text;

[0013] According to the structured text, perform a semantic structure tree generation operation to obtain an arthroscopic surgery semantic structure tree;

[0014] According to the arthroscopic surgery semantic structure tree, perform a visualization processing operation for querying and data mining.

[0015] Preferably, in an alternative embodiment, obtaining arthroscopic surgery data, creating a database, and obtaining an arthroscopic surgery database includes:

[0016] Obtain arthroscopic surgery data during the arthroscopic surgery process;

[0017] According to the arthroscopic surgery data, perform data preprocessing operation to obtain first arthroscopic surgery data;

[0018] According to the first arthroscopic surgery data, generate a surgery database to obtain an arthroscopic surgery database.

[0019] Preferably, in an alternative embodiment, according to the arthroscopic surgery database, performing key frame extraction operation to obtain key frame images includes:

[0020] According to the arthroscopic surgery database, perform video stream extraction operation to obtain an arthroscopic surgery video stream;

[0021] According to the arthroscopic surgery video stream, perform video stream segmentation operation to divide it into multiple smaller video segments to obtain a segmented video stream;

[0022] Based on the segmented video stream, perform key frame extraction calculation to obtain images that can accurately reflect the key moments of the surgical process, and obtain key frame images;

[0023] The calculation formula for the key frame extraction calculation is as follows:

[0024] K j =γ j ·R(V(j)), j = 1, …, n

[0025] Where K j represents the key degree of the j-th frame image, γ j represents the dynamic discount factor of the j-th frame image, which is used to adjust the importance weight between video frames, R(V(j)) represents the reward function of the segmented video stream V(j), n represents the total number of video frames in the segmented video stream, and the reward function R(V(j)) is designed according to the content characteristics of the video frames. Frames with a key degree greater than a certain threshold are selected as key frames.

[0026] Preferably, in an alternative embodiment, input the key frame images into a pre-trained image semantic segmentation model to obtain effective frame images, including:

[0027] Input the key frame images into a pre-trained image semantic segmentation model to obtain key frame semantic information;

[0028] According to the key frame semantic information, perform an effective frame judgment operation to obtain effective frame images;

[0029] Wherein, the training process of the image semantic segmentation model includes:

[0030] Initialize the parameters of the image semantic segmentation model;

[0031] Use the arthroscopic surgery images and their corresponding labeled data sets to train the initialized image semantic segmentation model, and end when the accuracy of the model reaches the preset target value to obtain a pre-trained image semantic segmentation model.

[0032] Preferably, in an alternative embodiment, according to the effective frame images, perform a lesion area information extraction operation to obtain lesion area information, including:

[0033] According to the effective frame images, perform a first feature extraction operation to obtain lesion area features;

[0034] According to the lesion area features, perform a feature matching operation to obtain lesion area information.

[0035] Preferably, in an alternative embodiment, a matching operation is performed according to the lesion area information, the effective frame image, and the arthroscopic surgery database to obtain structured text, including:

[0036] A second feature extraction operation is performed according to the lesion area information and the effective frame image to obtain effective frame features;

[0037] A feature matching operation is performed according to the effective frame features and the arthroscopic surgery database to obtain structured text.

[0038] Preferably, in an alternative embodiment, a semantic structure tree generation operation is performed according to the structured text to obtain an arthroscopic surgery semantic structure tree, including:

[0039] A structure tree generation is performed according to the structured text using a hierarchical generation tree algorithm to obtain an arthroscopic surgery semantic structure tree;

[0040] The arthroscopic surgery semantic structure tree is stored in the arthroscopic surgery database through a storage operation.

[0041] Preferably, in an alternative embodiment, a visualization processing operation is performed according to the arthroscopic surgery semantic structure tree for querying and data mining, including:

[0042] A user interface establishment operation is performed according to the arthroscopic surgery semantic structure tree to obtain an arthroscopic surgery user interface;

[0043] An interactive query system establishment operation is performed according to the arthroscopic surgery semantic structure tree to obtain an arthroscopic surgery interactive query system.

[0044] In a second aspect, the present invention provides an arthroscopic surgery data processing system based on a cloud platform, including:

[0045] A database generation module for obtaining arthroscopic surgery data, creating a database, and obtaining an arthroscopic surgery database;

[0046] A key frame extraction module for performing a key frame extraction operation according to the arthroscopic surgery database to obtain key frame images;

[0047] A semantic segmentation module for inputting the key frame images into an image semantic segmentation model to obtain effective frame images;

[0048] A lesion area information extraction module for performing a lesion area information extraction operation according to the effective frame images to obtain lesion area information;

[0049] A matching module, configured to perform a matching operation according to the lesion area information, the effective frame images, and the arthroscopic surgery database to obtain a structured text;

[0050] A semantic structure tree generation module, configured to perform a semantic structure tree generation operation according to the structured text to obtain an arthroscopic surgery semantic structure tree;

[0051] An interaction module, configured to perform a visualization processing operation according to the arthroscopic surgery semantic structure tree for query and data mining.

[0052] In a third aspect, the present invention further provides a computer-readable storage medium, which includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute any one of the above-mentioned arthroscopic surgery data processing methods based on a cloud platform.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] (1) The present invention realizes the intelligent processing and analysis of arthroscopic surgery data, realizes the automatic extraction and classification of arthroscopic surgery data, captures the key steps during the surgery, accurately identifies the lesion areas in the images, separates these areas from the background, matches the lesion area information, the effective frame images with other medical records in the arthroscopic surgery database, generates a structured text, realizes the deep integration of surgical video data and related medical information, and provides a comprehensive and accurate view of surgical information; transforms complex and abstract surgical data into an intuitive and easy-to-understand graphical interface, improving the efficiency and convenience of query and data mining, which is extremely beneficial to medical education and clinical practice teaching;

[0055] (2) Through the visualization processing operation, the present invention presents complex surgical data in an intuitive and easy-to-understand manner, facilitating query and data mining; uses advanced video stream extraction and segmentation technologies to accurately separate key frame images from continuous surgical videos, providing key materials for subsequent in-depth analysis; improves the utilization efficiency of surgical data, enables rapid and convenient retrieval and viewing of specific surgical videos and their related medical records, thereby improving the efficiency and accuracy of surgical decision-making, contributing to improving the surgical effect and the prognosis of patients, greatly improving the timeliness and accuracy of treatment, and ensuring the correct selection and implementation of treatment measures;

[0056] (3) The present invention innovatively combines cloud platform technology with arthroscopic surgery data processing, which can effectively utilize a large amount of surgical video materials from multiple channels for the analysis, processing, and storage of arthroscopic surgery techniques. By setting unified management and classification standards for the large amount of surgical video materials, big data medicine becomes possible, which has breakthrough significance in the era of intelligent big data. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 FIG. 6 is a schematic flowchart of a method for processing arthroscopic surgery data based on a cloud platform provided by the first embodiment of the present invention;

[0058] Figure 2 FIG. 10 is a schematic structural diagram of a system for processing arthroscopic surgery data based on a cloud platform provided by the second embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0060] Referring to Figure 1 , the first embodiment of the present invention provides a method for processing arthroscopic surgery data based on a cloud platform, including the following steps:

[0061] S11. Obtain arthroscopic surgery data, create a database, and obtain an arthroscopic surgery database;

[0062] S12. Perform key frame extraction operation according to the arthroscopic surgery database to obtain key frame images;

[0063] S13. Input the key frame images into a pre-trained image semantic segmentation model to obtain effective frame images;

[0064] S14. Perform lesion area information extraction operation according to the effective frame images to obtain lesion area information;

[0065] S15. Perform a matching operation according to the lesion area information, the effective frame images, and the arthroscopic surgery database to obtain structured text;

[0066] S16. Perform a semantic structure tree generation operation according to the structured text to obtain an arthroscopic surgery semantic structure tree;

[0067] S17. Perform a visualization processing operation according to the arthroscopic surgery semantic structure tree for querying and data mining.

[0068] It should be noted that the present invention realizes the efficient processing and in-depth mining of arthroscopic surgery data through a series of ordered steps. In step S11, the system acquires the original data of arthroscopic surgery and performs structured processing on it to create an arthroscopic surgery database. In step S12, based on this database, the system accurately extracts key frame images, which capture the key moments during the surgery. Subsequently, in step S13, the key frame images are input into a pre-trained image semantic segmentation model, which can accurately identify and separate the lesion areas in the images and generate clear and effective frame images. In step S14, based on the effective frame images, the system further extracts detailed information of the lesion areas, such as appearance features and geometric features, etc., providing a basis for the understanding of surgical data. In step S15, the system intelligently matches the lesion area information with other medical records in the surgical database to generate structured text data, realizing the in-depth integration of surgical videos and related medical information and breaking the previous situation of isolated data. In step S16, based on the structured text, the system constructs an arthroscopic surgery semantic structure tree, providing a clear framework for the organization and presentation of surgical data. In step S17, according to the semantic structure tree, the system performs visualization processing, converting complex and abstract surgical data into an intuitive and easy-to-understand graphical interface, improving the efficiency and convenience of querying and data mining.

[0069] In step S11, arthroscopic surgery data is acquired, a database is created, and an arthroscopic surgery database is obtained, including:

[0070] Arthroscopic surgery data during the arthroscopic surgery process is acquired;

[0071] According to the arthroscopic surgery data, data preprocessing operations are performed to obtain the first arthroscopic surgery data;

[0072] According to the first arthroscopic surgery data, a surgical database is generated to obtain the arthroscopic surgery database.

[0073] It should be noted that in step S11, the process of the present invention for acquiring arthroscopic surgery data and creating a database mainly focuses on key features such as surgical video data, surgical instrument operation data, and patient physiological parameter data. The surgical video data is obtained by the arthroscopic device through real-time shooting during the surgery, which contains detailed image information of the internal structure of the joint and can intuitively reflect the condition of the surgical area. The surgical instrument operation data records the usage of various instruments during the surgery, such as the type of instrument, usage time, operation actions, etc., and these data are collected by sensors and monitoring devices in the operating room. The patient physiological parameter data includes indicators such as heart rate, blood pressure, and blood oxygen saturation, and these data are monitored in real time by medical monitoring devices and transmitted to the data acquisition system. To obtain these data, the arthroscopic device is equipped with a high-definition camera and an image transmission module, which can transmit the captured surgical video to the data acquisition system in real time; sensors are installed on the surgical instruments to detect and record the usage of the instruments; the physiological parameters of the patient are monitored in real time through various sensors connected to the monitoring device. These devices and sensors transmit the collected data to the central data processing system in wired and wireless ways. After obtaining the original data, the system first performs data preprocessing operations on it. Data preprocessing includes data cleaning, denoising, and formatting. The data cleaning process will identify and remove outliers and error information in the data, including blurred frames and noisy images in the surgical video; the denoising process filters out the noise components in the data through the Gaussian filtering algorithm to improve the accuracy of the data; the formatting process uniformly converts data from different sources and formats into a standard format that the system can efficiently process, such as converting video data into a unified encoding format and converting physiological parameter data into a standardized numerical form, etc. The data after preprocessing is called the first arthroscopic surgery data, which already has relatively high quality and consistency.

[0074] It is worth noting that the system generates a structured arthroscopic surgery database based on the first arthroscopic surgery data. The database is constructed using a relational database management system. A surgical video data table can be set up to store information such as the frame data, timestamp, and video quality of the surgical video; a surgical instrument operation data table can be set up to record the type of instrument, usage time, operation actions, etc.; a patient physiological parameter data table can be set up to store the real-time data of indicators such as heart rate, blood pressure, and blood oxygen saturation. By establishing the association relationships between the data tables, the surgical video data is associated with the corresponding surgical stages and instrument operation data, and the patient physiological parameter data is corresponding to the key events during the surgery, etc., realizing the integration and associated query of the data. This structured storage method not only improves the data retrieval efficiency but also provides convenience for subsequent data analysis and mining, enabling the system to quickly and accurately obtain various types of surgical data required.

[0075] In step S12, according to the arthroscopic surgery database, a key frame extraction operation is performed to obtain key frame images, including:

[0076] According to the arthroscopic surgery database, a video stream extraction operation is performed on the surgical videos therein to obtain an arthroscopic surgery video stream;

[0077] According to the arthroscopic surgery video stream, a video stream segmentation operation is performed to divide it into multiple smaller video segments to obtain a segmented video stream;

[0078] According to the segmented video stream, a key frame extraction calculation is performed to obtain images that can accurately reflect the key moments of the surgical process, resulting in key frame images;

[0079] The calculation formula for the key frame extraction calculation is as follows:

[0080] K j =γ j ·R(V(j)), j = 1, …, n;

[0081] where K j represents the key degree of the j-th frame image, γ j represents the dynamic discount factor of the j-th frame image, which is used to adjust the importance weight between video frames, R(V(j)) represents the reward function of the segmented video stream V(j), n represents the total number of video frames in the segmented video stream, and the reward function R(V(j)) is designed according to the content features of the video frames, and frames with a key degree greater than a certain threshold are selected as key frames.

[0082] It should be noted that in step S12, first, the system extracts the surgical video stream from the database, reads and decodes the video data stored in the database to obtain a continuous video frame sequence. Then, a segmentation operation is performed on the extracted video stream. The purpose is to decompose the long surgical video into multiple shorter segments, and each segment contains a certain number of continuous frames. This segmentation helps to perform more detailed analysis and processing on each segment. Based on the segmented video stream, a key frame extraction calculation is performed. The calculation formula for the key degree is: K j =γ j ·R(V(j)), j = 1, …, n, where K j represents the key degree of the j-th frame image, γ j is the dynamic discount factor set for the j-th frame image, which is used to adjust the importance of this frame relative to other frames; R(V(j)) is a reward function designed based on content features, which is used to measure the value of the j-th frame image in the video. n is the total number of video frames in the segmented video stream. The dynamic discount factor γ jThe determination is based on the dynamic changes in the video content. By analyzing the differences between frames, such as calculating the pixel differences between adjacent frames to evaluate the degree of motion in the video. If a frame has a significant change compared to the previous frame, indicating an important event or action during the surgery, the discount factor for this frame can be set relatively large to increase its key importance. Conversely, if the differences between frames are small, indicating that the video content is relatively stable, the discount factor can be set small to reduce the key importance of these frames. The design of the reward function R(V(j)) depends on the specific content features of the video frames. These features include the visual saliency of the frame (such as brightness, contrast, color distribution, etc.), the appearance frequency and position of important surgical structures or instruments, etc. For example, if there is an obvious surgical instrument operation or diseased tissue in a frame, the value of the reward function for this frame should be high, thereby increasing its key importance. Finally, by comparing the key importance of each frame with a preset threshold, the frames with a key importance higher than this threshold are selected as key frames. These key frames are considered to accurately reflect the critical moments during the surgery. By flexibly adjusting these parameters and functions, different surgical videos can be adapted to provide high-quality key frame images for subsequent semantic segmentation and lesion area information extraction.

[0083] In step S13, input the key frame image into a pre-trained image semantic segmentation model to obtain an effective frame image, including:

[0084] Input the key frame image into a pre-trained image semantic segmentation model to obtain key frame semantic information;

[0085] According to the key frame semantic information, perform an effective frame judgment operation to obtain an effective frame image;

[0086] Among them, the training process of the image semantic segmentation model includes:

[0087] Initialize the parameters of the image semantic segmentation model;

[0088] Use the arthroscopic surgery image and its corresponding labeled dataset to train the initialized image semantic segmentation model, and end when the accuracy of the model reaches a preset target value to obtain a pre-trained image semantic segmentation model. Exemplarily, the preset target value for the accuracy in this embodiment is 80%.

[0089] It should be noted that in step S13, the key frame image is input into a pre-trained image semantic segmentation model to obtain the effective frame image. The image semantic segmentation model is constructed based on a deep convolutional neural network, and its core function is to identify and distinguish different objects and regions in the image. During the training process, the model learns a large number of labeled arthroscopic surgery images and their corresponding labels, and masters how to extract features from the images and classify them. Specifically, the model analyzes the pixel-level features of the image, and uses network structures such as convolutional layers, pooling layers, and fully connected layers to gradually extract the local features and global features of the image, so as to achieve semantic understanding of each region in the image. When the accuracy of the model on the validation set reaches 90%, the training ends. The model can identify different categories such as normal tissues, lesion regions, and surgical instruments in the image, and assign corresponding category labels to each pixel. After obtaining the key frame image, the system inputs it into the trained image semantic segmentation model, and the model will output the semantic information of the key frame, that is, the category label to which each pixel belongs. Then, based on this semantic information, the system performs an effective frame judgment operation. Effective frames refer to those frames that contain important surgical information and changes, such as obvious changes in the lesion region and key operations of surgical instruments. By analyzing the key features in the semantic information, such as the area and shape changes of the lesion region, and the position changes of surgical instruments, the system can accurately identify the effective frame image. If the semantic information shows that the area of the lesion region in a certain key frame suddenly increases and the surgical instrument enters a new operation area, then this frame can be judged as an effective frame. Of course, the judgment criteria for effective frames can be adjusted according to the actual surgical scenario and requirements. For example, for some surgical types, factors such as the contact situation between the surgical instrument and the tissue and the changes in the bleeding area will also be considered.

[0090] In step S14, based on the effective frame image, a lesion region information extraction operation is performed to obtain lesion region information, including:

[0091] Based on the effective frame image, a first feature extraction operation is performed to obtain lesion region features;

[0092] Based on the lesion region features, a feature matching operation is performed to obtain lesion region information.

[0093] It should be noted that in step S14, based on the valid frame image, an operation for extracting lesion area information is performed to obtain the key features and information of the lesion area. First, the system performs a first feature extraction operation on the valid frame image to identify and extract the features of the lesion area from the image. Feature extraction is an important part of image processing and analysis, which can represent the useful information in the image in the form of features. During the process of lesion area feature extraction, the system will analyze the attributes of the image such as pixel values, textures, and shapes. The first feature corresponds to at least one of the above attributes. After extracting the above first feature of the valid frame image, the lesion area can be identified according to features such as boundaries, sizes, or colors. The system can detect abnormal texture and color areas in the image through image processing algorithms, identify them as lesion areas, and extract information such as the area, shape, and edge features of the area, that is, the lesion area features.

[0094] After obtaining the lesion area features, the system further performs a feature matching operation to obtain the detailed information of the lesion area. Feature matching is to compare the extracted features with the known lesion features to determine the specific information of the lesion area. For example, the system matches the extracted lesion area features with various lesion features stored in the database to identify information such as the type and severity of the lesion. Through feature matching, the system can correspond the features of the lesion area with the specific lesion information, providing an accurate basis for the understanding and analysis of surgical data. If the extracted lesion area features highly match a certain type of lesion features in the database, then the system can determine that the lesion area belongs to this type of lesion and obtain its corresponding lesion information.

[0095] In step S15, based on the lesion area information, the valid frame image, and the arthroscopic surgery database, a matching operation is performed to obtain structured text, including:

[0096] Based on the lesion area information and the valid frame image, a second feature extraction operation is performed to obtain valid frame features;

[0097] Based on the valid frame features and the arthroscopic surgery database, a feature matching operation is performed to obtain structured text.

[0098] It should be noted that in step S15, the system performs a matching operation to generate structured text, which is based on the lesion area information, the effective frame images, and the arthroscopic surgery database. First, the system extracts the second features from the effective frame images, and these features include texture, shape, color, position, and semantic information. The texture features are calculated through the gray-level co-occurrence matrix, reflecting the texture characteristics of the lesion area; the shape features are extracted through the contour detection algorithm, including area, perimeter, and shape factor; the color features are obtained by converting the pixel values into the HSV color space and calculating the average color value and color histogram; the position features determine the central coordinates and bounding boxes of the lesion area in the image; the semantic features come from the class labels output by the pre-trained image semantic segmentation model, identifying the semantic categories of the lesion area. These features are combined into a feature vector for matching with the data in the arthroscopic surgery database. The matching process is achieved by calculating the cosine similarity between the feature vectors. Through this calculation, the system can identify the feature vector that best matches the records in the database. The database stores a large amount of pre-annotated and classified surgical information, including different types of lesion characteristics and surgical operation steps. When the system finds the record that best matches the extracted second features, it generates structured text based on these matching results. The structured text details the key information during the surgery, such as the type and location of the lesion, the steps of the surgical operation, etc., providing accurate, detailed, and structured surgical records for medical staff. If the extracted features show that there is an inflammatory reaction in a certain area, the system will search the database for lesion characteristics related to inflammation and find the matching records. Subsequently, the system generates structured text, clearly indicating that there is an inflammatory reaction in a specific area at a specific moment during the surgery and detailing information such as the scope and degree of the inflammation. This process ensures that the key information extracted from the surgical video data can be effectively integrated with the medical knowledge base, improving the efficiency and accuracy of surgical data analysis.

[0099] In step S16, according to the structured text, a semantic structure tree generation operation is performed to obtain an arthroscopic surgery semantic structure tree, including:

[0100] According to the structured text, a hierarchical generation tree algorithm is used to generate the structure tree to obtain an arthroscopic surgery semantic structure tree;

[0101] The arthroscopic surgery semantic structure tree is stored through a storage operation in the arthroscopic surgery database.

[0102] It should be noted that in step S16, a semantic structure tree generation operation is performed based on the structured text to obtain an arthroscopic surgery semantic structure tree. First, the system analyzes and processes the structured text using a hierarchical tree generation algorithm. The hierarchical tree generation algorithm is an algorithm for constructing hierarchical structures, which can organize and display complex information according to certain logical relationships and hierarchical structures. In this process, the system identifies the key information and concepts in the structured text and constructs a hierarchical semantic structure based on their association relationships. The system can use the different stages in the surgical process as the first-level nodes of the tree, the key operations and events under each stage as the second-level nodes, and the specific lesion information, instrument usage, etc. as deeper-level nodes. Through this hierarchical method, the system can clearly display various information in the surgical process and their relationships, forming a complete arthroscopic surgery semantic structure tree. Then, the system stores the generated arthroscopic surgery semantic structure tree, storing it in the arthroscopic surgery database. In this way, it is not only convenient for subsequent data query and access, but also can be integrated and compared with other surgical data. By constructing the arthroscopic surgery semantic structure tree, a clear and intuitive framework is provided for the organization and presentation of surgical data, enabling doctors and researchers to more conveniently understand and analyze various information in the surgical process.

[0103] In step S17, based on the arthroscopic surgery semantic structure tree, a visualization processing operation is performed for query and data mining, including:

[0104] Based on the arthroscopic surgery semantic structure tree, a user interface establishment operation is performed to obtain an arthroscopic surgery user interface;

[0105] Based on the arthroscopic surgery semantic structure tree, an interactive query system establishment operation is performed to obtain an arthroscopic surgery interactive query system.

[0106] It should be noted that in step S17, the present invention performs visualization processing operations according to the arthroscopic surgery semantic structure tree to achieve intuitive display and convenient query of data. First, the system designs and establishes a user-friendly arthroscopic surgery user interface based on the information of the semantic structure tree. This interface presents the hierarchical information of the semantic structure tree in a graphical way, and various stages of the surgical process, key operations, and lesion information can be intuitively viewed through the interface. Different stages of the surgery can be represented by different colors and icons, and key operations and events can be used as clickable nodes. After clicking, detailed operation steps and relevant data can be viewed. In addition, the interface can also display images and feature information of the lesion area, such as the size, shape, and color of the lesion, as well as the content of the related pathology report. Then, the system establishes an interactive query system according to the arthroscopic surgery semantic structure tree. This system allows for rapid retrieval of relevant surgical data and information according to different query conditions, such as surgery type, lesion type, surgical stage, etc. By inputting keywords and selecting query options, the system will automatically match the corresponding nodes and data in the semantic structure tree and display the query results in a clear and intuitive manner. For example, one can query surgical cases of a specific lesion type, and the system will display all the surgical semantic structure trees containing this lesion type and highlight the lesion-related information for easy comparison and analysis. Through visualization processing and interactive query, the present invention improves the query and data mining efficiency of arthroscopic surgery data.

[0107] For the convenience of understanding the present invention, some preferred embodiments of the present invention will be further described below.

[0108] Perform data processing and analysis on a complex arthroscopic surgery. This surgery requires repairing the patient's knee meniscus injury and reconstructing the anterior cruciate ligament.

[0109] Step 1: Obtain arthroscopic surgery data and create a database. Collect surgical video data from the arthroscopic equipment in the operating room. Perform data preprocessing, including data cleaning, denoising, and formatting. Then, generate a structured arthroscopic surgery database. Using a relational database management system, set up a surgical video data table to store frame data, timestamps, and video quality information. The surgical procedure data table records the surgical stages and key steps. The patient information data table stores medical records and preoperative examination results.

[0110] Step 2: Key frame extraction operation. Based on the arthroscopic surgery database, perform key frame extraction operations to obtain images that can accurately reflect the key moments of the surgical process. Extract the surgical video stream from the database, which contains a continuous sequence of video frames during the surgery. Segment the video stream into multiple smaller video segments, each corresponding to a key step of the surgery. Then, perform key frame extraction calculations based on the segmented video stream. Use a specific calculation formula to calculate the importance weights of each video frame, and finally select the frame with the largest weight as the key frame. In the meniscus repair segment, the frame for checking the repair effect after completion of the repair is extracted as the key frame; in the ligament reconstruction segment, the inspection frame after ligament fixation is extracted as the key frame.

[0111] Step 3: Semantic segmentation operation. Input the extracted key frame images into a pre-trained image semantic segmentation model to obtain valid frame images. In the key frames of meniscus repair, the image semantic segmentation model identifies the repaired meniscus area as normal tissue and labels the surrounding joint cavity and surgical instrument areas respectively. Obtain the valid frame images.

[0112] Step 4: Lesion area information extraction operation. Based on the valid frame images, further extract the detailed information of the lesion area. Perform feature extraction operations on the valid frame images. In the key frames of meniscus injury repair, extract the edge features and surface texture features of the repaired meniscus. According to the extracted lesion area features, perform feature matching operations, comparing these features with various lesion features stored in the database to determine the specific information of the lesion area. Match the extracted meniscus features with the meniscus injury features in the database to identify information such as the type and degree of injury of the repaired meniscus.

[0113] Step 5: Matching operation to generate structured text. According to the lesion area information, valid frame images, and arthroscopic surgery database, perform intelligent matching operations to generate structured text data. Match the features of the lesion area with the lesion descriptions in the patient's pathology report, and accurately record the type, location, and size information of the lesion in the structured text. At the same time, match the surgical operations in the valid frame images with the operation records in the intraoperative registration form, and record the key operation steps and instrument usage during the surgery.

[0114] Step 6: Semantic structure tree generation operation. Based on the structured text, use a hierarchical generation tree algorithm to generate a semantic structure tree to obtain the arthroscopic surgery semantic structure tree. Store the generated semantic structure tree in the arthroscopic surgery database for subsequent data query and access.

[0115] Step 7: Visualization processing operation. According to the arthroscopic surgery semantic structure tree, perform user interface establishment operations to obtain the arthroscopic surgery user interface.

[0116] In summary, through a series of innovative steps, the present invention realizes the comprehensive, efficient processing and in-depth mining of arthroscopic surgery data. The original surgical data is acquired and a structured database is created. Key frame images are accurately extracted, and the pre-trained image semantic segmentation model is used to generate effective frame images. The extraction and matching operations of the lesion area information enable the in-depth integration of the surgical video and other medical records of the patient, and the generated structured text facilitates the further processing of the data. The construction of the semantic structure tree not only clearly presents the hierarchical structure of the surgical process but also provides an intuitive framework for the organization and presentation of the data. The final visualization processing operation transforms complex data into a graphical interface that is easy to understand and operate, improving the efficiency of query and data mining.

[0117] Referring to Figure 2 , the second embodiment of the present invention provides an arthroscopic surgery data processing system based on a cloud platform, including:

[0118] A database generation module, configured to acquire arthroscopic surgery data, create a database, and obtain an arthroscopic surgery database;

[0119] A key frame extraction module, configured to perform key frame extraction operations according to the arthroscopic surgery database to obtain key frame images;

[0120] A semantic segmentation module, configured to input the key frame images into an image semantic segmentation model to obtain effective frame images;

[0121] A lesion area information extraction module, configured to perform lesion area information extraction operations according to the effective frame images to obtain lesion area information;

[0122] A matching module, configured to perform matching operations according to the lesion area information, the effective frame images, and the arthroscopic surgery database to obtain structured text;

[0123] A semantic structure tree generation module, configured to perform semantic structure tree generation operations according to the structured text to obtain an arthroscopic surgery semantic structure tree;

[0124] An interaction module, configured to perform visualization processing operations according to the arthroscopic surgery semantic structure tree for query and data mining.

[0125] It should be noted that the arthroscopic surgery data processing system based on a cloud platform provided in the embodiment of the present invention is used to execute all the process steps of the arthroscopic surgery data processing method based on a cloud platform in the above embodiment. Their working principles and beneficial effects correspond one by one, so they will not be elaborated here.

[0126] An embodiment of the present invention also provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a lesion area information extraction program. When the processor executes the computer program, the steps in the above embodiments of each of the arthroscopic surgery data processing methods based on a cloud platform are implemented, such as Figure 1 step S11 shown. Alternatively, when the processor executes the computer program, the functions of each module / unit in the above device embodiments are implemented, such as an interaction module.

[0127] Exemplarily, the computer program may be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0128] The electronic device may be a computing device such as a desktop computer, a notebook, a palm computer, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above components are only examples of the electronic device and do not constitute a limitation on the electronic device. It may include more or fewer components than the above, or combine certain components, or different components. For example, the electronic device may further include input / output devices, network access devices, a bus, etc.

[0129] The so-called processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the electronic device and connects various parts of the entire electronic device through various interfaces and lines.

[0130] The memory can be used to store the computer programs and / or modules. By running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory, the processor can implement various functions of the electronic device. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory can include high-speed random access memory and can also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0131] Among them, if the modules / units integrated in the electronic device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, an executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disc, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0132] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the device embodiments provided by the present invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement this without creative efforts.

[0133] The specific embodiments described above have further elaborated on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. In particular, it is pointed out that for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An arthroscopic surgery data processing method based on a cloud platform, characterized in that, Executed by a computer, including: Obtain arthroscopic surgery data, create a database, and obtain an arthroscopic surgery database; According to the arthroscopic surgery database, perform a key frame extraction operation to obtain key frame images; Input the key frame images into a pre-trained image semantic segmentation model to obtain valid frame images; According to the valid frame images, perform a lesion area information extraction operation to obtain lesion area information; According to the lesion area information, the valid frame images, and the arthroscopic surgery database, perform a matching operation to obtain structured text; According to the structured text, perform a semantic structure tree generation operation to obtain an arthroscopic surgery semantic structure tree; According to the arthroscopic surgery semantic structure tree, perform a visualization processing operation for querying and data mining.

2. The arthroscopic surgery data processing method based on a cloud platform according to claim 1, wherein Obtain arthroscopic surgery data, create a database, and obtain an arthroscopic surgery database, including: Obtain arthroscopic surgery data during the arthroscopic surgery process; According to the arthroscopic surgery data, perform a data preprocessing operation to obtain first arthroscopic surgery data; According to the first arthroscopic surgery data, generate a surgery database to obtain an arthroscopic surgery database.

3. The arthroscopic surgery data processing method based on a cloud platform according to claim 1, wherein, According to the arthroscopic surgery database, perform a key frame extraction operation to obtain key frame images, including: According to the arthroscopic surgery database, perform a video stream extraction operation on the surgery video therein to obtain an arthroscopic surgery video stream; According to the arthroscopic surgery video stream, perform a video stream segmentation operation to divide it into multiple smaller video segments to obtain a segmented video stream; According to the segmented video stream, perform a key frame extraction calculation to obtain images that can accurately reflect the key moments of the surgery process to obtain key frame images; The calculation formula for the key frame extraction calculation is as follows: K j = γ j ·R(V(j)), j = 1, …, n Among them, K j represents the key degree of the j-th frame image, and γ j represents the dynamic discount factor of the j-th frame image, which is used to adjust the importance weight between video frames. R(V(j)) represents the reward function for segmenting the video stream V(j), n represents the total number of video frames in the segmented video stream. The reward function R(V(j)) is designed based on the content features of the video frames, and frames with a key degree greater than a certain threshold are selected as key frames.

4. The arthroscopic surgery data processing method based on a cloud platform according to claim 1, characterized in that Input the key frame images into a pre-trained image semantic segmentation model to obtain valid frame images, including: Input the key frame images into a pre-trained image semantic segmentation model to obtain key frame semantic information; According to the key frame semantic information, perform a valid frame judgment operation to obtain valid frame images; Among them, the training process of the image semantic segmentation model includes: Initialize the parameters of the image semantic segmentation model; Use the arthroscopic surgery images and their corresponding labeled datasets to train the initialized image semantic segmentation model, and end when the accuracy of the model reaches a preset target value to obtain a pre-trained image semantic segmentation model.

5. The arthroscopic surgery data processing method based on a cloud platform according to claim 1, wherein According to the valid frame images, perform a lesion area information extraction operation to obtain lesion area information, including: According to the valid frame images, perform a first feature extraction operation to obtain lesion area features; According to the lesion area features, perform a feature matching operation to obtain lesion area information.

6. The arthroscopic surgery data processing method based on a cloud platform according to claim 1, wherein According to the lesion area information, the valid frame images, and the arthroscopic surgery database, perform a matching operation to obtain structured text, including: According to the lesion area information and the valid frame images, perform a second feature extraction operation to obtain valid frame features; According to the valid frame features and the arthroscopic surgery database, perform a feature matching operation to obtain structured text.

7. The arthroscopic surgery data processing method based on a cloud platform according to claim 1, characterized in that Perform a semantic structure tree generation operation according to the structured text to obtain an arthroscopic surgery semantic structure tree, including: Perform a structure tree generation using a hierarchical generation tree algorithm according to the structured text to obtain an arthroscopic surgery semantic structure tree; Perform a storage operation on the arthroscopic surgery semantic structure tree and store it in an arthroscopic surgery database.

8. The arthroscopic surgery data processing method based on a cloud platform according to claim 1, wherein, Perform a visualization processing operation according to the arthroscopic surgery semantic structure tree for query and data mining, including: Perform a user interface establishment operation according to the arthroscopic surgery semantic structure tree to obtain an arthroscopic surgery user interface; Perform an interactive query system establishment operation according to the arthroscopic surgery semantic structure tree to obtain an arthroscopic surgery interactive query system.

9. An arthroscopic surgery data processing system based on a cloud platform, characterized in that, Including: A database generation module for obtaining arthroscopic surgery data, creating a database, and obtaining an arthroscopic surgery database; A key frame extraction module for performing a key frame extraction operation according to the arthroscopic surgery database to obtain key frame images; A semantic segmentation module for inputting the key frame images into an image semantic segmentation model to obtain effective frame images; A lesion area information extraction module for performing a lesion area information extraction operation according to the effective frame images to obtain lesion area information; A matching module for performing a matching operation according to the lesion area information, the effective frame images, and the arthroscopic surgery database to obtain structured text; A semantic structure tree generation module for performing a semantic structure tree generation operation according to the structured text to obtain an arthroscopic surgery semantic structure tree; An interaction module for performing a visualization processing operation according to the arthroscopic surgery semantic structure tree for query and data mining.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein when the computer program runs, it controls the device where the computer-readable storage medium is located to execute the arthroscopic surgery data processing method based on a cloud platform as described in any one of claims 1 to 8.

Images