Operation data AI optimal storage method based on deep learning

Abstract

The invention discloses an operation data AI optimal storage method based on deep learning, and relates to the technical field of medical information processing, and the method comprises the following steps: a video file uploading end uploads an operation video file to a system from an external device or a storage medium, and the operation video file is used as an input source of a whole video abstract processing flow; the system verifies the format of the uploaded video file, checks the size of the file, and ensures that no data loss or file damage occurs in the uploading process; for a large video file, a fragment uploading technology is supported, after file uploading is completed, the video is stored in a specified file system or database, and a unique identifier is generated for subsequent processing. According to the method, the user can quickly retrieve the key information fragment, and the use efficiency in video review, scientific research and teaching is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of medical information processing technology, and in particular to an AI optimization method for surgical data based on deep learning. Background Art

[0002] With the rapid development of modern medicine, the recording, management and storage of data during surgery have become an important part of medical informationization. Existing surgical data mainly include surgical videos, images, patient medical records, intraoperative monitoring data, etc. These data are not only used for postoperative review and teaching, but also provide important basis for surgical quality assessment, postoperative patient care, and condition analysis.

[0003] At present, the collection and storage of surgical data mainly rely on traditional video recording equipment and storage systems. These systems usually directly store the entire surgical video in the original format. Although the integrity of the data is guaranteed, there are also the following major problems and defects:

[0004] 1. Large data storage space consumption: Surgical video data is usually long and high-resolution. Especially with the popularization of high-quality video equipment such as HD and 4K, the amount of raw video data for a single operation can reach tens of GB. Long-term storage will greatly occupy the hospital's information storage space, increasing data management and storage costs.

[0005] 2. High data redundancy: In an operation, only some video clips involve key operations or important events (such as incisions, organ exposure, surgical steps, etc.), while a large number of clips such as preoperative preparations, instrument debugging and other redundant information occupy a large amount of storage space, and there is a lack of intelligent screening and compression processing of video content.

[0006] 3. Lack of intelligent storage optimization: The current surgical video storage system relies on traditional compression algorithms (such as H.264, etc.), and cannot effectively combine the content characteristics of surgical videos for intelligent processing. Existing technologies fail to use artificial intelligence, deep learning and other technologies to intelligently analyze and optimize the storage of surgical videos, resulting in low storage efficiency and inconvenient retrieval.

[0007] 4. Difficulty in retrieving and managing surgical data: Due to the large amount of surgical video data, the existing system has difficulty in quickly locating key operations or events during postoperative review, research or teaching, resulting in a long time spent on video retrieval and playback, reducing the efficiency of clinical use.

[0008] Therefore, the existing surgical data storage technology has obvious shortcomings when facing the needs of efficiency and intelligence, and there is an urgent need for deep learning-based surgical data AI storage methods to change this situation. Summary of the invention

[0009] The purpose of the present invention is to solve the shortcomings of the prior art and propose an AI storage method for surgical data based on deep learning. Its advantage is that users can quickly retrieve key information fragments, which significantly improves the efficiency of use in video review, scientific research, and teaching.

[0010] In order to achieve the above object, the present invention adopts the following technical solutions:

[0011] The deep learning-based AI optimization method for surgical data includes the following steps:

[0012] Step 1: The video file uploader uploads the surgical video file from an external device or storage medium to the system as the input source for the entire video summary processing flow; the system verifies the format and size of the uploaded video file and ensures that no data is lost or the file is damaged during the upload process; for larger video files, the system supports segmented upload technology. After the file is uploaded, the video is stored in the specified file system or database and a unique identifier is generated for subsequent processing;

[0013] Step 2: The image processing end receives the uploaded video file, performs image preprocessing and frame decomposition operations, and provides a data basis for subsequent key frame extraction and event recognition; image preprocessing is performed using OpenCV or other image processing libraries, using Gaussian filtering for denoising and CLAHE for image contrast enhancement; it supports multi-threaded processing to speed up frame decomposition and image preprocessing;

[0014] Step 3: The key frame extraction end automatically extracts key frames from the processed frame sequence, focusing on retaining frames representing important operations or events and deleting redundant and irrelevant frame fragments; through deep learning models or algorithms based on specific rules, each frame is analyzed to identify key moments in the surgical process, including organ exposure, bleeding, and suturing; the corresponding frames are automatically extracted according to the detected key operation moments and saved as the core content of the video summary;

[0015] Step 4: The result viewing terminal provides a user interface, allowing the user to view the results of key frame extraction, that is, the generated surgical video summary, and supports playback, modification and export operations.

[0016] The present invention is further configured such that the video file upload end allows a user to upload a locally stored surgical video file through a user interface or an API interface, and supports a variety of common video formats, including MP4, AVI and MOV.

[0017] The present invention is further configured such that the image processing end uses the FFmpeg tool to decode and extract video frames, converts the video stream into a frame sequence of static image files frame by frame, and preprocesses each frame of the image, such as denoising, contrast enhancement, color correction, etc., to ensure that subsequent processing algorithms can extract clear and accurate image information.

[0018] The present invention is further configured such that the key frame extraction result can measure the importance of the frame through a scoring mechanism, and finally save several frames with the highest scores.

[0019] The present invention is further configured such that the result viewing end displays the extracted key frame sequence through a front-end interface, and the user previews the surgical video summary fragments, manually adjusts and annotates certain important fragments as needed; and exports the video summary into different formats, including short videos or picture sequences.

[0020] The present invention is further configured such that the result viewing terminal supports exporting and downloading of video summaries, and the export format may be a compressed version of a video file or a sequence of images containing key frames.

[0021] The present invention is further configured such that the key frame extraction algorithm adopts a DenseNet algorithm, wherein the DenseNet is composed of a plurality of dense blocks, and in each dense block, the output of each layer is connected to all subsequent layers; the key frame extraction algorithm extracts local spatial features through a convolutional layer, and dense connections fuse features of different depths.

[0022] The present invention is further configured such that the DenseNet algorithm extracts surgical operation features present in each frame through multiple convolution operations, including the use of instruments, changes in organs, and the presence of blood; the network captures the complex structure and surgical details in the frame through layer-by-layer feature superposition;

[0023] After the convolution operation is completed, a global average pooling layer is used to convert the feature map of each frame into a feature vector of fixed length to characterize the importance of the frame.

[0024] The present invention is further configured to evaluate the DenseNet algorithm model on a test set to verify the performance of the model on unknown data, and the verification indicators include accuracy, precision, recall rate and F1 score.

[0025] The present invention is further configured such that the accuracy calculation formula is:

[0026] Accuracy=(TP+TN) / (TP+FP+FN+TN); the accuracy calculation formula is:

[0027] Precision = TP / (TP+FP); the recall rate calculation formula is: Recall = TP / (TP+FN); the F1 score calculation formula is: F1Score = 2*TP / (2*TP+FP+FN).

[0028] The beneficial effects of the present invention are:

[0029] 1. Significantly reduce storage space usage

[0030] The present invention uses deep learning algorithms to intelligently screen surgical videos and data, automatically identify and save key operations and important event clips, filter out redundant and irrelevant parts, and significantly reduce the storage space requirements for surgical data. Compared with traditional full-process video storage, it can save more than 50% of storage space and effectively reduce storage costs.

[0031] 2. Improve the intelligence level of data storage

[0032] The existing technology relies on traditional compression algorithms for data storage, which does not fully utilize the key information in the surgical scene. The present invention introduces deep learning technology to intelligently identify important links and operation nodes in the surgery, optimize the storage strategy, improve the intelligence and pertinence of data storage, and make the stored content more in line with actual needs.

[0033] 3. Improve the efficiency of surgical video retrieval

[0034] The present invention automatically marks important events and operation points in surgical videos through a deep learning model, providing an efficient key segment location function. Compared with the existing manual search or full playback, users can quickly retrieve key information segments, significantly improving the efficiency of video review, scientific research, and teaching. The retrieval speed of surgical videos can be increased by more than 30%.

[0035] 4. Reduce redundant information in surgical data

[0036] Surgical videos often contain a large number of invalid or repeated operation shots, such as preoperative preparation, waiting time, etc. These redundant information takes up a lot of space in traditional storage systems. The present invention uses a deep learning algorithm to identify these irrelevant segments, optimize the storage content, and ensure that the surgical data is more streamlined and efficient.

[0037] 5. Improve data management and retrieval experience

[0038] Traditional surgical data management and retrieval are inefficient, especially when faced with massive amounts of data, which often requires a lot of time to search manually. This invention greatly optimizes the data management and retrieval experience through automated data tagging and classification functions, and provides a more intuitive and faster access method, which is particularly suitable for postoperative review, academic research, and surgical teaching scenarios.

[0039] In general, the present invention is superior to the existing technology in terms of storage efficiency, intelligent processing, data retrieval and management, and can significantly improve the storage optimization and usage experience of surgical data, bringing higher medical data management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic diagram of the workflow of the deep learning-based AI optimization method for surgical data proposed in the present invention;

[0041] Figure 2 This is a schematic diagram of the structure of the image processing module of the deep learning-based AI optimization method for surgical data proposed in the present invention;

[0042] Figure 3 Schematic diagram of the key frame extraction principle of the deep learning-based AI optimization method for surgical data proposed in the present invention. DETAILED DESCRIPTION

[0043] The technical solution of this patent is further described in detail below in conjunction with specific implementation methods.

[0044] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0045] Reference Figure 1-3 , an AI optimized storage method for surgical data based on deep learning, including video file upload end, image processing end, key frame extraction end and result viewing end;

[0046] The video file uploader is responsible for uploading surgical video files from external devices or storage media to the system as the input source for the entire video summary processing flow. This module allows users to upload locally stored surgical video files through a user interface or API interface, and supports a variety of common video formats (such as MP4, AVI, MOV, etc.). The system verifies the format and file size of the uploaded video files, and ensures that no data is lost or the file is damaged during the upload process. For larger video files, segmented upload technology is supported to improve the stability and efficiency of the upload. After the file is uploaded, the video is stored in the specified file system or database, and a unique identifier (UUID) is generated for subsequent processing.

[0047] The image processing end is responsible for receiving the uploaded video files, performing image preprocessing, frame decomposition and other operations, and providing a data basis for subsequent key frame extraction and event recognition; this module uses the FFmpeg tool to decode and extract video frames, convert the video stream into a frame sequence of static image files frame by frame (each frame represents an image), and preprocess each frame of the image, such as denoising, contrast enhancement, color correction, etc., to ensure that the subsequent processing algorithm can extract clear and accurate image information; image preprocessing is performed using OpenCV or other image processing libraries, such as using Gaussian filtering for denoising and CLAHE (adaptive histogram equalization) for image contrast enhancement; it supports multi-threaded processing to speed up frame decomposition and image preprocessing, especially to improve efficiency when processing long surgical videos;

[0048] The key frame extraction end mainly automatically extracts key frames from the processed frame sequence, focusing on retaining frames representing important operations or events, and deleting redundant and irrelevant frame fragments. Through a deep learning model or an algorithm based on specific rules, each frame is analyzed to identify key moments in the surgical process, such as organ exposure, bleeding, suturing, etc. The system automatically extracts the corresponding frames based on the detected key operation moments and saves them as the core content of the video summary; by combining time series analysis methods, LSTM (long short-term memory network) or GRU (gated recurrent unit), etc., the timing relationship of the video is analyzed to ensure the continuity and coherence of the selected frames; the system supports dynamic adjustment of the frequency and accuracy of frame extraction to meet the needs of different types of surgical videos; the key frame extraction results can measure the importance of the frame through a scoring mechanism, and finally save the highest-scoring frames;

[0049] The result viewing terminal allows users to view the results of key frame extraction, that is, the generated surgical video summary, and supports playback, modification, and export operations. The extracted key frame sequence is displayed through the front-end interface, and users can preview the surgical video summary clips, or manually adjust and annotate certain important clips as needed. Users can also choose to export the video summary into different formats, such as short videos or image sequences, for subsequent clinical analysis, teaching use, etc.; the system supports the export and download of video summaries, and the export format can be a compressed video file or an image sequence containing key frames, which is convenient for users to archive and use.

[0050] The key frame extraction algorithm uses the DenseNet algorithm:

[0051] DenseNet (Dense Convolutional Network) is an improved convolutional neural network, which is characterized by direct connection of each convolutional layer with all previous layers in each layer, avoiding the bottleneck of information transmission between layers in traditional convolutional neural networks, greatly improving the efficiency of feature transmission and parameter utilization of the model. The present invention uses the DenseNet algorithm to extract key frames in the video to reduce redundant frames and retain important information during the operation.

[0052] Implementation of DenseNet:

[0053] Input layer: Video files are converted into frame sequences through the image processing end, and the DenseNet algorithm uses these frames as input. The pixel information of each frame is standardized and adjusted to a suitable resolution (for example, 224x224 pixels) to meet the input requirements of the DenseNet model.

[0054] Convolutional layers and dense connections: DenseNet consists of multiple dense blocks, in which the output of each layer is connected to all subsequent layers. This design enables the network to capture more local features in each layer, thereby improving the model's recognition ability in complex surgical scenarios. The convolutional layer is mainly responsible for extracting local spatial features, while the dense connection fuses features of different depths, improving the accuracy of detecting important surgical steps.

[0055] Feature extraction: DenseNet extracts possible surgical operation features in each frame through multiple convolution operations, including the use of instruments, changes in organs, the presence of blood, etc. The network captures the complex structure and surgical details in the frame by superimposing layers of features.

[0056] Global pooling layer: After the convolution operation is completed, a global average pooling layer is used to convert the feature map of each frame into a feature vector of fixed length to characterize the importance of the frame.

[0057] Classification layer: The feature vector after global pooling is input to the classification layer to determine whether the frame is a key frame. By training DenseNet, the model can identify frames of different steps and important events in the surgical video and score each frame. Frames with high scores will be extracted as key frames for generating video summaries.

[0058] Training of DenseNet model

[0059] Dataset preparation:

[0060] By annotating the key frames of a large number of surgical videos, a multi-category classification dataset was constructed. The dataset contains image frames of various important operations in surgical videos, such as cutting, suturing, and hemostasis. The dataset has been preprocessed, including image size normalization, data enhancement (such as rotation, flipping, and color adjustment).

[0061] Data size:

[0062] Number of surgical videos: 2,000 surgical videos of different types, each of which is 45-60 minutes long on average. Frame sequence: Each video is broken down into 25 frames per second, for a total of about 225 million frames of data. Keyframe annotation: 10,000 keyframes were extracted through manual annotation, which correspond to important moments in the surgery (such as bleeding, organ exposure, instrument cutting, suturing, etc.), and category labels were attached to these keyframes;

[0063] Training process:

[0064] Use these labeled data sets to conduct supervised training on DenseNet. During the training process, the cross entropy loss function is used to optimize the weight parameters of the model, and the gradient is updated through the Adam optimizer. After the training is completed, the model can automatically determine whether it is a key frame in the operation based on the characteristics of the input frame.

[0065] The training process adopts supervised learning method and is mainly divided into three stages: training, verification, and testing.

[0066] Training set: Of the entire dataset, 70% (7000 keyframes) is used to train the DenseNet model.

[0067] Validation set: used to verify the effect of the model, the validation set accounts for 20% (2000 key frames).

[0068] Test set: The remaining 10% (1000 key frames) is used for the final performance evaluation of the model.

[0069] Data augmentation: To increase data diversity and prevent model overfitting, some common augmentation operations such as rotation, translation, scaling, color adjustment, noise addition, etc. are performed on each frame, thereby expanding the size of the dataset and enhancing the generalization ability of the model.

[0070] Model input: Each surgical video frame is standardized to a size of 224x224 pixels and normalized to meet the input requirements of DenseNet.

[0071] Hyperparameter settings:

[0072] Batch size: 32.

[0073] The learning rate is 0.001, and the learning rate decay strategy is used during training, which is reduced by half every 10 epochs.

[0074] Optimizer: Adam optimizer, used to adjust model parameters.

[0075] Loss function: The cross entropy loss function is used, combined with class imbalance weight adjustment, to improve the model's recognition ability for minority classes (such as rare surgical events).

[0076] Training duration: The training lasts for 100 epochs, and each epoch passes through 7,000 frames of training samples.

[0077] Computing resources:

[0078] We use NVIDIA Tesla V100 GPU for accelerated training. A single epoch takes about 30 minutes, and the total training time is about 50 hours.

[0079] Validation set evaluation:

[0080] After each epoch, the model is evaluated on the validation set, and the accuracy, precision, recall, and F1 score of the validation set are recorded. These indicators are used to evaluate the generalization performance of the model and decide whether to adjust hyperparameters or perform early stopping.

[0081] Model tuning:

[0082] By observing the performance of the validation set, the learning rate and regularization parameters of the model can be adjusted in a timely manner, or the generalization ability of the model can be improved through data enhancement strategies. To prevent overfitting, an early stopping strategy is adopted, that is, training is stopped when the validation set loss no longer decreases.

[0083] Test set evaluation:

[0084] The final model is evaluated on the test set to verify the performance of the model on unknown data. We recorded the following metrics:

[0085] Accuracy: According to the calculation formula:

[0086] Accuracy = (TP + TN) / (TP + FP + FN + TN) shows that the overall accuracy of the model on the test set is about 94.5%, that is, the key frames of most surgical steps can be correctly identified.

[0087] Precision: According to the calculation formula:

[0088] Precision = TP / (TP+FP) shows that the accuracy of the key frame category is 92% on average, that is, the model has a high accuracy in identifying a certain type of event during the surgery.

[0089] Recall: According to the calculation formula:

[0090] Recall = TP / (TP+FN) shows that the recall rate is 90%, that is, the model can better capture all key moments from the frame sequence.

[0091] F1 score: According to the calculation formula:

[0092] F1Score=2*TP / (2*TP+FP+FN) yields: the F1 score is 91%.

[0093] Error analysis:

[0094] In the test set, incorrect keyframe extraction is mainly concentrated in the clips where the light of the surgical scene changes dramatically, and in the case where the instrument blocks important surgical actions. In these cases, the feature extraction ability of the DenseNet model is reduced, resulting in misjudgment.

[0095] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. The AI ​​optimization method for surgical data based on deep learning is characterized by: The following steps are involved: Step 1: The video file uploader uploads the surgical video file from an external device or storage medium to the system as the input source for the entire video summary processing flow; the system verifies the format and size of the uploaded video file and ensures that no data is lost or the file is damaged during the upload process; for larger video files, the system supports segmented upload technology. After the file is uploaded, the video is stored in the specified file system or database and a unique identifier is generated for subsequent processing; Step 2: The image processing end receives the uploaded video file, performs image preprocessing and frame decomposition operations, and provides a data basis for subsequent key frame extraction and event recognition; Image preprocessing is performed using OpenCV or other image processing libraries, using Gaussian filtering for denoising and CLAHE for image contrast enhancement; multi-threaded processing is supported to speed up frame decomposition and image preprocessing; Step 3: The key frame extraction end automatically extracts key frames from the processed frame sequence, focusing on retaining frames representing important operations or events and deleting redundant and irrelevant frame fragments; through deep learning models or algorithms based on specific rules, each frame is analyzed to identify key moments in the surgical process, including organ exposure, bleeding, and suturing; the corresponding frames are automatically extracted according to the detected key operation moments and saved as the core content of the video summary; Step 4: The result viewing terminal provides a user interface, allowing the user to view the results of key frame extraction, that is, the generated surgical video summary, and supports playback, modification and export operations.

2. The deep learning-based surgical data AI storage method according to claim 1 is characterized in that: The video file upload end allows the user to upload locally stored surgical video files through a user interface or an API interface, and supports a variety of common video formats, including MP4, AVI and MOV.

3. The deep learning-based surgical data AI optimization method according to claim 1 is characterized in that: The image processing end uses the FFmpeg tool to decode and extract video frames, converts the video stream into a frame sequence of static image files frame by frame, and preprocesses each frame of the image, including denoising, contrast enhancement and color correction.

4. The deep learning-based surgical data AI optimization method according to claim 1 is characterized in that: The key frame extraction results can measure the importance of the frames through a scoring mechanism, and finally save the frames with the highest scores.

5. The deep learning-based surgical data AI optimization method according to claim 4 is characterized in that: The result viewing terminal displays the extracted key frame sequence through the front-end interface, and the user previews the surgical video summary fragments, manually adjusts and annotates certain important fragments as needed; and exports the video summary into different formats, including short videos or picture sequences.

6. The deep learning-based AI optimization method for surgical data according to claim 5 is characterized in that: The result viewing terminal supports the export and download of video summaries, and the export format is a compressed video file or a sequence of images containing key frames.

7. The deep learning-based surgical data AI optimization method according to claim 1 is characterized in that: The key frame extraction algorithm adopts the DenseNet algorithm. DenseNet consists of multiple dense blocks. In each dense block, the output of each layer is connected to all subsequent layers. The key frame extraction algorithm extracts local spatial features through convolutional layers, and dense connections fuse features of different depths.

8. The deep learning-based AI optimization method for surgical data according to claim 7 is characterized in that: The DenseNet algorithm extracts surgical operation features in each frame through multiple convolution operations, including the use of instruments, changes in organs, and the presence of blood. The network captures the complex structure and surgical details in the frame by stacking layers of features. After the convolution operation is completed, a global average pooling layer is used to convert the feature map of each frame into a feature vector of fixed length to characterize the importance of the frame.

9. The deep learning-based AI optimization method for surgical data according to claim 8, characterized in that: The DenseNet algorithm model is evaluated on the test set to verify the performance of the model on unknown data. The verification indicators include accuracy, precision, recall and F1 score.

10. The deep learning-based AI storage method for surgical data according to claim 9, characterized in that: The accuracy calculation formula is: Accuracy=(TP+TN) / (TP+FP+FN+TN); the accuracy calculation formula is: Precision = TP / (TP+FP); the recall rate calculation formula is: Recall = TP / (TP+FN); the F1 score calculation formula is: F1Score = 2*TP / (2*TP+FP+FN).

Images