Preoperative frame automatic calling system based on artificial intelligence

Abstract

The invention discloses an automatic pre-operation frame calling system based on artificial intelligence, which relates to the technical field of medical instruments and comprises an image acquisition module, an image processing module, a bleeding detection module, a frame calling logic module and a user interface module, the image acquisition module is used for acquiring operation image data of a patient in real time in the endoscopic operation process, real-time image signals in an operation scene are acquired and transmitted through a high-definition camera or related medical imaging equipment, and the image acquisition module is connected with the image processing module. A clear operation view can be obtained without manual operation of a doctor, the burden of the doctor is reduced, the operation process is simplified, and the operation accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and in particular to an automatic pre-operative frame retrieval system based on artificial intelligence. Background Art

[0002] During surgery, doctors need to continuously monitor the situation in the surgical area, especially when bleeding occurs, doctors need to take prompt measures; however, bleeding during surgery often blocks the doctor's vision and affects the accuracy of the surgical operation. In complex surgical environments, doctors need to deal with bleeding problems quickly and accurately.

[0003] Current technology mainly relies on the doctor's experience or manually rewinding the video to check the source of bleeding, and lacks an intelligent automatic feedback mechanism, which affects the cost and safety of the operation. Therefore, an artificial intelligence-based automatic pre-operative frame retrieval system is urgently needed to change this situation. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose an artificial intelligence-based automatic pre-operative frame retrieval system, which has the advantage of improving the accuracy of the operation.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] An artificial intelligence-based pre-operative frame automatic retrieval system, comprising an image acquisition module, an image processing module, a bleeding detection module, a frame retrieval logic module and a user interface module; the image acquisition module is used to collect the patient's surgical image data in real time during the laparoscopic surgery, and collect and transmit the real-time image signal in the surgical scene through a high-definition camera or related medical imaging equipment, and the image acquisition module is connected to the image processing module;

[0007] The image processing module is used to pre-process the surgical image data transmitted by the image acquisition module; the processing methods include image denoising, enhancement, edge detection and image segmentation; the bleeding detection module uses a time series analysis method to continuously monitor and analyze surgical video frames; the module analyzes the change rules between frames and identifies potential bleeding points by comparing the color information, brightness distribution and texture features of pixel values ​​in consecutive frames;

[0008] The frame retrieval logic module is used to control the timing of video playback. When a bleeding point is detected, it automatically rewinds to the picture before the bleeding, and resumes real-time playback after the bleeding is processed; the user interface module integrates a real-time video playback function. When the bleeding detection module detects a bleeding point, the interface will automatically pop up a prompt message to remind the user that the bleeding has been detected, and at the same time display the regressed picture in real time.

[0009] The present invention is further configured such that the bleeding detection module first segments the video frame, then extracts the color histogram, brightness distribution and texture features in the frame, and identifies the rapidly changing area by calculating the similarity between frames; once the rapidly changing area is detected, it is marked as a bleeding point; after detecting the bleeding point, the module will mark the position of the bleeding area and send a signal to trigger the frame call logic module.

[0010] The present invention is further configured such that the frame retrieval logic module receives a signal from a bleeding detection module, and when a bleeding point is detected, the current real-time playback is paused, and the video is rewound to the frame before the bleeding according to the time series record; the playback order of each frame of the video is managed by using a timestamp, and when bleeding is detected, the corresponding historical frame is retrieved to play back the key picture before the bleeding; when the bleeding point is processed, the frame retrieval logic module will resume the real-time picture playback and clear the frame buffer at the same time.

[0011] The present invention is further configured such that the user interface module provides a manual adjustment function, allowing medical staff to review the detected bleeding points through the interface and replay videos of specific time periods as needed; display the marked positions of the bleeding areas and highlight the bleeding points; and provide a progress bar to mark the switching between the real-time screen and the historical playback screen.

[0012] The present invention is further configured such that the image processing module performs calculations through an image processing algorithm, the image processing algorithm includes image denoising, and the steps of the image denoising are: firstly perform a two-dimensional transformation on the two-dimensional blocks in each three-dimensional matrix and then perform a one-dimensional transformation in the third dimension of the matrix, usually a Hadamard Transform; after the transformation, the components less than the hyperparameter λ3D are set to 0 by a hard threshold method; at the same time, the number of non-zero components is counted as a reference for subsequent weights; finally, the denoised image block is obtained by a one-dimensional inverse transformation and a two-dimensional inverse transformation in the third dimension;

[0013] The image denoising calculation formula is: Q(P)=T3Dhard-1(γ(T3Dhard(Q(P))).

[0014] The present invention is further configured such that the image processing algorithm also includes contrast enhancement, which is to add some information or transform data to the original image by certain means, selectively highlight the features of interest in the image or suppress certain unnecessary features in the image, so that the image matches the visual response characteristics.

[0015] The present invention is further configured such that the image contrast calculation formula is: Among them, δ(i,j)=|ij|, and the grayscale difference between adjacent pixels; P δ(i, j) is the grayscale difference between adjacent pixels, and δ is the distribution probability of the pixel.

[0016] The present invention is further configured that the frame retrieval module uses a time series analysis algorithm for calculation, and the time series analysis algorithm includes the following steps:

[0017] Step 1: During the operation, each frame of video image will be transmitted to the system through the image acquisition terminal; the system will extract features from each frame of image, obtain image features such as color, brightness, texture, etc., and construct time series data;

[0018] Step 2: Arrange the extracted features in chronological order to form a multidimensional time series; use the ARIMA detection algorithm to model and analyze the time series; identify whether there are abnormal changes by comparing the differences or trends between the current frame and the previous frames in the time series; if the change of the abnormal value exceeds the threshold, the module will mark the point as a possible bleeding point;

[0019] Step 3: When a bleeding point is detected, the system automatically triggers an alarm and pauses the real-time playback of the video, entering the frame retracing mode; the frame retrieval logic module will automatically rewind to the frame before the bleeding based on the bleeding detection results for the doctor to handle; when the doctor confirms that the bleeding has stopped or the treatment has been completed, the system automatically resumes the real-time playback of the video.

[0020] The beneficial effects of the present invention are:

[0021] 1. Enhance surgical safety: Existing surgical video systems cannot provide an immediate, blood-free surgical field of view when bleeding occurs. However, the present invention can effectively avoid surgical operation risks caused by limited field of view by intelligently detecting bleeding points and automatically calling back to the key frame before bleeding. This function helps doctors obtain a clearer surgical field of view before bleeding treatment, thereby greatly improving the safety of surgery.

[0022] 2. Improve surgical accuracy: When bleeding is detected during surgery, the present invention immediately returns to the clear frame before bleeding, ensuring that the doctor has the clearest view during the operation. This instant picture adjustment helps the doctor to operate more accurately, reduces possible misoperation caused by blood blocking the line of sight, and improves the accuracy of the operation.

[0023] 3. Improve surgical efficiency: The existing technology usually requires doctors to pause the operation when bleeding occurs and manually replay the video to find the bloodless picture. The present invention can automatically detect and switch to the bloodless key frame without the doctor's manual intervention, thereby reducing the operation time and improving the operation efficiency. The automated management during the operation reduces the time wasted on operating the video system, allowing doctors to focus more on the operation itself.

[0024] 4. Reduce the burden on doctors: Traditional surgical video systems require doctors to manually control video playback to obtain a blood-free field of view when dealing with bleeding, which not only increases the workload of doctors, but also may affect the continuity of the surgical process. The present invention uses an automated intelligent system to obtain a clear surgical field of view without manual operation by doctors, reducing the burden on doctors and simplifying the surgical operation process.

[0025] 5. Real-time response and automated operation: The existing technology lacks real-time bleeding detection and automatic playback functions during surgery. The present invention introduces artificial intelligence technology to automatically process bleeding at the moment it occurs and automatically resume video playback after the blood is cleared. This real-time response mechanism ensures the continuity and smoothness of the surgical process and significantly improves the intelligence level of the surgical video system.

[0026] 6. Wide application: The system of the present invention can be seamlessly integrated with the existing surgical video management system and is applicable to various types of surgical operations. It can be used not only in routine operations, but also in complex and delicate surgical procedures. Therefore, the present invention has wide applicability and promotion value.

[0027] 7. Reduce emergency response time during surgery: The present invention automatically detects and handles bleeding, allowing doctors to respond to sudden bleeding more quickly, reducing emergency response time during surgery. This rapid response mechanism helps improve the collaboration efficiency and response speed of the entire surgical team, further improving the success rate and safety of the surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a schematic diagram of the overall structure of an artificial intelligence-based pre-operative frame automatic retrieval system proposed by the present invention;

[0029] Figure 2 This is a schematic diagram of an image processing module of an artificial intelligence-based pre-operative frame automatic retrieval system proposed by the present invention;

[0030] Figure 3 This is a schematic diagram of time series analysis of an artificial intelligence-based pre-operative frame automatic retrieval system proposed by the present invention;

[0031] Figure 4 The present invention is a schematic diagram of the frame retrieval logic of an artificial intelligence-based pre-operative frame automatic retrieval system proposed by the present invention. DETAILED DESCRIPTION

[0032] The technical solution of this patent is further described in detail below in conjunction with specific implementation methods.

[0033] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0034] Reference Figure 1-4 , an artificial intelligence-based pre-operative frame automatic retrieval system, comprising an image acquisition module, an image processing module, a bleeding detection module, a frame retrieval logic module and a user interface module; the image acquisition module is used to collect the patient's surgical image data in real time during the laparoscopic surgery, and collect and transmit the real-time image signal in the surgical scene through a high-definition camera or related medical imaging equipment, and the image acquisition module is connected to the image processing module;

[0035] Ensure that the collected image data can be transmitted to subsequent modules in a stable and continuous form for analysis and processing; the acquisition end uses high-resolution camera equipment with high frame rate, low latency and automatic focus functions to ensure image clarity and real-time performance, while supporting multi-angle image acquisition to enhance the surgical field of view.

[0036] The image processing module is used to pre-process the surgical image data transmitted by the image acquisition module; the processing methods include image denoising, enhancement, edge detection and image segmentation; through the set image enhancement algorithm and filter, the image noise is eliminated, the image contrast and clarity are improved, and the optimized image data is provided to the bleeding detection end. Improve the extraction accuracy of image features;

[0037] The bleeding detection module uses a time series analysis method to continuously monitor and analyze surgical video frames; the module analyzes the change patterns between frames and identifies potential bleeding points by comparing the color information, brightness distribution and texture features of pixel values ​​in consecutive frames; first, the video frames are segmented, and then the color histogram, brightness distribution and texture features in the frames are extracted, and the rapidly changing areas are identified by calculating the similarity between frames; once the rapidly changing areas are detected, they are marked as bleeding points; after detecting the bleeding points, the module will mark the location of the bleeding area and send a signal to trigger the frame retrieval logic module.

[0038] The frame retrieval logic module is used to control the timing of video playback. When a bleeding point is detected, it automatically rewinds to the picture before the bleeding, and resumes real-time playback after the bleeding is processed. It receives the signal sent by the bleeding detection module, pauses the current real-time playback when a bleeding point is detected, and rewinds the video to the frame before the bleeding according to the time series record. It uses the timestamp to manage the playback order of each frame of the video, and retrieves the corresponding historical frame when bleeding is detected to play back the key picture before the bleeding. When the bleeding point is processed, the frame retrieval logic module will resume real-time picture playback and clear the frame buffer to ensure smooth playback. This module is closely integrated with the bleeding detection module to ensure that important pictures can be reviewed during the operation and avoid omissions.

[0039] The user interface module integrates the real-time video playback function. When the bleeding detection module detects a bleeding point, the interface will automatically pop up a prompt message to remind the user that bleeding has been detected, and display the rewind image in real time. It provides a manual adjustment function, allowing medical staff to review the detected bleeding points through the interface and replay the video of a specific time period as needed. It displays the marked position of the bleeding area and highlights the bleeding point. It also provides a progress bar to mark the switching between the real-time image and the historical playback image. The user interface module is tightly integrated with the bleeding detection module and the frame retrieval logic module to ensure the real-time display of the progress of bleeding detection and the status information of the frame rewind.

[0040] The image processing module performs calculations through an image processing algorithm, which includes image denoising. The steps of image denoising are as follows: first perform a two-dimensional transformation on the two-dimensional blocks in each three-dimensional matrix and then perform a one-dimensional transformation in the third dimension of the matrix, usually a Hadamard Transform; after the transformation, the components less than the hyperparameter λ3D are set to 0 using a hard threshold; at the same time, the number of non-zero components is counted as a reference for subsequent weights; finally, the denoised image block is obtained through a one-dimensional inverse transformation and a two-dimensional inverse transformation in the third dimension; the image denoising calculation formula is: Q(P)=T3Dhard-1(γ(T3Dhard(Q(P))).

[0041] Image processing algorithms also include contrast enhancement, which is to add some information or transform data to the original image by certain means, selectively highlight the features of interest in the image or suppress some unnecessary features in the image, so that the image matches the visual response characteristics; the image contrast calculation formula is: Among them, δ(i,j)=|ij|, and the grayscale difference between adjacent pixels; P δ (i, j) is the grayscale difference between adjacent pixels, and δ is the distribution probability of the pixel.

[0042] Time series analysis is a statistical method for processing and analyzing data based on time sequence. Its core idea is to predict future trends or detect anomalies by analyzing the temporal evolution of data. For the task of bleeding detection in surgical videos, time series analysis is mainly used to process continuous image frame data to identify significant changes (bleeding phenomena) that occur in the frame sequence.

[0043] Processing of input data

[0044] Input data: Continuous image frames in surgical videos are used as input data in time series. Each frame of the image is converted into a data point in the time series.

[0045] Feature extraction: In order to convert each frame of image into a data point suitable for time series analysis, relevant features need to be extracted from each frame of image. These features include but are not limited to:

[0046] Color information (RGB channel value or HSV color space value): used to monitor the color change of the bleeding area. Usually, the bleeding area will have obvious red changes.

[0047] Brightness information: Changes in the brightness of the image can reflect changes in blood or blood volume.

[0048] Texture features: Use methods such as LBP (local binary pattern) and Gabor filter to extract texture features of the image to capture subtle changes in the image.

[0049] Time Series Modeling

[0050] After the image features are extracted, they are arranged in chronological order to form a multidimensional time series. The following methods are then used to model and analyze the series:

[0051] Autoregressive Integrated Moving Average Model

[0052] Principle: ARIMA model is a classic method in time series forecasting, which combines autoregression (AR), difference (I) and moving average (MA). The autoregression part is used to model the relationship between data and its previous value, the moving average part is used to smooth noise, and the difference part is used to deal with non-stationary series.

[0053] Application: ARIMA can be used to predict the color and brightness values ​​of the next frame of the image. If the difference between the actual image value and the predicted value exceeds the set threshold, it can be considered that an abnormality has occurred, that is, the detection of bleeding points.

[0054] The frame acquisition module uses a time series analysis algorithm to calculate, and the time series analysis algorithm includes the following steps:

[0055] Step 1: During the operation, each frame of video image will be transmitted to the system through the image acquisition terminal; the system will extract features from each frame of image, obtain image features such as color, brightness, texture, etc., and construct time series data;

[0056] Step 2: Arrange the extracted features in chronological order to form a multidimensional time series; use the ARIMA detection algorithm to model and analyze the time series; identify whether there are abnormal changes by comparing the differences or trends between the current frame and the previous frames in the time series; if the change of the abnormal value exceeds the threshold, the module will mark the point as a possible bleeding point;

[0057] Step 3: When a bleeding point is detected, the system automatically triggers an alarm and pauses the real-time playback of the video, entering the frame retracing mode; the frame retrieval logic module will automatically rewind to the frame before the bleeding based on the bleeding detection results for the doctor to handle; when the doctor confirms that the bleeding has stopped or the treatment has been completed, the system automatically resumes the real-time playback of the video.

[0058] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An artificial intelligence-based automatic pre-operative frame retrieval system, characterized in that: It includes an image acquisition module, an image processing module, a bleeding detection module, a frame acquisition logic module and a user interface module; the image acquisition module is used to collect the patient's surgical image data in real time during the laparoscopic surgery, and collect and transmit the real-time image signal in the surgical scene through a high-definition camera or related medical imaging equipment. The image acquisition module is connected to the image processing module; The image processing module is used to pre-process the surgical image data transmitted by the image acquisition module; the processing methods include image denoising, enhancement, edge detection and image segmentation; the bleeding detection module uses a time series analysis method to continuously monitor and analyze surgical video frames; the module analyzes the change rules between frames and identifies potential bleeding points by comparing the color information, brightness distribution and texture features of pixel values ​​in consecutive frames; The frame retrieval logic module is used to control the timing of video playback. When a bleeding point is detected, it automatically rewinds to the picture before the bleeding, and resumes real-time playback after the bleeding is processed; the user interface module integrates a real-time video playback function. When the bleeding detection module detects a bleeding point, the interface will automatically pop up a prompt message to remind the user that the bleeding has been detected, and at the same time display the regressed picture in real time.

2. The artificial intelligence-based automatic pre-operative frame retrieval system according to claim 1, characterized in that: The bleeding detection module first segments the video frame, then extracts the color histogram, brightness distribution and texture features in the frame, and identifies the rapidly changing area by calculating the similarity between frames; once the rapidly changing area is detected, it is marked as a bleeding point; after detecting the bleeding point, the module will mark the position of the bleeding area and send a signal to trigger the frame call logic module.

3. The artificial intelligence-based automatic pre-operative frame retrieval system according to claim 2, characterized in that: The frame retrieval logic module receives the signal sent by the bleeding detection module, and when a bleeding point is detected, the current real-time playback is paused, and the video is rewound to the frame before the bleeding according to the time series record; The timestamp is used to manage the playback order of each frame of video, and when bleeding is detected, the corresponding historical frame is retrieved to play back the key pictures before the bleeding; when the bleeding point is processed, the frame retrieval logic module will resume the real-time picture playback and clear the frame buffer at the same time.

4. The artificial intelligence-based automatic pre-operative frame retrieval system according to claim 3, characterized in that: The user interface module provides a manual adjustment function, allowing medical staff to review the detected bleeding points through the interface and replay videos of specific time periods as needed; display the marked positions of the bleeding areas and highlight the bleeding points; and provide a progress bar to mark the switching between real-time images and historical playback images.

5. The artificial intelligence-based automatic pre-operative frame retrieval system according to claim 1, characterized in that: The image processing module performs calculations through an image processing algorithm, and the image processing algorithm includes image denoising. The steps of image denoising are: first perform a two-dimensional transformation on the two-dimensional blocks in each three-dimensional matrix and then perform a one-dimensional transformation in the third dimension of the matrix, usually a Hadamard Transform; after the transformation, the components less than the hyperparameter λ3D are set to 0 by using a hard threshold; at the same time, the number of non-zero components is counted as a reference for subsequent weights; finally, the denoised image block is obtained by a one-dimensional inverse transformation and a two-dimensional inverse transformation in the third dimension; The image denoising calculation formula is: Q(P)=T3Dhard-1(γ(T3Dhard(Q(P))).

6. The artificial intelligence-based automatic pre-operative frame retrieval system according to claim 5, characterized in that: The image processing algorithm also includes contrast enhancement, which is to add some information or transform data to the original image by certain means, selectively highlight the features of interest in the image or suppress some unnecessary features in the image, so that the image matches the visual response characteristics.

7. The artificial intelligence-based automatic pre-operative frame retrieval system according to claim 6, characterized in that: The image contrast calculation formula is: Among them, δ(i,j)=|ij|, and the grayscale difference between adjacent pixels; P δ (i, j) is the grayscale difference between adjacent pixels, and δ is the distribution probability of the pixel.

8. The artificial intelligence-based automatic pre-operative frame retrieval system according to claim 7, characterized in that: The frame retrieval module uses a time series analysis algorithm for calculation, and the time series analysis algorithm includes the following steps: Step 1: During the operation, each frame of video image will be transmitted to the system through the image acquisition terminal; the system will extract features from each frame of image, obtain image features such as color, brightness, texture, etc., and construct time series data; Step 2: Arrange the extracted features in chronological order to form a multidimensional time series; use the ARIMA detection algorithm to model and analyze the time series; identify whether there are abnormal changes by comparing the differences or trends between the current frame and the previous frames in the time series; if the change of the abnormal value exceeds the threshold, the module will mark the point as a possible bleeding point; Step 3: When a bleeding point is detected, the system automatically triggers an alarm and pauses the real-time playback of the video, entering the frame retracing mode; the frame retrieval logic module will automatically rewind to the frame before the bleeding based on the bleeding detection results for the doctor to handle; when the doctor confirms that the bleeding has stopped or the treatment has been completed, the system automatically resumes the real-time playback of the video.

Images