Abnormity recognition method and system based on robot laparoscope

Abstract

The invention provides an anomaly identification method and system based on a robot laparoscope, relates to the technical field of laparoscope data processing, and aims to identify short-time anomaly. The method comprises the following steps: collecting video data of a working area by a laparoscope; obtaining extraction features of each video frame, identifying changed extraction features in adjacent video frames, and recording the changed extraction features and corresponding feature values and variations; determining whether the extracted features are abnormal or not by comparing feature thresholds and feature values of the extracted features; obtaining the variable quantity of the extracted features in a preset time period, and determining the change rate of the extracted features in the preset time period; determining whether the change of the extracted features is abnormal or not by comparing the change rate threshold value and the change rate of the extracted features in the preset time period; and generating a solution strategy based on the extracted features, the feature values and the change rate of the anomalies. And by paying attention to the change of the features in the adjacent video frames, fine changes in a short time are found in time, and short-time anomaly recognition is realized.

Description

Technical Field

[0001] This invention relates to the field of laparoscopic data processing technology, specifically to an anomaly identification method and system based on robotic laparoscopy. Background Technology

[0002] Laparoscopic procedures typically require the use of specialized instruments. The laparoscope acquires real-time dynamic video images of the abdominal cavity, and image analysis technology identifies key structures and lesion areas to formulate a surgical strategy. The operator then uses this strategy information in real-time to coordinate with the instruments and perform the corresponding procedures. To reduce the risk of postoperative complications, timely detection and management of abnormalities are essential during the analysis and identification of the laparoscopic procedure.

[0003] Currently, the identification of abnormalities in laparoscopic procedures mainly relies on the analysis of static images, which makes it difficult to capture the dynamic correlation between image sequences. This results in insufficient perception of dynamic changes and an inability to effectively detect short-term abnormalities (such as transient tissue ischemia, minor local bleeding, or momentary accidental instrument contact). When short-term abnormalities occur, if measures are not taken in time, the abnormal state may continue to accumulate or worsen, leading to postoperative complications such as infection, adhesion, and bleeding. Summary of the Invention

[0004] The purpose of this invention is to provide an abnormality identification method and system based on robotic laparoscopy, and the technical problem to be solved is how to identify short-term abnormalities.

[0005] This invention is achieved through the following technical solution:

[0006] The first aspect provides an anomaly identification method based on robotic laparoscopy, including the following steps:

[0007] A pre-defined working area is selected, and video data of the working area is acquired using a laparoscope.

[0008] Based on the above video data, extract the features of each video frame, identify the changing extract features in adjacent video frames, and record the changing extract features and their corresponding feature values ​​and changes.

[0009] By comparing the feature threshold and feature value of the extracted feature, it is determined whether the extracted feature is abnormal; if the extracted feature is abnormal, the extracted feature and its corresponding feature value are stored in the abnormal database.

[0010] Obtain the amount of change of the extracted features during a preset time period, and determine the rate of change of the extracted features during that preset time period;

[0011] By comparing the change rate threshold and change rate of the extracted features in the preset time period, it is determined whether the change of the extracted features is abnormal; if the change of the extracted features is abnormal, the extracted features and the corresponding change rate are stored in the abnormal database.

[0012] A solution strategy is generated based on the extracted features, feature values, and rates of change from the aforementioned abnormal database.

[0013] Due to the limited acquisition range of the laparoscopy, complete video acquisition of the working area takes time. During acquisition, the laparoscopy moves gradually, and while new features are added for extraction, some features gradually disappear in subsequent video frames. By monitoring changes in features in adjacent video frames, subtle changes within a short period can be detected promptly. By setting feature thresholds and comparing them with actual feature values, the abnormality of extracted features can be quantitatively determined, achieving static anomaly identification. In addition, the rate of change of extracted features within a preset time period is considered to reflect the dynamic changes of extracted features over a period of time. For some dynamic changes, there are requirements for the changes in extracted features within a short period of time. Even if the analysis of each frame meets the static requirements, there may be situations that do not meet the requirements for dynamic changes. Therefore, by setting a rate of change threshold and comparing it with the actual rate of change, the abnormality of extracted feature changes can be further determined from the perspective of dynamic changes, achieving short-term anomaly identification.

[0014] Furthermore, the above-mentioned process of identifying and extracting features that change in adjacent video frames, and recording the extracted features, their corresponding feature values, and the magnitude of change, includes the following specific steps:

[0015] Acquire video frames at adjacent time points, use the video frame of the previous time point in the adjacent time point video frames as the background image, and the video frame of the next time point as the processed image;

[0016] Based on the background image above, the background area of ​​the processed image is removed to obtain the changed area;

[0017] Extract features and feature values ​​from the aforementioned regions of change;

[0018] After associating the extracted features, corresponding feature values, and video frame timestamps, the associated data is obtained; the associated data is then stored in the record database.

[0019] The feature value of the extracted feature is retrieved from the background image from the aforementioned record database. The difference between the feature values ​​of the extracted feature in the processed image and the background image is determined to obtain the change in the extracted feature.

[0020] This method analyzes the dynamic changes between adjacent video frames based on the video frame rate, which can be automatically adjusted. By treating the previous frame as the background and the next frame as the processed image, background areas are removed, highlighting the parts of the processed image that differ from the background. This reduces interference from irrelevant information, allowing the analysis to focus on the areas that are actually changing, thus improving the efficiency of short-term anomaly detection. Features and feature values ​​are extracted from the changing areas to obtain information related to short-term anomalies. Extracting features from the changing areas avoids the influence of features from irrelevant regions, improving the targeting of feature extraction and aiding in the identification of short-term anomalies.

[0021] Furthermore, if the extracted feature is retrieved from the background image, the extracted feature is marked as a change feature; the difference between the feature values ​​of the change feature in the processed image and the background image is taken as the change amount of the extracted feature.

[0022] If the extracted feature is not retrieved from the background image, the extracted feature is marked as a new feature; the feature value of the new feature is used as the change in the extracted feature.

[0023] Features that can be extracted from the background image are labeled as changed features, clearly indicating that the feature has altered from its original state. The focus is on features that undergo changes in state within a short period. For example, during transient ischemic attacks, the color and texture of tissue may undergo subtle changes. By labeling these as changed features, potentially abnormal features can be located, improving the targeting of short-term anomaly detection. The difference between the feature values ​​of these changed features in the processed image and the background image is used as the change quantity to quantify the degree of feature change. For short-term anomalies, the severity and development trend of the anomaly are quantified. For instance, in cases of minor local bleeding, the color feature values ​​of the bleeding area will change. By calculating the change quantity, it can be determined whether the bleeding is worsening, providing a basis for timely intervention.

[0024] Marking features that cannot be retrieved from the background image as new features allows for the timely detection of newly emerging features (i.e., features that appear when the laparoscopy moves to different areas). During laparoscopic procedures, instruments may create new features at contact points, such as new indentations. By marking these as new features, unexpected situations can be quickly detected, preventing more serious consequences due to delayed detection. New features themselves represent change information; in short-term anomaly identification, new features may foreshadow the occurrence of anomalies. Treating the feature values ​​of new features as variables of change helps in the timely detection of potential short-term anomalies.

[0025] Furthermore, the changes in the extracted features over a preset time period are obtained, and the rate of change of the extracted features during that preset time period is determined. Specific steps include:

[0026] Obtain the change in the extracted features of each timestamp within the aforementioned preset time period;

[0027] By using the change in the extracted features and the feature value of the extracted features in the background image, the rate of change of the extracted features at each time point is determined.

[0028] The cumulative rate of change of the extracted features at each timestamp within the preset time period is used to obtain the sum of the rate of change of the extracted features within the preset time period.

[0029] Obtain the total number of timestamps for the aforementioned preset time period, and determine the rate of change of the extracted features within that preset time period by using the ratio of the total number of timestamps to the sum of the change rates.

[0030] The preset time period can be set according to actual needs, such as a few seconds or tens of seconds. It acquires the changes in extracted features at each timestamp, recording the changes in features at different times within a short period. For short-term anomalies, this recording helps capture the dynamic process of the anomaly's occurrence; for example, during transient tissue ischemia, the changes in features at different timestamps can reflect the dynamic changes in the degree of ischemia.

[0031] The rate of change reflects the speed of feature change. It is calculated by comparing the change with the feature value of the extracted feature in the background image, measuring the intensity of feature change at each time point. For minor local bleeding, the feature value changes with the bleeding situation; the calculated rate of change can intuitively reflect the speed of bleeding, helping to determine the severity and trend of the abnormality. The sum of rates of change comprehensively considers the feature's changes at various time points within a preset time period, calculating the average rate of change over the preset time period, reflecting the overall degree of feature change throughout the entire preset time period. For example, for momentary accidental instrument contact, although the contact time is short, by accumulating the rate of change at each time point, the impact of the entire accidental contact process on the feature can be quantified, helping to assess the impact of the abnormality on the entire operation process.

[0032] Furthermore, the steps for determining the rate of change of the extracted features at each time stamp include:

[0033] Obtain the timestamp markers for the extracted features;

[0034] When the extracted features are marked as change features at the timestamp, the change amount of the extracted features at that timestamp and the feature value of the corresponding background image are obtained to determine the change rate of the extracted features at that timestamp.

[0035] When the extracted features are marked as newly added features at the timestamp, the rate of change of the extracted features at that timestamp is 1.

[0036] By acquiring markers, the type of extracted features at corresponding time points is clearly identified, i.e., whether they are changing features or newly added features. For changing features, the rate of change is calculated by acquiring the amount of change at the current timestamp and the corresponding feature value of the background image, quantifying the intensity of the feature change at that time point. This reflects the dynamic change process of features over time in real time, and by calculating the rate of change at different time points, it is understood whether the feature change is gradually intensifying or tending to stabilize. For example, during transient tissue ischemia, the ratio of the change in tissue color features to the feature value in the background image can reflect the change in the degree of ischemia at that time point, helping to determine the severity of the abnormality.

[0037] In laparoscopic procedures, a new feature is a significant dynamic change. Setting it to 1 can reflect the intensity of the change to a certain extent and help to detect potential abnormalities in a timely manner. Examples include new indentations or reflections caused by accidental instrument touch, new ranges of the laparoscope moving into the working area, and new features acquired.

[0038] Furthermore, based on the aforementioned background image, the background region of the processed image is removed to obtain the changed region. Specific steps include:

[0039] The processed image is compared pixel by pixel with the background image to determine the difference value of each pixel between the processed image and the background image;

[0040] A preset difference threshold is set, and pixels with a difference value greater than the difference threshold are marked as foreground pixels, while pixels with a difference value less than or equal to the difference threshold are marked as background pixels.

[0041] Based on the foreground and background pixels mentioned above, a binarized foreground mask image is generated;

[0042] Perform a bitwise AND operation between the foreground mask image and the processed image to obtain the changed region.

[0043] By comparing the processed image with the background image pixel by pixel and calculating the difference between the two, it is possible to distinguish between static background and dynamic foreground, which helps to locate changing areas and speeds up image recognition.

[0044] A difference threshold is set to determine whether a pixel belongs to the foreground or background. Based on the division of foreground and background pixels, a binary (black and white) foreground mask image is generated, where foreground pixels are typically represented as white (value 1) and background pixels as black (value 0). This foreground mask image clearly distinguishes the foreground region of interest (i.e., the change region) from the background region that does not need to be considered, simplifying the complexity of subsequent processing. Using the foreground mask image as a "filter," a bitwise AND operation is performed with the processed image to retain only the parts considered to be foreground, thus obtaining the final change region. This process effectively removes background interference, ensuring that the change region contains only the foreground of interest, improving the efficiency and accuracy of subsequent feature extraction and recognition steps.

[0045] The second aspect provides an anomaly recognition system based on robotic laparoscopy, which employs the aforementioned anomaly recognition method;

[0046] The anomaly detection system includes:

[0047] Laparoscope, used to acquire video data of the working area;

[0048] The control center, connected to the laparoscope, is used to preset the working area and execute the following steps:

[0049] Based on video data, extract features from each video frame, identify the changing extract features in adjacent video frames, and record the changing extract features, as well as the corresponding feature values ​​and changes.

[0050] By comparing the feature threshold and feature value of the extracted features, it is determined whether the extracted feature is abnormal;

[0051] Obtain the amount of change of the extracted features in the preset time period, and determine the rate of change of the extracted features in the preset time period; by comparing the rate of change threshold and the rate of change of the extracted features in the preset time period, determine whether the change of the extracted features is abnormal;

[0052] Based on the above-mentioned anomalies, extract features, feature values, and rates of change to generate a solution strategy;

[0053] An anomaly database is connected to the control center; the anomaly database is used to store the extracted features, feature values, and change rates of anomalies and anomaly changes.

[0054] Furthermore, the anomaly identification system also includes a record database, which stores the associated extracted features, corresponding feature values, and associated data after video frame timestamps.

[0055] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0056] Due to the limited acquisition range of the laparoscopy, complete video acquisition of the working area takes time. During acquisition, the laparoscopy moves gradually, and while new features are added for extraction, some features gradually disappear in subsequent video frames. By monitoring changes in features in adjacent video frames, subtle changes within a short period can be detected promptly. By setting feature thresholds and comparing them with actual feature values, the abnormality of extracted features can be quantitatively determined, achieving static anomaly identification. In addition, the rate of change of extracted features within a preset time period is considered to reflect the dynamic changes of extracted features over a period of time. For some dynamic changes, there are requirements for the changes in extracted features within a short period of time. Even if the analysis of each frame meets the static requirements, there may be situations that do not meet the requirements for dynamic changes. Therefore, by setting a rate of change threshold and comparing it with the actual rate of change, the abnormality of extracted feature changes can be further determined from the perspective of dynamic changes, achieving short-term anomaly identification. Attached Figure Description

[0057] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0058] Figure 1 This is a flowchart of the anomaly detection method. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this invention are only for explaining this invention and are not intended to limit this invention.

[0060] First embodiment:

[0061] Combination Figure 1 An anomaly identification method based on robotic laparoscopy includes the following steps:

[0062] A pre-defined working area is selected, and video data of the working area is acquired using a laparoscope.

[0063] Based on the above video data, extract the features of each video frame, identify the changing extract features in adjacent video frames, and record the changing extract features and their corresponding feature values ​​and changes.

[0064] By comparing the feature threshold and feature value of the extracted feature, it is determined whether the extracted feature is abnormal; if the extracted feature is abnormal, the extracted feature and its corresponding feature value are stored in the abnormal database.

[0065] Obtain the amount of change of the extracted features during a preset time period, and determine the rate of change of the extracted features during that preset time period;

[0066] By comparing the change rate threshold and change rate of the extracted features in the preset time period, it is determined whether the change of the extracted features is abnormal; if the change of the extracted features is abnormal, the extracted features and the corresponding change rate are stored in the abnormal database.

[0067] A solution strategy is generated based on the extracted features, feature values, and rates of change from the aforementioned abnormal database.

[0068] Due to the limited acquisition range of the laparoscopy, complete video acquisition of the working area takes time. During acquisition, the laparoscopy moves gradually, and while new features are added for extraction, some features gradually disappear in subsequent video frames. By monitoring changes in features in adjacent video frames, subtle changes within a short period can be detected promptly. By setting feature thresholds and comparing them with actual feature values, the abnormality of extracted features can be quantitatively determined, achieving static anomaly identification. In addition, the rate of change of extracted features within a preset time period is considered to reflect the dynamic changes of extracted features over a period of time. For some dynamic changes, there are requirements for the changes in extracted features within a short period of time. Even if the analysis of each frame meets the static requirements, there may be situations that do not meet the requirements for dynamic changes. Therefore, by setting a rate of change threshold and comparing it with the actual rate of change, the abnormality of extracted feature changes can be further determined from the perspective of dynamic changes, achieving short-term anomaly identification.

[0069] Based on the abnormal data identified and stored in the abnormal database, corresponding solutions are generated to take timely measures against short-term abnormalities, thereby preventing the abnormal state from accumulating or worsening and reducing the risk of postoperative complications. This solves the problem that existing technologies cannot effectively handle short-term abnormalities.

[0070] One possible use case is in a urological experiment, where carbon dioxide is injected into the abdominal cavity of a target (which may be a human model) through a pneumoperitoneum needle to inflate the target's abdomen. A laparoscope and instruments are then inserted into the abdominal cavity, and video data of the target's abdominal cavity is acquired by the laparoscope. Features and feature values ​​in adjacent video frames are extracted, and based on the extracted features and feature values, it is determined whether the features and feature changes are abnormal, thus achieving short-term anomaly identification.

[0071] Second embodiment:

[0072] Based on the first embodiment, the above-mentioned steps for identifying changes in adjacent video frames, recording the changed features and their corresponding feature values ​​and changes include:

[0073] Acquire video frames at adjacent time points, use the video frame of the previous time point in the adjacent time point video frames as the background image, and the video frame of the next time point as the processed image;

[0074] Based on the background image above, the background area of ​​the processed image is removed to obtain the changed area;

[0075] Extract features and feature values ​​from the aforementioned regions of change;

[0076] After associating the extracted features, corresponding feature values, and video frame timestamps, the associated data is obtained; the associated data is then stored in the record database.

[0077] The feature value of the extracted feature is retrieved from the background image from the aforementioned record database. The difference between the feature values ​​of the extracted feature in the processed image and the background image is determined to obtain the change in the extracted feature.

[0078] This method analyzes the dynamic changes between adjacent video frames based on the video frame rate, which can be automatically adjusted. By treating the previous frame as the background and the next frame as the processed image, background areas are removed, highlighting the parts of the processed image that differ from the background. This reduces interference from irrelevant information, allowing the analysis to focus on the areas that are actually changing, thus improving the efficiency of short-term anomaly detection. Features and feature values ​​are extracted from the changing areas to obtain information related to short-term anomalies. Extracting features from the changing areas avoids the influence of features from irrelevant regions, improving the targeting of feature extraction and aiding in the identification of short-term anomalies.

[0079] In a specific embodiment, if the extracted feature is retrieved from the background image, the extracted feature is marked as a changed feature; the difference between the feature values ​​of the changed feature in the processed image and the background image is taken as the change amount of the extracted feature, and the formula is: ,in, Indicates feature extraction The change This indicates the extracted feature in the processed image. eigenvalues, This indicates the extracted feature in the background image. eigenvalues;

[0080] If the extracted feature is not retrieved from the background image, then the extracted feature is marked as a new feature; the feature value of the new feature is used as the change in the extracted feature. .

[0081] Features that can be extracted from the background image are labeled as changed features, clearly indicating that the feature has altered from its original state. The focus is on features that undergo changes in state within a short period. For example, during transient ischemic attacks, the color and texture of tissue may undergo subtle changes. By labeling these as changed features, potentially abnormal features can be located, improving the targeting of short-term anomaly detection. The difference between the feature values ​​of these changed features in the processed image and the background image is used as the change quantity to quantify the degree of feature change. For short-term anomalies, the severity and development trend of the anomaly are quantified. For instance, in cases of minor local bleeding, the color feature values ​​of the bleeding area will change. By calculating the change quantity, it can be determined whether the bleeding is worsening, providing a basis for timely intervention.

[0082] Marking features that cannot be retrieved from the background image as new features allows for the timely detection of newly emerging features (i.e., features that appear when the laparoscopy moves to different areas). During laparoscopic procedures, instruments may create new features at contact points, such as new indentations. By marking these as new features, unexpected situations can be quickly detected, preventing more serious consequences due to delayed detection. New features themselves represent change information; in short-term anomaly identification, new features may foreshadow the occurrence of anomalies. Treating the feature values ​​of new features as variables of change helps in the timely detection of potential short-term anomalies.

[0083] Third embodiment:

[0084] Based on the second embodiment, the change amount of the extracted features over a preset time period is obtained, and the change rate of the extracted features during that preset time period is determined. Specific steps include:

[0085] Obtain the change in the extracted features of each timestamp within the aforementioned preset time period;

[0086] By using the change in the extracted features and the feature value of the extracted features in the background image, the rate of change of the extracted features at each time point is determined.

[0087] The cumulative rate of change of the extracted features at each timestamp within the preset time period is used to obtain the sum of the rate of change of the extracted features within the preset time period.

[0088] Obtain the total number of timestamps for the aforementioned preset time period. Determine the rate of change of the extracted features within that preset time period by the ratio of the total number of timestamps to the sum of the change rates, as shown in the following formula: ,in, Indicates feature extraction The rate of change during the preset time period Indicates the total number of timestamps. Indicates feature extraction In timestamp The rate of change.

[0089] The preset time period can be set according to actual needs, such as a few seconds or tens of seconds. It acquires the changes in extracted features at each timestamp, recording the changes in features at different times within a short period. For short-term anomalies, this recording helps capture the dynamic process of the anomaly's occurrence; for example, during transient tissue ischemia, the changes in features at different timestamps can reflect the dynamic changes in the degree of ischemia.

[0090] The rate of change reflects the speed of feature change. It is calculated by comparing the change with the feature value of the extracted feature in the background image, measuring the intensity of feature change at each time point. For minor local bleeding, the feature value changes with the bleeding situation; the calculated rate of change can intuitively reflect the speed of bleeding, helping to determine the severity and trend of the abnormality. The sum of rates of change comprehensively considers the feature's changes at various time points within a preset time period, calculating the average rate of change over the preset time period, reflecting the overall degree of feature change throughout the entire preset time period. For example, for momentary accidental instrument contact, although the contact time is short, by accumulating the rate of change at each time point, the impact of the entire accidental contact process on the feature can be quantified, helping to assess the impact of the abnormality on the entire operation process.

[0091] In a specific embodiment, the step of determining the rate of change of the extracted features at each time stamp includes:

[0092] Obtain the timestamp markers for the extracted features;

[0093] When the extracted features are marked as change features at the timestamp, the change amount of the extracted features at that timestamp and the corresponding feature value of the background image are obtained to determine the rate of change of the extracted features at that timestamp, as shown in the following formula:

[0094] ,in, Indicates feature extraction In timestamp The change This indicates the extracted feature in the background image. eigenvalues;

[0095] When the extracted features are marked as newly added features at the timestamp, the rate of change of the extracted features at that timestamp is 1.

[0096] By acquiring markers, the type of extracted features at corresponding time points is clearly identified, i.e., whether they are changing features or newly added features. For changing features, the rate of change is calculated by acquiring the amount of change at the current timestamp and the corresponding feature value of the background image, quantifying the intensity of the feature change at that time point. This reflects the dynamic change process of features over time in real time, and by calculating the rate of change at different time points, it is understood whether the feature change is gradually intensifying or tending to stabilize. For example, during transient tissue ischemia, the ratio of the change in tissue color features to the feature value in the background image can reflect the change in the degree of ischemia at that time point, helping to determine the severity of the abnormality.

[0097] In laparoscopic procedures, a new feature is a significant dynamic change. Setting it to 1 can reflect the intensity of the change to a certain extent and help to detect potential abnormalities in a timely manner. Examples include new indentations or reflections caused by accidental instrument touch, new ranges of the laparoscope moving into the working area, and new features acquired.

[0098] Fourth embodiment:

[0099] Based on the second embodiment, and based on the aforementioned background image, the background region of the processed image is removed to obtain the changed region. Specific steps include:

[0100] The processed image is compared pixel by pixel with the background image to determine the difference value of each pixel between the processed image and the background image;

[0101] A preset difference threshold is set, and pixels with a difference value greater than the difference threshold are marked as foreground pixels, while pixels with a difference value less than or equal to the difference threshold are marked as background pixels.

[0102] Based on the foreground and background pixels mentioned above, a binarized foreground mask image is generated;

[0103] Perform a bitwise AND operation between the foreground mask image and the processed image to obtain the changed region.

[0104] By comparing the processed image with the background image pixel by pixel and calculating the difference between the two, it is possible to distinguish between static background and dynamic foreground, which helps to locate changing areas and speeds up image recognition.

[0105] A difference threshold is set to determine whether a pixel belongs to the foreground or background. Based on the division of foreground and background pixels, a binary (black and white) foreground mask image is generated, where foreground pixels are typically represented as white (value 1) and background pixels as black (value 0). This foreground mask image clearly distinguishes the foreground region of interest (i.e., the change region) from the background region that does not need to be considered, simplifying the complexity of subsequent processing. Using the foreground mask image as a "filter," a bitwise AND operation is performed with the processed image to retain only the parts considered to be foreground, thus obtaining the final change region. This process effectively removes background interference, ensuring that the change region contains only the foreground of interest, improving the efficiency and accuracy of subsequent feature extraction and recognition steps.

[0106] Fifth embodiment:

[0107] An anomaly recognition system based on robotic laparoscopy, which employs the aforementioned anomaly recognition method;

[0108] The anomaly detection system includes:

[0109] Laparoscope, used to acquire video data of the working area;

[0110] The control center, connected to the laparoscope, is used to preset the working area and execute the following steps:

[0111] Based on video data, extract features from each video frame, identify the changing extract features in adjacent video frames, and record the changing extract features, as well as the corresponding feature values ​​and changes.

[0112] By comparing the feature threshold and feature value of the extracted features, it is determined whether the extracted feature is abnormal;

[0113] Obtain the amount of change of the extracted features in the preset time period, and determine the rate of change of the extracted features in the preset time period; by comparing the rate of change threshold and the rate of change of the extracted features in the preset time period, determine whether the change of the extracted features is abnormal;

[0114] Based on the above-mentioned anomalies, extract features, feature values, and rates of change to generate a solution strategy;

[0115] An anomaly database is connected to the control center; the anomaly database is used to store the extracted features, feature values, and change rates of anomalies and anomaly changes.

[0116] In a specific embodiment, the anomaly recognition system also includes a record database, which stores associated extracted features, corresponding feature values, and associated data after video frame timestamps.

[0117] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An anomaly identification method based on robotic laparoscopy, characterized in that, Includes the following steps: A pre-defined working area is selected, and video data of the working area is acquired using a laparoscope. Based on the video data, extract features of each video frame are obtained, and the changing extract features in adjacent video frames are identified. The changing extract features, their corresponding feature values, and the amount of change are recorded. By comparing the feature threshold and feature value of the extracted feature, it is determined whether the extracted feature is abnormal; if the extracted feature is abnormal, the extracted feature and its corresponding feature value are stored in the abnormal database. Obtain the amount of change of the extracted features during a preset time period, and determine the rate of change of the extracted features during that preset time period; By comparing the rate of change threshold and the rate of change of the extracted feature in the preset time period, it is determined whether the change of the extracted feature is abnormal; if the change of the extracted feature is abnormal, the extracted feature and the corresponding rate of change are stored in the abnormal database. A solution strategy is generated based on the extracted features, feature values, and rates of change from the aforementioned anomaly database.

2. The anomaly identification method according to claim 1, characterized in that, The steps for identifying and extracting features that change in adjacent video frames, and recording the extracted features, their corresponding feature values, and the magnitude of change, include: Acquire video frames at adjacent time points, use the video frame of the previous time point in the adjacent time point video frames as the background image, and the video frame of the next time point as the processed image; Based on the background image, the background area of ​​the processed image is removed to obtain the changed area; Extract features and feature values ​​from the changed region; After associating the extracted features, corresponding feature values, and video frame timestamps, associated data is obtained; the associated data is then stored in a record database. The feature value of the extracted feature is retrieved from the background image from the recorded database, and the difference between the feature values ​​of the extracted feature in the processed image and the background image is determined to obtain the change in the extracted feature.

3. The anomaly identification method according to claim 2, characterized in that, If the extracted feature is retrieved from the background image, the extracted feature is marked as a changed feature; the difference between the feature values ​​of the changed feature in the processed image and the background image is taken as the change amount of the extracted feature; If the extracted feature is not retrieved from the background image, the extracted feature is marked as a new feature; the feature value of the new feature is used as the change in the extracted feature.

4. The anomaly identification method according to claim 1, characterized in that, The steps for obtaining the change amount of the extracted features during a preset time period and determining the change rate of the extracted features during that preset time period include: Obtain the change in the extracted features of each timestamp within the preset time period; The rate of change of the extracted feature at each time point is determined by the amount of change of the extracted feature and the feature value of the extracted feature in the background image. The cumulative rate of change of the extracted features at each timestamp within the preset time period is used to obtain the sum of the rate of change of the extracted features within the preset time period. Obtain the total number of timestamps for the preset time period, and determine the rate of change of the extracted features during the preset time period by the ratio of the total number of timestamps to the sum of the change rates.

5. The anomaly identification method according to claim 1, characterized in that, The steps to determine the rate of change of the extracted features at each time stamp include: Obtain the timestamp marker of the extracted features; When the extracted feature is marked as a change feature at the timestamp, the change amount of the extracted feature at that timestamp and the feature value of the corresponding background image are obtained to determine the change rate of the extracted feature at that timestamp. When the extracted feature is marked as a newly added feature at the timestamp, the change rate of the extracted feature at that timestamp is 1.

6. The anomaly identification method according to claim 2, characterized in that, Based on the background image, the background region of the processed image is removed to obtain the changed region. The specific steps include: The processed image is compared pixel by pixel with the background image to determine the difference value of each pixel between the processed image and the background image; A preset difference threshold is set, and pixels with a difference value greater than the difference threshold are marked as foreground pixels, while pixels with a difference value less than or equal to the difference threshold are marked as background pixels. A binarized foreground mask image is generated based on the foreground pixels and background pixels; The foreground mask image and the processed image are subjected to a bitwise AND operation to obtain the changed region.

7. An anomaly recognition system based on robotic laparoscopy, characterized in that, The anomaly identification system employs the anomaly identification method described in any one of claims 1 to 6; The anomaly detection system includes: Laparoscope, used to acquire video data of the working area; The control center, connected to the laparoscope, is used to preset the working area and execute the following steps: Based on video data, extract features from each video frame, identify the changing extract features in adjacent video frames, and record the changing extract features, as well as the corresponding feature values ​​and changes. By comparing the feature threshold and feature value of the extracted feature, it is determined whether the extracted feature is abnormal; The change amount of the extracted features within a preset time period is obtained, and the change rate of the extracted features within that preset time period is determined. By comparing the change rate threshold and the change rate of the extracted features within that preset time period, it is determined whether the change of the extracted features is abnormal. Based on the extracted features, feature values, and rate of change of the aforementioned anomalies, a solution strategy is generated. An anomaly database is connected to the control center; the anomaly database is used to store the extracted features, feature values, and change rates of anomalies and anomaly changes.

8. The anomaly identification system according to claim 7, characterized in that, The anomaly detection system also includes a record database, which stores associated extracted features, corresponding feature values, and associated data after video frame timestamps.

Images