Endoscope video processing system

Through the acquisition, identification and correction module of the endoscopic video processing system, the target video clips and status identification in the endoscopic video are extracted and corrected, which solves the problem of indistinguishability in mild UC and CD, and achieves the accuracy of endoscopic video reporting Sex improvement.

CN119092057APending Publication Date: 2024-12-06CHANGZHOU UNITED IMAGING HEALTHCARE SURGICAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310665678.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-06
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the existing endoscopic video technology, it is difficult to distinguish UC and CD accurately through manual means in mild symptoms, resulting in inaccurate endoscopic video reports.

Method used

Design an endoscopic video processing system, including acquisition module, identification module, correction module and reporting module, to generate an accurate endoscopic video report by extracting target video clips from endoscopic videos, identifying and correcting status marks.

Benefits of technology

It improves the accuracy of endoscopic video reports, can accurately distinguish between UC and CD, reduces artificial errors, and improves the diagnostic accuracy of intestinal conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119092057A_ABST
    Figure CN119092057A_ABST
Patent Text Reader

Abstract

The invention relates to an endoscope video processing system. The system comprises an acquisition module, an identification module, a correction module and a report module. The acquisition module is used for determining at least one target video clip from an endoscope video of a target object; the recognition module is used for recognizing target condition identifiers of the target objects corresponding to the target video clips, and obtaining a condition identifier list of the target objects according to the target condition identifiers; the correction module is used for correcting the status identifier list to obtain a corrected status identifier list; and the reporting module is used for determining an endoscope video report of the target object according to the corrected condition identification list. By adopting the system, the accuracy of the obtained endoscope video report is relatively high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to an endoscopic video processing system. Background Art

[0002] IBD (Inflammatory Bowel Disease) is an idiopathic intestinal inflammatory disease, mainly including UC (Ulcerative Colitis) and CD (Crohn's Disease).

[0003] In the existing technology, a colonoscope can be used to photograph the intestines, and then manually determine whether there is IBD in the intestines, and when there is IBD, the type of IBD. Ideally, the endoscopic manifestations of UC are shallow ulcers, diffuse mucosal congestion, edema, granularity, increased brittleness, etc., and the endoscopic manifestations of CD are longitudinal ulcers, mucosal warm stone changes, normal mucosa between lesions, etc. However, in actual clinical applications, when UC and CD manifest as mild symptoms, the endoscopic manifestations of the two are similar and are relatively close to the characteristics of normal cases. It is difficult to accurately determine the intestinal condition manually, resulting in inaccurate endoscopic video reports.

[0004] Therefore, current endoscopic video technology suffers from the problem of inaccurate endoscopic video reporting. Summary of the invention

[0005] Based on this, it is necessary to provide an endoscopic video processing system, method, computer device, computer-readable storage medium and computer program product that can improve the accuracy of endoscopic video reporting in response to the above-mentioned technical problems.

[0006] In a first aspect, the present application provides an endoscopic video processing system. The system includes an acquisition module, an identification module, a correction module and a reporting module;

[0007] The acquisition module is used to determine at least one target video segment from the endoscopic video of the target object;

[0008] The identification module is used to identify the target status identifier of the target object corresponding to each of the target video clips, and obtain a status identifier list of the target object according to each of the target status identifiers;

[0009] The correction module is used to correct the status identification list to obtain a corrected status identification list;

[0010] The reporting module is used to determine the endoscopic video report of the target object according to the corrected status identification list.

[0011] In one of the embodiments, the acquisition module is further used to determine at least one original video segment from the endoscopic video, and pre-process each of the original video segments to obtain the at least one target video segment.

[0012] In one of the embodiments, the acquisition module is further used to determine a border area of ​​the original video segment, and remove the border area from the original video segment to obtain the target video segment.

[0013] In one of the embodiments, the acquisition module is further used to extract spatial features and temporal features of the original video clip to obtain a video feature vector of the original video clip; perform feature classification on the video feature vector to obtain a first classification result of the original video clip; when the first classification result meets a preset result, the original video clip is determined as the target video clip; the preset result includes that the proportion of non-information video frames in the original video clip does not exceed a preset threshold.

[0014] In one of the embodiments, the acquisition module is further used to input the original video clip into a pre-trained video classification model to obtain a second classification result corresponding to the original video clip, and when the second classification result meets a preset result, the original video clip is determined as the target video clip; the preset result includes that the proportion of non-information video frames in the original video clip does not exceed a preset threshold.

[0015] In one of the embodiments, the identification module is also used to identify the acquisition position of the current target video segment and the initial state identifier corresponding to the current target video segment; when the acquisition position matches the preset position and the initial state identifier matches the preset identifier, the target state identifier corresponding to the current target video segment is obtained according to the initial state identifier.

[0016] In one of the embodiments, the identification module is further used to set the target status identifier corresponding to the current target video segment to a null value when the acquisition position does not match the preset position, or the initial status identifier does not match the preset identifier; use the next target video segment as the current target video segment, and return to the step of identifying the acquisition position of the current target video segment and the initial status identifier corresponding to the current target video segment.

[0017] In one of the embodiments, the correction module is also used to segment the target status identifiers in the status identifier list according to the acquisition position of the target video segment to obtain a status identifier subset; determine the identification type of each target status identifier in the status identifier subset, and determine the identification type in the status identifier subset whose proportion exceeds a preset threshold as the target type; and correct the status identifier subset according to the target type.

[0018] In one of the embodiments, the endoscopic video report includes the effective video frame ratio of the endoscopic video; the reporting module is further used to determine the effective video frame ratio as the ratio of the total duration of the at least one target video segment to the total duration of the endoscopic video.

[0019] In one of the embodiments, the endoscopic video report includes the abnormality level of the target object; the reporting module is further used to determine the abnormal video segments in the endoscopic video based on the corrected condition identification list, and obtain the abnormality level of the target object based on the ratio of the total duration of the abnormal video segments to the total duration of the endoscopic video.

[0020] In a second aspect, the present application provides an endoscopic video processing method. The method comprises:

[0021] determining at least one target video segment from the endoscopic video of the target object;

[0022] Identifying the target status identifier of the target object corresponding to each of the target video clips, and obtaining a status identifier list of the target object according to each of the target status identifiers;

[0023] Correcting the status identification list to obtain a corrected status identification list;

[0024] An endoscopic video report of the target object is determined based on the corrected status identification list.

[0025] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0026] determining at least one target video segment from the endoscopic video of the target object;

[0027] Identifying the target status identifier of the target object corresponding to each of the target video clips, and obtaining a status identifier list of the target object according to each of the target status identifiers;

[0028] Correcting the status identification list to obtain a corrected status identification list;

[0029] An endoscopic video report of the target object is determined based on the corrected status identification list.

[0030] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0031] determining at least one target video segment from the endoscopic video of the target object;

[0032] Identifying the target status identifier of the target object corresponding to each of the target video clips, and obtaining a status identifier list of the target object according to each of the target status identifiers;

[0033] Correcting the status identification list to obtain a corrected status identification list;

[0034] An endoscopic video report of the target object is determined based on the corrected status identification list.

[0035] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0036] determining at least one target video segment from the endoscopic video of the target object;

[0037] Identifying the target status identifier of the target object corresponding to each of the target video clips, and obtaining a status identifier list of the target object according to each of the target status identifiers;

[0038] Correcting the status identification list to obtain a corrected status identification list;

[0039] An endoscopic video report of the target object is determined based on the corrected status identification list.

[0040] The above-mentioned endoscopic video processing system, method, computer device, storage medium and computer program product determine at least one target video segment from the endoscopic video of the target object, identify the target condition identifier of the target object corresponding to each target video segment, and obtain a condition identifier list of the target object based on each target condition identifier, correct the condition identifier list to obtain a corrected condition identifier list, and determine an endoscopic video report of the target object based on the corrected condition identifier list; multiple video segments in the endoscopic video can be identified separately to obtain the condition identifier corresponding to each video segment, wherein each video segment corresponds to a small section of the intestine of the target object, and the condition identifier is used to characterize the condition of this section of the intestine. By correcting the condition identifier list and removing inaccurate condition identifiers, the intestinal condition can be accurately determined based on the remaining relatively accurate condition identifiers, and the obtained endoscopic video report of the target object has high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a structural block diagram of an endoscope video processing system in one embodiment;

[0042] Figure 2 is a schematic diagram of a non-information video frame in one embodiment;

[0043] Figure 3 is a schematic flow chart of an endoscopic video processing method in one embodiment;

[0044] Figure 4 is a flow chart of an endoscopic video processing method in another embodiment;

[0045] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] In one embodiment, Figure 1 As shown, an endoscopic video processing system is provided, including an acquisition module 110, an identification module 120, a correction module 130 and a reporting module 140;

[0048] An acquisition module 110, configured to determine at least one target video segment from an endoscopic video of a target object;

[0049] The identification module 120 is used to identify the target status identifier of the target object corresponding to each target video clip, and obtain a status identifier list of the target object according to each target status identifier;

[0050] A correction module 130, used for performing correction processing on the status identification list to obtain a corrected status identification list;

[0051] The reporting module 140 is used to determine an endoscopic video report of the target object according to the modified condition identification list.

[0052] The endoscopic video processing system may be, but is not limited to, a terminal or server having a video signal processing function.

[0053] Among them, the acquisition module, the identification module, the correction module and the reporting module can be electronic circuit function modules with corresponding video signal processing functions.

[0054] The target object may be the object targeted by the endoscopic video report, for example, a patient.

[0055] The target video segment may be a plurality of consecutive frames from the endoscopic video.

[0056] The target status identifier may be an identifier that represents the status of the target object.

[0057] The status identifier list may be a vector composed of multiple status identifiers.

[0058] The endoscopic video report may be text information generated based on the endoscopic video and used to describe the condition of the target object.

[0059] In a specific implementation, the acquisition module can extract at least one target video segment from the endoscopic video according to a preset time interval during the process of collecting the endoscopic video of the target object, and input the extracted target video segments into the recognition module in sequence. The recognition module recognizes each target video segment to obtain a target status identifier corresponding to each target video segment. The recognition module also composes a status identifier list of the target status identifiers corresponding to all the target video segments, and inputs the list into the correction module. The correction module corrects the status identifier list, and inputs the corrected status identifier list into the reporting module. The reporting module extracts information from the corrected status identifier list to generate an endoscopic video report of the target object.

[0060] For example, a camera is installed at the front end of the endoscope to collect intestinal videos of the target object during the endoscopic examination. The acquisition module divides the collected intestinal videos into multiple video clips. Assuming that the duration of each video clip is 3 seconds, after preprocessing, n target video clips v1, v2, v3, v4, v5, v6, ..., v with a duration of 3 seconds are obtained. nThe recognition module recognizes each target video clip and obtains the target condition identification 0, 0, 1, 1, -1, 1, ..., 0 corresponding to each target video clip, where 0 indicates a normal intestine, 1 indicates CD, and -1 indicates UC. The target condition identifications are arranged in sequence to form a condition identification list [0, 0, 1, 1, -1, 1, ..., 0]. The correction module corrects the condition identification list by correcting -1 to 1 to obtain a corrected condition identification list [0, 0, 1, 1, 1, 1, ..., 0]. The reporting module determines that the target object suffers from CD based on the corrected condition identification list [0, 0, 1, 1, 1, 1, ..., 0] and displays this result in the endoscopic video report.

[0061] The above-mentioned endoscopic video processing system determines at least one target video segment from the endoscopic video of the target object, identifies the target condition identifier of the target object corresponding to each target video segment, and obtains a condition identifier list of the target object based on each target condition identifier, corrects the condition identifier list to obtain a corrected condition identifier list, and determines an endoscopic video report of the target object based on the corrected condition identifier list; multiple video segments in the endoscopic video can be identified separately to obtain a condition identifier corresponding to each video segment, wherein each video segment corresponds to a small section of the intestine of the target object, and the condition identifier is used to characterize the condition of this section of the intestine. By correcting the condition identifier list and removing inaccurate condition identifiers, the intestinal condition can be accurately determined based on the remaining relatively accurate condition identifiers, and the obtained endoscopic video report of the target object has high accuracy.

[0062] In one embodiment, the acquisition module is further used to determine at least one original video segment from the endoscopic video, and pre-process each original video segment to obtain at least one target video segment.

[0063] The original video clip may be a video clip directly extracted from the endoscopic video.

[0064] The preprocessing includes but is not limited to border cropping and elimination of non-information video segments.

[0065] In a specific implementation, the acquisition module can extract at least one original video segment from the endoscopic video according to a preset video segment length, and preprocess each original video segment, including but not limited to border cropping and non-information video segment removal, to finally obtain at least one target video segment.

[0066] For example, the acquisition module divides the intestinal video collected during the endoscopy into m original video segments, each of which has a duration of 3 seconds, and obtains original video segments v1′, v2′, v3′, v4′, v5′, v6′, …, vm ′, remove the black borders in each original video segment, and obtain the video segments v1", v2", v3", v4", v5", v6", ..., v m ", then determine whether there are many non-information video frames such as blur, artifacts, liquid reflections, etc. in the video clip after the border is removed, and remove the video clip after the border is removed with many non-information video frames to obtain n (n≤m) target video clips v1, v2, v3, v4, v5, v6, ..., v n .

[0067] In this embodiment, by determining at least one original video segment from the endoscopic video and preprocessing each original video segment to obtain at least one target video segment, information irrelevant to condition identification can be removed from the original video segment, thereby reducing the amount of data that needs to be processed by subsequent identification modules and improving the processing speed of the endoscopic video.

[0068] In one embodiment, the acquisition module is further used to determine a border area of ​​the original video segment, and remove the border area from the original video segment to obtain the target video segment.

[0069] In a specific implementation, the acquisition module may determine the border region in each original video segment, remove the edge region from the original video segment, and obtain the target video segment.

[0070] For example, the black border of each video frame in the original video segment v1′ records information such as the shooting time, hospital name, and patient age. This information is irrelevant to the identification of the target condition marker. Therefore, the black border area can be removed to increase the calculation speed and reduce the calculation memory. After deletion, the target video segment v1 is obtained.

[0071] In this embodiment, by determining the border area of ​​the original video clip and removing the border area from the original video clip to obtain the target video clip, the amount of data that needs to be processed by the subsequent recognition module can be reduced, thereby improving the processing speed of the endoscopic video.

[0072] In one embodiment, the above-mentioned acquisition module is also used to extract spatial features and temporal features of the original video clip to obtain a video feature vector of the original video clip; perform feature classification on the video feature vector to obtain a first classification result of the original video clip; when the first classification result meets the preset result, the original video clip is determined as the target video clip; the preset result includes that the proportion of non-information video frames in the original video clip does not exceed a preset threshold.

[0073] The spatial features may be information obtained from video frames, such as intestinal color, texture and other features.

[0074] The temporal feature may be information obtained from a plurality of consecutive video frames, for example, change information of a certain voxel of a target part of a human body in a plurality of consecutive video frames.

[0075] The first classification result may be a classification result of the original video segment obtained based on the video feature vector.

[0076] The non-information video frame may be a video frame that displays invalid information, for example, a video frame that contains blur, artifacts, or liquid reflection.

[0077] In a specific implementation, spatial features and temporal features can be extracted from the original video clip to obtain a video feature vector of the original video clip, and the video feature vector is input into a classification model to obtain a first classification result of the original video clip. If the first classification result indicates that the proportion of non-information video frames in the original video clip does not exceed a preset threshold, the original video clip is determined to be a target video clip. Otherwise, if the first classification result indicates that the proportion of non-information video frames in the original video clip does not exceed the preset threshold, the original video clip is deleted, and the first classification result of the next original video clip is used to determine whether the next original video clip is the target video clip.

[0078] For example, a video feature vector is obtained based on the spatial information (intestinal color, texture) provided by each video frame in the original video clip v1′ and the temporal information (change information of a voxel in the previous and next frames) provided by multiple consecutive video frames in the original video clip v1′, and a support vector machine is used to classify the video feature vector. If the classification result meets the preset result, that is, the proportion of non-information video frames such as blur, artifacts, liquid reflections, etc. contained in v1′ in v1′ does not exceed the preset threshold, then v1′ is determined as the target video clip v1; otherwise, if the classification result does not meet the preset result, that is, the proportion of non-information video frames such as blur, artifacts, liquid reflections, etc. contained in v1′ in v1′ exceeds the preset threshold, then v1′ is deleted, and it is continued to be determined whether the next original video clip v2′ can be used as the target video clip.

[0079] Figure 2 A schematic diagram of a non-informative video frame is provided. Figure 2 , count the proportion of non-information video frames in the original video clip, if it does not exceed 40%, then the original video clip is determined to meet the preset result, and the original video clip is used as the target video clip; otherwise, if it exceeds 40%, then the original video clip is determined to not meet the preset result, and the original video clip is discarded. Among them, non-information video frames include but are not limited to video frames outside the patient's body, endoscope examination, highlight or liquid reflection, abnormal contrast, motion blur, out-of-focus blur, etc.

[0080] In this embodiment, feature extraction is performed on spatial features and temporal features of the original video clip to obtain a video feature vector of the original video clip; feature classification is performed on the video feature vector to obtain a first classification result of the original video clip; when the first classification result meets the preset result, the original video clip is determined as the target video clip, and the original video clip with a relatively large proportion of non-information video frames can be removed, thereby ensuring the amount of information in the target video clip and improving the accuracy of identifying the target video clip.

[0081] In one embodiment, the above-mentioned acquisition module is also used to input the original video clip into a pre-trained video classification model to obtain a second classification result corresponding to the original video clip; when the second classification result meets the preset result, the original video clip is determined as the target video clip; the preset result includes that the proportion of non-information video frames in the original video clip does not exceed the preset threshold.

[0082] The second classification result may be a classification result of the original video clip obtained by a video classification model.

[0083] In a specific implementation, a video classification model can be pre-trained, and the original video clip can be input into the video classification model to obtain a second classification result of the original video clip. If the second classification result indicates that the proportion of non-information video frames in the original video clip does not exceed a preset threshold, the original video clip is determined to be a target video clip. Otherwise, if the second classification result indicates that the proportion of non-information video frames in the original video clip does not exceed the preset threshold, the original video clip is deleted, and the second classification result of the next original video clip is used to determine whether the next original video clip is the target video clip.

[0084] For example, a deep learning-based video classification model is pre-trained to classify the original video clip v1′. If the classification result meets the preset result, that is, the proportion of non-information video frames such as blur, artifacts, liquid reflections, etc. contained in the original video clip v1′ in v1′ does not exceed the preset threshold, then v1′ is determined as the target video clip v1; otherwise, if the classification result does not meet the preset result, that is, the proportion of non-information video frames such as blur, artifacts, liquid reflections, etc. contained in the original video clip v1′ in v1′ exceeds the preset threshold, then v1′ is deleted, and it continues to be determined whether the next original video clip v2′ can be used as the target video clip.

[0085] In this embodiment, the original video clip is input into a pre-trained video classification model to obtain a second classification result corresponding to the original video clip. When the second classification result meets the preset result, the original video clip is determined as the target video clip. The original video clip in which non-information video frames account for a relatively large proportion can be removed, thereby ensuring the amount of information in the target video clip and improving the accuracy of identifying the target video clip.

[0086] In one embodiment, the above-mentioned identification module is also used to identify the acquisition position of the current target video segment and the initial state identifier corresponding to the current target video segment; when the acquisition position matches the preset position and the initial state identifier matches the preset identifier, the target state identifier corresponding to the current target video segment is obtained according to the initial state identifier.

[0087] The acquisition position may be the position of the target object corresponding to the current target video segment.

[0088] The initial status identifier may be an initially identified status identifier.

[0089] In a specific implementation, the recognition module can respectively identify the acquisition position and initial state identifier of the current target video clip, and compare the acquisition position with the preset position, and compare the initial state identifier with the preset identifier. If the acquisition position matches the preset position, and the initial state identifier matches the preset identifier, the initial state identifier can be used as the target state identifier corresponding to the current target video clip.

[0090] For example, during the endoscopic diagnosis of IBD, the terminal ileum is examined through the ileocecal opening and an endoscopic video is taken. The affected area of ​​CD is mostly the terminal ileum. Therefore, if the acquisition location of the video clip is identified as the terminal ileum, and the condition in the video clip is identified as CD, the video clip can be marked with CD. Furthermore, the patient's condition can be directly determined as CD.

[0091] In this embodiment, by identifying the acquisition position of the current target video clip and the initial status identifier corresponding to the current target video clip; when the acquisition position matches the preset position and the initial status identifier matches the preset identifier, the target status identifier corresponding to the current target video clip is obtained according to the initial status identifier. The status of the target object can be determined through the acquisition position and status identification, thereby improving the accuracy of the endoscopic video report.

[0092] In one embodiment, the above-mentioned identification module is also used to set the target status identifier corresponding to the current target video segment to a null value when the acquisition position does not match the preset position, or the initial status identifier does not match the preset identifier; take the next target video segment as the current target video segment, and return to the step of identifying the acquisition position of the current target video segment and the initial status identifier corresponding to the current target video segment.

[0093] In a specific implementation, the recognition module recognizes the acquisition position and initial status identifier of the current target video segment respectively. If the acquisition position does not match the preset position, or the initial status identifier does not match the preset identifier, it means that the initial status identifier of the current target video segment is inaccurate. The target status identifier corresponding to the current target video segment is set to a null value, and the recognition of the next target video segment is continued. Specifically, the next target video segment can be used as the current target video segment, and the step of identifying the acquisition position of the current target video segment and the initial status identifier corresponding to the current target video segment is returned to identify the acquisition position and initial status identifier of the next target video segment, and the target status identifier corresponding to the next target video segment is determined according to the comparison result between the acquisition position and the preset position, and the comparison result between the initial status identifier and the preset identifier.

[0094] For example, during the endoscopic diagnosis of IBD, the terminal ileum is entered through the ileocecal opening for inspection and endoscopic video is taken. The affected part of CD is mostly at the terminal ileum. Therefore, if it is recognized that the acquisition position of the current video segment is not the terminal ileum, or it is recognized that the condition in the current video segment is not CD, the recognition result of the current video segment can be ignored, and the recognition is continued for the next video segment to obtain the condition identification of the next video segment. Specifically, if it is recognized that the acquisition position of the current video segment is the terminal ileum and the condition identification is UC or normal, the recognition result of the current video segment is ignored, and the patient's condition is determined based on the IBD recognition results of other intestinal segments; if it is recognized that the acquisition position of the current video segment is not the terminal ileum, the patient's condition is determined based on the IBD recognition results of other intestinal segments.

[0095] In this embodiment, when the acquisition position does not match the preset position, or the initial status identifier does not match the preset identifier, the target status identifier corresponding to the current target video segment is set to a null value; the next target video segment is used as the current target video segment, and the step of returning to the acquisition position of the current target video segment and the initial status identifier corresponding to the current target video segment is performed. The status of the target object can be determined through acquisition position and status identification, thereby improving the accuracy of the endoscopic video report.

[0096] In one embodiment, the correction module is also used to segment the target status identifiers in the status identifier list according to the acquisition position of the target video clip to obtain a status identifier subset; determine the identifier type of each target status identifier in the status identifier subset, determine the identifier type in the status identifier subset that accounts for more than a preset threshold as the target type, and based on the target type; correct the status identifier subset.

[0097] In a specific implementation, the correction module can determine the segment length according to the acquisition position of the target video clip, and divide the target status identifier in the status identifier list into multiple small segments according to the determined segment length, each segment corresponds to a status identifier subset, and the status identifier subset contains multiple target status identifiers. For each status identifier subset, the identification type of each target status identifier in the status identifier subset is determined, and the identification type whose proportion exceeds a preset threshold is determined as the target type, and the identification types in the status identifier subset that are different from the target type are all corrected to the target type.

[0098] For example, the status identification list output by the recognition module is [0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,-1,1,1,1,-1,1,1,1,0,1,1,1,-1,1,1,1,1,1,1,0,0,0,0,0,0], where 0 represents normal, 1 represents CD, and -1 represents UC. Each status identification in the status identification list corresponds to a video clip, and the segment length t is determined according to the acquisition position of the video clip. For example, during endoscopic examination, the endoscope passes through the rectum first and then through the colon. Since the length of the rectum is usually shorter than that of the colon, the segment length of the rectum can be set to 9 and the segment length of the colon can be set to 15. The condition identification subset of the intestine is R1 = [0,0,0,0,0,0,0,0,0], and the condition identification subset of the colon is R2 = [1,1,1,1,1,1,1,-1,1,1,1,-1,1,1,1,1]. The preset threshold is 80%. The identification type with a proportion of more than 80% in R1 is 0, and 0 is determined as the target identification of R1. Since the identification types in R1 are all target identifications, no correction is required; the identification type with a proportion of more than 80% in R2 is 1, and 1 is determined as the target identification of R2. The identification type -1 in R2 is corrected to 1, and the corrected condition identification subset of the colon is R2 = [1,1,1,1,1,1,1,1,1,1,1,1,1,1,1].

[0099] In this embodiment, the target status identifiers in the status identifier list are segmented according to the acquisition position of the target video clip to obtain a status identifier subset; the identifier type of each target status identifier in the status identifier subset is determined, and the identifier type whose proportion in the status identifier subset exceeds a preset threshold is determined as the target type, and according to the target type; the status identifier subset is corrected, which can reduce the false positive of the recognition result of the recognition module, increase the accuracy of the status identifier list, and thereby improve the accuracy of the endoscopic video report.

[0100] In one embodiment, the endoscopic video report includes the effective video frame ratio of the endoscopic video; the reporting module is further used to determine the effective video frame ratio as the ratio of the total duration of at least one target video segment to the total duration of the endoscopic video.

[0101] The effective video frame ratio may be the ratio of effective video frames to the total endoscopic video frames.

[0102] In a specific implementation, the reporting module can count the total duration of at least one target video segment, take the ratio between the total duration of at least one target video segment and the total duration of the endoscopic video as the effective video frame ratio, and generate an endoscopic video report of the target object based on the effective video frame ratio.

[0103] In this embodiment, the effective video frame ratio is determined by the ratio of the total duration of at least one target video segment to the total duration of the endoscopic video. The effective video frame ratio can be used to reflect the quality of the endoscopic inspection. A lower effective video frame ratio indicates that the quality of the endoscopic inspection is poor. At this time, the maintenance personnel can be notified to check the endoscopic inspection equipment.

[0104] In one embodiment, the endoscopic video report includes the abnormality level of the target object; the reporting module is further used to determine the abnormal video segments in the endoscopic video based on the corrected condition identification list, and obtain the abnormality level of the target object based on the ratio of the total duration of the abnormal video segments to the total duration of the endoscopic video.

[0105] The abnormal video clip may be a video clip that represents an abnormal condition of the target object.

[0106] In a specific implementation, the reporting module can also determine the abnormal video segments in the endoscopic video based on the corrected condition identification list, and calculate the ratio between the total duration of the abnormal video segments and the total duration of the endoscopic video to characterize the degree of abnormality of the target object.

[0107] For example, it is known that the corrected status identification list is [0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1,1,1,1,1,1,1,1,1,0,0,0,0,0,0], and the video segment corresponding to status identification 1 is determined as an abnormal video segment, and 25 abnormal video segments are obtained. Each video segment is 3 seconds long, and the total length of the abnormal video segments is 75 seconds. Assuming that the total length of the endoscopic video is 150 seconds, the abnormality level of the target object is 75 / 150*100%=50%.

[0108] In this embodiment, by determining the abnormal video segments in the endoscopic video based on the corrected condition identification list, and obtaining the abnormality level of the target object based on the ratio of the total duration of the abnormal video segments to the total duration of the endoscopic video, the abnormality level of the target object can be quantified in the endoscopic video report, thereby improving the reference value of the endoscopic video report.

[0109] In one embodiment, Figure 3 As shown, an endoscopic video processing method is provided, which is described by taking the method applied to an endoscopic video processing system as an example, and includes the following steps:

[0110] Step S310, determining at least one target video segment from the endoscopic video of the target object;

[0111] Step S320, identifying the target status identifier of the target object corresponding to each target video clip, and obtaining a status identifier list of the target object according to each target status identifier;

[0112] Step S330, modifying the status identification list to obtain a modified status identification list;

[0113] Step S340: Determine the endoscopic video report of the target object according to the corrected status identification list.

[0114] In a specific implementation, an endoscopic video of the target object can be collected, the endoscopic video can be divided according to a preset duration to obtain at least one target video segment, each target video segment can be identified to obtain a target status identifier corresponding to each target video segment, the target status identifiers corresponding to all the target video segments can be combined into a status identifier list, the status identifier list can be corrected, information can be extracted from the corrected status identifier list, and an endoscopic video report of the target object can be generated.

[0115] Since the specific processing process of the endoscopic video processing system has been described in detail in the aforementioned embodiment, it will not be repeated here.

[0116] The above-mentioned endoscopic video processing method determines at least one target video segment from the endoscopic video of the target object, identifies the target condition identifier of the target object corresponding to each target video segment, and obtains a condition identifier list of the target object based on each target condition identifier, corrects the condition identifier list to obtain a corrected condition identifier list, and determines an endoscopic video report of the target object based on the corrected condition identifier list; multiple video segments in the endoscopic video can be identified separately to obtain a condition identifier corresponding to each video segment, wherein each video segment corresponds to a small section of the intestine of the target object, and the condition identifier is used to characterize the condition of this section of the intestine. By correcting the condition identifier list and removing inaccurate condition identifiers, the intestinal condition can be accurately determined based on the remaining relatively accurate condition identifiers, and the obtained endoscopic video report of the target object has high accuracy.

[0117] In order to facilitate those skilled in the art to have a deeper understanding of the embodiments of the present application, a specific example will be described below.

[0118] Figure 4A schematic flow chart of an endoscopic video processing method is provided. Figure 4 The endoscopic video processing method may specifically include the following steps:

[0119] Step S410, border area cropping. After reading the video clip (for example, reading the video signal with 3 seconds as one video clip), in order to improve the effect of feature extraction and increase the calculation speed, the border area cropping process can be performed on the video clip. Specifically, the black border of the video frame in the video clip can be cropped to retain only valid information, wherein the black border records invalid information such as shooting time, hospital, patient age, etc., and the black border accounts for a large proportion of the endoscopic image. Cropping the black border can increase the calculation speed, reduce the calculation memory, and facilitate the subsequent extraction of useful information.

[0120] Step S420, non-information segment identification. Digestive endoscopy videos usually contain non-information video frames such as blur, artifacts, and liquid reflections. The proportion of non-information frames in colonoscopy videos can reach up to about 30% to 40%. These non-information frames will affect the subsequent algorithm's extraction of lesion features, thereby affecting the algorithm's accuracy in identifying IBD disease types.

[0121] Specifically, a video classification method can be used to identify non-information segments. The input is a short continuous colonoscopy video frame. A single frame image provides spatial information (intestinal color, texture and other features), and continuous multiple frames provide temporal information (change information of the same target in the previous and next frames). The video classification algorithm model is used to determine whether the video segment is a non-information segment. The specific video classification algorithm can use a traditional video classification algorithm and a deep learning video classification algorithm. The traditional video classification algorithm first uses a feature extraction algorithm, such as DT (dense trajectories) to extract spatial and temporal features in the colonoscopy video segment, and then uses a classification algorithm such as SVM (Support Vector Machine) to classify the features, thereby achieving the purpose of video classification and recognition.

[0122] You can also use a video classification algorithm based on deep learning. The deep learning method can achieve end-to-end feature extraction and classification. Common methods include the CNN-LSTM method, in which CNN (Convolutional Neural Networks) can well extract the spatial features in each frame of video, and LSTM (Long Short-Term Memory) can model the temporal change information between consecutive frames, and finally classify the features to complete the classification and recognition of the video. If the video clip is a non-information clip, delete it and continue to input the next video clip for calculation.

[0123] Step S430, IBD disease type identification. Based on the recognition result of non-information segment identification, a clear, continuous, uniform information segment with very obvious intestinal characteristics is obtained. The segment is input into the IBD disease type recognition algorithm for identification. The specific recognition algorithm adopts the video classification algorithm in step S420. Because the lesions in the endoscopic manifestations of IBD are mostly distributed along the intestinal wall, and endoscopic examination is generally along the centerline of the intestine, the video signal restores the distribution characteristics of endoscopic examination and IBD lesions very well, so the video-based algorithm model can achieve a higher recognition accuracy.

[0124] In addition, during the endoscopic diagnosis of IBD, the terminal ileum is examined through the ileocecal opening and an endoscopic video is taken. The affected part of CD is mostly at the terminal ileum, which can be used to distinguish CD from UC. Based on the recognition results of non-information fragments, clear, continuous, uniform information fragments with very obvious intestinal characteristics are obtained. The information fragments are input into the video classification algorithm for terminal ileum recognition. The results of terminal ileum recognition and IBD disease type recognition are comprehensively evaluated as follows:

[0125] 1. If it is identified that the patient is currently located at the terminal ileum and the disease type is CD, the disease type is determined to be CD;

[0126] 2. If the patient is currently located at the terminal ileum and the disease type is UC or normal, the identification result of the terminal ileum is ignored, and the IBD identification results of other intestinal segments are comprehensively considered to determine the final disease type;

[0127] 3. If the terminal ileum is not identified (considered as the doctor not reaching the terminal ileum for examination as required), the final disease type is determined by comprehensively considering the IBD identification results of other intestinal segments.

[0128] In summary, combined with the endoscopic imaging features of the terminal ileum, the type of IBD can be identified more accurately.

[0129] Step S440, post-processing operation. The algorithm will inevitably produce a small number of erroneous recognition results, so it is necessary to perform post-processing operations on the recognition results of multiple consecutive segments to reduce the false positives of the recognition results. Specifically, a temporal smoothing method can be used. For example, for a video, it is divided into several shorter video segments. When the result predicted by the IBD disease type recognition algorithm is CD, it is recorded as 1, when it is predicted to be normal, it is recorded as 0, and when it is predicted to be UC, it is recorded as -1. Therefore, the prediction result for the entire video can be recorded as a digital list [0,0,0,0,0,0,0,…0,0,1,1,1,1,1,1,1,-1,1,1,1,-1,1,1,1,0,1,1,1,-1,1,1,1,1…,1,1,0,0,0,0,0,0,0]. By determining a threshold, such as n = 8, taking t (t ≥ n, such as t = 10) consecutive short video clips for algorithm recognition, if the number of video clips recognized as 1 (i.e. CD) is greater than n, then the recognition results of the remaining tn video clips are considered to be wrong, and the recognition results of the remaining tn video clips are all changed to 1. This method is based on the continuity between short video clips, so it is more effective. Where n can also be selected from other numbers such as 2, 3, 4, 5, 6, etc.

[0130] Among them, CD / UC / normal can also be represented by other numbers or symbols, and the order can be changed.

[0131] Step S450: Output the colonoscopy diagnosis report. Based on the above method, the information frame ratio in the colonoscopy video and the severity of IBD can be quantified. info The total duration of all retained video clips can be divided by the total length of the video, that is, P info =T info / T full , where T info is the total duration of all retained video clips, T full The severity of IBD can be calculated by dividing the total duration of all video clips containing IBD disease by the total length of the video, that is, P IBD =T IBD / T full , where T IBD is the total duration of all video clips containing IBD disease. info It can reflect the quality of the endoscopic examination and the severity of IBD. IBD It can provide reference for doctors' treatment plans.

[0132] It should be noted that the time length of the above video clips can be 3 seconds, or 1 second, 2 seconds, 5 seconds, etc. Before identifying the IBD disease type, the video clips can also be pre-processed by data augmentation, cropping, flipping, twisting, removing Gaussian noise, etc.

[0133] It should also be noted that in the process of identifying non-information segments and IBD disease types, the traditional video classification algorithm can also use HOG (Histogram of Oriented Gridients) as a video feature extractor and then use SVM as a classifier; the video classification algorithm based on deep learning can also use other networks, such as 3DCNN (three-dimensional convolutional neural network) and other methods.

[0134] The above-mentioned endoscopic video processing method combines video preprocessing and post-processing methods with the IBD disease type recognition algorithm, takes video clips as input, and makes full use of the temporal and spatial information of the video to achieve IBD auxiliary diagnosis that is more accurate, fully automated, and closer to clinical scenarios.

[0135] Moreover, in addition to cropping the black border of the colonoscopy video, the above-mentioned video preprocessing method also proposes a non-information frame video segment recognition algorithm, which can remove invalid information in the video while maintaining the continuity of the remaining video segments, thereby providing better data input for the subsequent video-based IBD disease type recognition algorithm, helping the algorithm to extract effective features and more accurately identify the disease type of IBD.

[0136] Furthermore, the above-mentioned video-based IBD disease type recognition algorithm uses video clips as input and truly restores the visual features of clinical endoscopic diagnosis, where single-frame images provide spatial feature information (such as intestinal color, texture, etc.), and continuous multi-frame images provide temporal change information (motion information of the same target between continuous frames). The endoscopic manifestations of IBD are mostly longitudinal, with large areas of ulcers (lesions are mostly distributed along the intestinal wall), and endoscopic examinations are generally conducted along the centerline of the intestine. The video signal well restores the distribution characteristics of endoscopic examinations and IBD lesions. Therefore, the video-based algorithm model is closer to the clinical scenario and can achieve higher accuracy than the existing algorithms based on single images.

[0137] Finally, the above post-processing algorithm, based on the continuity characteristics of the video, filters out and corrects the false positive points of the recognition results by performing temporal smoothing, which can achieve a higher recognition accuracy compared to the existing recognition method based only on images.

[0138] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0139] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an endoscopic video processing method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0140] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0141] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0142] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0143] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0145] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0146] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An endoscopic video processing system, characterized in that: The system includes an acquisition module, an identification module, a correction module and a reporting module; The acquisition module is used to determine at least one target video segment from the endoscopic video of the target object; The identification module is used to identify the target status identifier of the target object corresponding to each of the target video clips, and obtain a status identifier list of the target object according to each of the target status identifiers; The correction module is used to correct the status identification list to obtain a corrected status identification list; The reporting module is used to determine the endoscopic video report of the target object according to the corrected status identification list.

2. The system according to claim 1, characterized in that The acquisition module is further used to determine at least one original video segment from the endoscopic video, and pre-process each of the original video segments to obtain the at least one target video segment.

3. The system according to claim 2, characterized in that The acquisition module is further used to determine the border area of ​​the original video segment and remove the border area from the original video segment to obtain the target video segment.

4. The system according to claim 2, characterized in that The acquisition module is also used to extract the spatial features and temporal features of the original video clip to obtain a video feature vector of the original video clip; perform feature classification on the video feature vector to obtain a first classification result of the original video clip; when the first classification result meets a preset result, the original video clip is determined as the target video clip; the preset result includes that the proportion of non-information video frames in the original video clip does not exceed a preset threshold.

5. The system according to claim 2, characterized in that The acquisition module is further used to input the original video segment into a pre-trained video classification model to obtain a second classification result corresponding to the original video segment; When the second classification result meets a preset result, the original video segment is determined as the target video segment; the preset result includes that the proportion of non-information video frames in the original video segment does not exceed a preset threshold.

6. The system according to claim 1, characterized in that The identification module is also used to identify the acquisition position of the current target video segment and the initial state identifier corresponding to the current target video segment; when the acquisition position matches the preset position and the initial state identifier matches the preset identifier, the target state identifier corresponding to the current target video segment is obtained according to the initial state identifier.

7. The system according to claim 6, characterized in that The identification module is further configured to set the target status identifier corresponding to the current target video segment to a null value when the acquisition position does not match the preset position or the initial status identifier does not match the preset identifier; The next target video segment is used as the current target video segment, and the process returns to the step of identifying the acquisition position of the current target video segment and the initial status identifier corresponding to the current target video segment.

8. The system according to claim 1, characterized in that The correction module is further used to segment the target status identifier in the status identifier list according to the acquisition position of the target video segment to obtain a status identifier subset; determine the identifier type of each target status identifier in the status identifier subset, and determine the identifier type whose proportion in the status identifier subset exceeds a preset threshold as the target type; And according to the target type, the status identification subset is modified.

9. The system according to claim 1, characterized in that The endoscopic video report includes the abnormality degree of the target object; the reporting module is also used to determine the abnormal video segments in the endoscopic video based on the corrected condition identification list, and obtain the abnormality degree of the target object based on the ratio of the total duration of the abnormal video segments to the total duration of the endoscopic video.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the data processing steps of the system according to any one of claims 1 to 9 are implemented.