Endoscope picture generation system and method and endoscope system

Abstract

The invention relates to an endoscope picture generation system and method and an endoscope system. The endoscope picture generation system comprises a video acquisition module used for acquiring an endoscope video obtained by shooting a target part by an endoscope; the state determination module is used for determining a target video clip in the endoscope video and determining a target shooting state corresponding to the target video clip; the target selection module is used for determining a target video frame from the target video clip under the condition that the target shooting state accords with a preset state; the center position of the target video frame comprises an abnormal area; and the picture generation module is used for obtaining a target endoscope picture of the target part according to the target video frame. By adopting the system, the efficiency of endoscopy can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an endoscopic image generation system, method and endoscopic system. Background Art

[0002] Gastrointestinal endoscopy is of great significance for the analysis of the patient's condition and subsequent treatment. The diagnostic report of gastrointestinal endoscopy usually includes endoscopic images and the doctor's diagnosis conclusion.

[0003] At present, endoscopic images in diagnostic reports are usually generated manually. During the endoscopic examination, when the doctor encounters a place where an image needs to be taken, he will keep the scope stable, and then trigger the shooting button on the scope to take an image to obtain a candidate endoscopic image. After the endoscopic examination is completed, the target endoscopic image suitable for placement in the diagnostic report is selected from multiple candidate endoscopic images. However, manual filming can easily affect the doctor's workflow for endoscopic examination operations, causing the endoscopic examination time to be prolonged, thereby reducing the efficiency of the endoscopic examination.

[0004] Therefore, the current endoscopic image generation technology has the problem of reduced endoscopic examination efficiency. Summary of the invention

[0005] Based on this, it is necessary to provide an endoscopic image generation system, method, computer device, computer readable storage medium, computer program product and endoscope system that can avoid the reduction of endoscopic examination efficiency in response to the above-mentioned technical problems.

[0006] In a first aspect, the present application provides an endoscopic image generation system, the system comprising:

[0007] A video acquisition module is used to acquire an endoscopic video obtained by photographing a target part with an endoscope;

[0008] A state determination module, used to determine a target video segment in the endoscopic video, and determine a target shooting state corresponding to the target video segment;

[0009] A target selection module, used for determining a target video frame from the target video clip when the target shooting state meets a preset state; the center position of the target video frame contains an abnormal area;

[0010] The image generation module is used to obtain a target endoscopic image of the target part according to the target video frame.

[0011] In one of the embodiments, the state determination module is further used to determine an original video segment from the endoscopic video based on a preset duration; remove the border area in the original video segment to obtain a video segment after the border is removed; remove invalid video frames from the video segment after the border is removed to obtain the target video segment.

[0012] In one embodiment, there is at least one target video clip; the state determination module is further used to perform state identification on each of the target video clips to obtain the original shooting state corresponding to each of the target video clips; obtain an original state list based on at least one of the original shooting states; perform correction processing on the original state list to obtain a corrected state list; and obtain the target shooting state corresponding to each of the target video clips based on the corrected state list.

[0013] In one of the embodiments, the state determination module is further used to determine at least two adjacent video frames from the target video clip; determine the displacement vector of the target point on the target part in the at least two adjacent video frames; and classify the displacement vector to obtain the original shooting state of the target video clip.

[0014] In one of the embodiments, the target selection module is further used to determine, when the target shooting state meets the preset state, a candidate video frame containing an abnormal area from the target video clip; determine the center point of the abnormal area as the first center point, and determine the center point of the candidate video frame as the second center point; and determine the candidate video frame as the target video frame when the distance between the first center point and the second center point is less than a preset distance threshold.

[0015] In one of the embodiments, the target selection module is further used to determine the first center point according to the outline of the abnormal area, or to determine the first center point according to the minimum circumscribed rectangle of the abnormal area.

[0016] In one of the embodiments, the image generation module is further used to perform threshold segmentation processing on the target video frame to obtain a video frame after threshold segmentation; perform interference frame recognition processing on the video frame after threshold segmentation to obtain an initial endoscopic image of the target part; and determine the target endoscopic image of the target part based on the initial endoscopic image.

[0017] In one of the embodiments, when there are at least two initial endoscopic images, the image generation module is further used to determine the histogram similarity between different initial endoscopic images; based on the histogram similarity, redundant endoscopic images are removed from the initial endoscopic images to obtain the target endoscopic image of the target part.

[0018] In a second aspect, the present application provides a method for generating an endoscopic image. The method comprises:

[0019] Acquire an endoscopic video obtained by photographing the target area with an endoscope;

[0020] Determining a target video segment in the endoscopic video, and determining a target shooting state corresponding to the target video segment;

[0021] In the case where the target shooting state meets the preset state, determining a target video frame from the target video segment; the center position of the target video frame includes an abnormal area;

[0022] A target endoscopic image of the target part is obtained according to the target video frame.

[0023] In one embodiment, determining a target video segment in the endoscopic video and determining a target shooting state corresponding to the target video segment includes:

[0024] Determining an original video segment from the endoscopic video based on a preset duration;

[0025] Removing the border area in the original video clip to obtain a video clip after the border is removed;

[0026] Invalid video frames are removed from the video segment after the border is removed to obtain the target video segment.

[0027] In one embodiment, the number of the target video segment is at least one, and the step of determining the target video segment in the endoscopic video and determining the target shooting state corresponding to the target video segment further includes:

[0028] Performing state recognition on each of the target video clips to obtain an original shooting state corresponding to each of the target video clips;

[0029] Obtaining an original state list according to at least one of the original shooting states;

[0030] Correcting the original status list to obtain a corrected status list;

[0031] The target shooting state corresponding to each target video segment is obtained according to the corrected state list.

[0032] In one embodiment, the step of identifying the state of each target video segment to obtain the original shooting state corresponding to each target video segment includes:

[0033] Determining at least two adjacent video frames from the target video segment;

[0034] Determine a displacement vector of a target point on the target part in the at least two adjacent video frames;

[0035] The displacement vectors are classified to obtain the original shooting state of the target video segment.

[0036] In one of the embodiments, when the target shooting state meets a preset state, determining a target video frame from the target video segment includes:

[0037] When the target shooting state meets the preset state, determining a candidate video frame containing an abnormal area from the target video segment;

[0038] Determine the center point of the abnormal area as a first center point, and determine the center point of the candidate video frame as a second center point;

[0039] When the distance between the first center point and the second center point is less than a preset distance threshold, the candidate video frame is determined as the target video frame.

[0040] In one embodiment, determining the center point of the abnormal area as the first center point includes:

[0041] The first center point is determined according to the outline of the abnormal area, or the first center point is determined according to the minimum circumscribed rectangle of the abnormal area.

[0042] In one embodiment, obtaining a target endoscopic image of the target part according to the target video frame includes:

[0043] Performing threshold segmentation processing on the target video frame to obtain a video frame after threshold segmentation;

[0044] Performing interference frame recognition processing on the video frames after the threshold segmentation to obtain an initial endoscopic image of the target part;

[0045] The target endoscopic image of the target part is determined according to the initial endoscopic image.

[0046] In one embodiment, when there are at least two initial endoscopic images, determining the target endoscopic image of the target part according to the initial endoscopic images includes:

[0047] Determining the histogram similarity between different initial endoscopic images;

[0048] According to the histogram similarity, redundant endoscopic images are removed from the initial endoscopic images to obtain the target endoscopic image of the target part.

[0049] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the system as described in any one of the first aspects or the method as described in any one of the second aspects is implemented.

[0050] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the system described in any one of the first aspects or the method described in any one of the second aspects is implemented.

[0051] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the system as described in any one of the first aspect or the method as described in any one of the second aspect is implemented.

[0052] In a sixth aspect, the present application further provides an endoscope system, which includes an endoscope, a display, a light source device, and an endoscope image generation system as described in any one of the first aspects above.

[0053] The above-mentioned endoscopic image generation system, method, computer device, storage medium, computer program product and endoscopic system obtain the endoscopic video obtained by endoscope shooting the target part through the video acquisition module, the state determination module determines the target video segment in the endoscopic video, and determines the target shooting state corresponding to the target video segment. The target selection module determines the target video frame from the target video segment when the target shooting state meets the preset state, and the image generation module obtains the target endoscopic image of the target part based on the target video frame; during the endoscopic examination process, the target video segment and the shooting state corresponding to the target video segment can be automatically identified from the endoscopic video, the target video frame can be selected from the target video segment according to the shooting state, and the endoscopic image can be directly generated based on the target video frame. Since there is no need for manual filming, the efficiency of endoscopic examination is improved.

[0054] Moreover, the automatic generation of endoscopic images avoids the subjective factors existing in manual filming and improves the reliability of endoscopic image generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a structural block diagram of an endoscopic image generation system in one embodiment;

[0056] Figure 2 is a scatter diagram of an original shooting state in one embodiment;

[0057] Figure 3 A schematic diagram of determining a target video frame in one embodiment;

[0058] Figure 4 A schematic diagram of interference frame identification in one embodiment;

[0059] Figure 5 The figure is a flow chart of an automatic filming method for digestive tract endoscopy in one embodiment;

[0060] Figure 6 is a schematic diagram of a process of generating an endoscopic image in one embodiment;

[0061] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] Gastrointestinal endoscopy is to pass an endoscope through a designated part of the human digestive tract and transmit the internal image of that part to the patient's body for the doctor to view, so as to assist the doctor in analyzing the patient's condition and formulating a treatment plan. The diagnostic report of gastrointestinal endoscopy usually contains endoscopic images and the doctor's diagnosis conclusion. Among them, the endoscopic images are usually obtained by manual filming during the endoscopic examination. When the doctor encounters a place where filming is required during the examination, he will keep the scope stable and then press the "freeze" button on the scope to take a film. After the examination, the image suitable for placement in the diagnostic report is selected from all the images obtained by filming as the endoscopic image. However, the operation of filming by using the "freeze" button on the scope will affect the doctor's operational workflow, prolong the time of endoscopic examination, and reduce the efficiency of endoscopic examination.

[0064] In order to improve the efficiency of endoscopic examination, in the embodiment of the present application, the device can obtain the endoscopic video obtained by photographing the target part with an endoscope, determine the target video segment in the endoscopic video and the target shooting state corresponding to the target video segment, determine the target video frame from the target video segment according to the target shooting state, and generate the target endoscopic picture of the target part according to the target video frame. Since the endoscopic picture can be automatically generated during the endoscopic examination, there is no need for manual filming, which improves the efficiency of the endoscopic examination.

[0065] The embodiments of the present application can be applied to any endoscopic examination scenario and can be applied to various endoscopic devices, terminals or servers. Among them, the endoscopic device can be but not limited to various colonoscopy devices, gastroscopy devices, bronchoscope devices, etc. The terminal can be but not limited to various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0066] In one embodiment, Figure 1 As shown, an endoscopic image generation system is provided, which is described by taking the application in an endoscopic device as an example. The system includes:

[0067] The video acquisition module 110 is used to acquire an endoscopic video obtained by photographing a target part with an endoscope.

[0068] The target site may be a site targeted for endoscopic examination, such as the digestive tract.

[0069] The endoscopic video may be a video captured by an endoscope.

[0070] In a specific implementation, the endoscopic image generation system can be deployed on the endoscopic device. During the endoscopic examination, the endoscopic device is controlled to photograph the target area, and the photographed endoscopic video is input into the video acquisition module 110 of the endoscopic image generation system, so that the video acquisition module 110 acquires the endoscopic video.

[0071] In actual applications, an image sensor can be set at the front end of the endoscopic device. During the endoscopic examination, the front end is controlled to pass through the digestive tract, and the image sensor is controlled to shoot the inside of the digestive tract to obtain an endoscopic video of the digestive tract. The image sensor can also transmit the endoscopic video of the digestive tract to the video acquisition module 110 of the endoscopic image generation system on the endoscopic device.

[0072] The state determination module 120 is used to determine a target video segment in the endoscopic video, and determine a target shooting state corresponding to the target video segment.

[0073] The target video segment may be a video segment used to generate an endoscopic image.

[0074] The target shooting state may be a shooting state of a target video clip, for example, an advancing state, an retreating state, or a stationary state.

[0075] In a specific implementation, the video acquisition module 110 can transmit the endoscopic video of the target part to the state determination module 120. The state determination module 120 determines at least one target video segment from the endoscopic video according to a preset duration. After performing processing such as border removal and invalid information removal on each target video segment, the state determination module 120 can identify the shooting status of each target video segment to obtain the target shooting status.

[0076] As an optional embodiment of the present application, the above-mentioned state determination module 120 is also used to determine the original video segment from the endoscopic video based on a preset duration; remove the border area in the original video segment to obtain a video segment after the border is removed; remove invalid video frames from the video segment after the border is removed to obtain a target video segment.

[0077] The original video clip may be a video clip originally collected from the endoscopic video.

[0078] The invalid video frame may be a video frame containing invalid information such as blur, artifacts, etc.

[0079] In a specific implementation, the state determination module 120 can determine at least one original video segment from the endoscopic video according to a preset duration. Since the border area accounts for a large proportion of the original video segment, and the border area records the shooting time of the endoscopic video, the shooting hospital, the patient's age and other information that is irrelevant to the generation of the endoscopic image, the border of each original video segment can be removed to obtain at least one video segment after the border is removed. In addition, since invalid video frames containing invalid information such as blur and artifacts may exist in the original video segment, in order to improve the accuracy of subsequent shooting state determination, each video segment after the border is removed can also be identified to obtain the invalid video frames therein, and the invalid video frames are removed from the video segment after the border is removed to obtain the target video segment.

[0080] For all the video frames in the video clip after the border is removed, an image classification algorithm can be used to classify them, obtain valid video frames and invalid video frames in the video clip after the border is removed, and delete the invalid video frames. The image classification algorithm can be, but is not limited to, a classification algorithm based on a convolutional neural network.

[0081] In the present embodiment, an original video segment is determined from the endoscopic video based on a preset duration; the border area in the original video segment is removed to obtain a video segment after the border is removed; invalid video frames are removed from the video segment after the border is removed to obtain a target video segment. By removing the border area, the calculation speed of endoscopic image generation can be improved and the calculation memory can be reduced. By removing the invalid video frames, the shooting status of the target video segment can be determined based on valid information only, thereby improving the accuracy of the shooting status determination and thereby improving the reliability of endoscopic image generation.

[0082] As an optional embodiment of the present application, the above-mentioned state determination module 120 is also used to determine at least two adjacent video frames from the target video clip; determine the displacement vector of the target point on the target part in at least two adjacent video frames; classify the displacement vector to obtain the original shooting state of the target video clip.

[0083] The target point may be a voxel on the target part.

[0084] The original shooting state may be the motion state of the front end of the endoscope device initially identified, including but not limited to an advancing state, a retracting state, and a stationary state.

[0085] In a specific implementation, the state determination module 120 can select at least two adjacent video frames from the target video clip, select a target point of the target part, determine the position information of the target point in each video frame in at least two adjacent video frames, generate a displacement vector according to the determined position information, and obtain the original shooting state corresponding to the target video clip by classifying the displacement vector.

[0086] For example, the state determination module 120 selects two adjacent video frames F1 and F2 from the current target video clip, and for a specified voxel P on the digestive tract, determines that the position coordinates of P in F1 are (x1, y1), and the position coordinates of P in F2 are (x2, y2), obtains the position vector I based on (x1, y1) and (x2, y2), and classifies I, so as to determine whether the current target video clip is in an advance state, a retreat state, or a stationary state.

[0087] In this embodiment, by determining at least two adjacent video frames from the target video clip; determining the displacement vector of the target point on the target part in at least two adjacent video frames; and classifying the displacement vector to obtain the original shooting state of the target video clip, the shooting state of the target video clip can be quickly determined, thereby improving the efficiency of determining the shooting state, and thereby improving the efficiency of generating endoscopic images.

[0088] As an optional embodiment of the present application, there is at least one target video clip; the above-mentioned state determination module 120 is also used to identify the state of each target video clip to obtain the original shooting state corresponding to each target video clip; according to at least one original shooting state, an original state list is obtained; the original state list is corrected to obtain a corrected state list; according to the corrected state list, the target shooting state corresponding to each target video clip is obtained.

[0089] The original state list may be a list consisting of the original shooting states corresponding to the video frames in the target video segments.

[0090] The target shooting state may be a corrected motion state of the front end of the endoscope device used to generate the endoscope image.

[0091] In a specific implementation, the state determination module 120 can identify the original shooting state of each target video clip, represent the identified original shooting state with an original state identifier as the state identifier of each video frame in the target video clip, arrange the state identifiers corresponding to each video frame in each target video clip to obtain an original state list, and use filtering, time smoothing and other methods to correct the original state list to obtain a corrected state list. The corrected state list is composed of the corrected state identifiers. According to the corrected state identifiers, the target shooting state corresponding to each target video clip can be obtained.

[0092] For example, assuming that each target video clip contains 10 video frames, the state identifier of the camera-in state is recorded as 1, the state identifier of the camera-out state is recorded as -1, and the state identifier of the camera-still state is recorded as 0. The shooting state of each video frame in each target video clip is represented by the state identifier, and they are arranged in sequence to obtain an original state list [1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, -1, -1, -1, -1, -1, 0, 0, -1, -1, -1]. In order to improve the accuracy of determining the shooting state, the original state list can be sorted. The target shooting state of the target video clip corresponding to the 1st to 10th video frames is determined as the advance state, the target shooting state of the target video clip corresponding to the 11th to 20th video frames is the stationary state, and the target shooting state of the target video clip corresponding to the 21st to 30th video frames is the retreat state.

[0093] Figure 2A scatter diagram of the original shooting state is provided, where the solid circle ● indicates the correctly identified camera-in or camera-out state, the hollow circle ○ indicates the correctly identified still state, and the square ■ indicates that the camera-in or camera-out state is incorrectly identified as the still state, that is, a false positive occurs in the determination of the shooting state. Figure 2 The original state list can be [1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, …, -1, -1, -1, -1, -1, 0, 0, -1, -1]. In order to avoid false positives in the determination of the shooting state, the original state list can be time-smoothed based on the continuity between video frames. Specifically, a threshold can be set in advance, for example, n=8, and t consecutive (t≥n, for example, t=10) video frames are taken for recognition. If the number of video frames recognized as 1 is greater than n, it is determined that the recognition results of the remaining tn video frames are false positives, and their recognition results are changed to 1.

[0094] In this embodiment, the state of each target video clip is identified to obtain the original shooting state corresponding to each target video clip; based on at least one original shooting state, an original state list is obtained; the original state list is corrected to obtain a corrected state list; based on the corrected state list, the target shooting state corresponding to each target video clip is obtained, which can avoid false positives in the determination of the shooting state, improve the accuracy of the determination of the shooting state, and improve the accuracy of the generation of endoscopic images.

[0095] The target selection module 130 is used to determine a target video frame from the target video clip when the target shooting state meets the preset state; the center position of the target video frame contains an abnormal area.

[0096] Among them, the preset state may be a static state.

[0097] The target video frame may be a video frame used to generate an endoscopic image.

[0098] The abnormal area may be an area where an abnormal situation occurs, for example, a lesion area.

[0099] In a specific implementation, the state determination module 120 can input the target video segment in the endoscopic video and the target shooting state corresponding to the target video segment into the target selection module 130. The target selection module 130 identifies the target shooting state. If it is identified that the target shooting state meets the preset state, for example, the target shooting state is a static state, then a video frame containing an abnormal area at the center position is selected from the target video segment corresponding to the target shooting state as the target video frame. For example, a video frame in which the lesion area in the target video segment is located in the center of the field of view can be used as the target video frame and input into the image generation module 140.

[0100] As an optional embodiment of the present application, the above-mentioned target selection module 130 is also used to determine, when the target shooting state meets the preset state, a candidate video frame containing an abnormal area from the target video clip; determine the center point of the abnormal area as the first center point, and determine the center point of the candidate video frame as the second center point; when the distance between the first center point and the second center point is less than a preset distance threshold, determine the candidate video frame as the target video frame.

[0101] In a specific implementation, the target selection module 130 can, when recognizing that the target shooting state conforms to the static state, determine a video frame containing an abnormal area from the target video clip corresponding to the target shooting state as a candidate video frame, determine the center point of the abnormal area in the candidate video frame as the first center point, determine the center point of the candidate video frame as the second center point, determine the distance between the first center point and the second center point, and if the distance between the first center point and the second center point is less than a preset distance threshold, determine that the candidate video frame is the target video frame; otherwise, if the distance between the first center point and the second center point is greater than or equal to the preset distance threshold, determine that the candidate video frame is not the target video frame, and then continue to look for the next candidate video frame and determine whether it is the target video frame.

[0102] For example, the target selection module 130 can sequentially search in the static target video clip, and use the first video frame containing the lesion area as the candidate video frame. If the lesion area in the candidate video frame is located in the center of the visual field, the candidate video frame is used as the target video frame. Otherwise, if the lesion area is not located in the center of the visual field, the next candidate video frame is sequentially searched and its lesion area is detected to be located in the center of the visual field. Whether the lesion area is located in the center of the visual field can be determined by judging whether the distance between the center point of the lesion area and the center point of the video frame exceeds a preset distance threshold.

[0103] In this embodiment, when the target shooting state meets the preset state, a candidate video frame containing an abnormal area is determined from the target video segment; the center point of the abnormal area is determined as the first center point, and the center point of the candidate video frame is determined as the second center point; when the distance between the first center point and the second center point is less than a preset distance threshold, the candidate video frame is determined as the target video frame, so that it can be ensured that the lesion area in the target video frame is located in the center of the field of view, thereby obtaining a target endoscopic image suitable for display.

[0104] As an optional embodiment of the present application, the target selection module 130 is further configured to determine the first center point according to the outline of the abnormal area, or to determine the first center point according to the minimum circumscribed rectangle of the abnormal area.

[0105] In a specific implementation, after determining the candidate video frame, the target selection module 130 can determine the abnormal area from the candidate video frame, determine the major axis and minor axis of the abnormal area according to the outline of the abnormal area, and determine the intersection of the major axis and the minor axis as the first center point. Alternatively, the target selection module 130 can also determine the minimum circumscribed rectangle of the abnormal area, and determine the center point of the minimum circumscribed rectangle as the first center point.

[0106] Figure 3 A schematic diagram of determining a target video frame is provided. Figure 3 , when it is determined that the front end of the endoscope device is in a stationary state, the target video frame used to generate the endoscopic image in the diagnostic report can be determined in the corresponding target video clip. Since the endoscopic image in the diagnostic report usually contains the lesion area, and the actual endoscopic video shooting process is often recorded from near to far, the doctor will select the video frame with the lesion area located in the center of the field of view, without interference or distortion, and clearly reflecting the characteristics of the lesion as the endoscopic image. Specifically, after identifying the candidate video frames containing the lesion area, the contour of the lesion area in each candidate video frame can be determined by using segmentation, detection and other algorithms. For example, a multi-label segmentation network can be trained, such as UNet (U-shaped neural network), to segment lesions such as polyps and adenomas in the intestine to obtain the contour of the lesion area, and then the center point coordinates P of the lesion area contour and the center point coordinates O of the candidate video frame are calculated to obtain the distance OP between the center point of the lesion area and the center point of the candidate video frame, wherein the center point coordinates of the lesion area contour can be determined by the intersection method of the long and short diameters, and a smaller threshold t is set, for example, t is 10 pixels. If OP<t, it is considered that the lesion area is located in the center of the field of view, and the current candidate video frame is retained as the target video frame for generating endoscopic images in the diagnosis report.

[0107] In this embodiment, by determining the first center point based on the outline of the abnormal area, or determining the first center point based on the minimum circumscribed rectangle of the abnormal area, the center point of the lesion area in the candidate video frame can be quickly determined, thereby improving the speed of selecting the target video frame and the speed of generating the endoscopic image.

[0108] The image generation module 140 is used to obtain a target endoscopic image of a target part according to a target video frame.

[0109] In a specific implementation, the target selection module 130 can input at least one determined target video frame into the image generation module 140. After the image generation module 140 performs interference frame recognition and redundant image removal on each target video frame, it generates a target endoscopic image of the target part and displays the target endoscopic image in the diagnostic report of the endoscopic examination.

[0110] As an optional embodiment of the present application, the above-mentioned image generation module 140 is also used to perform threshold segmentation processing on the target video frame to obtain a video frame after threshold segmentation; perform interference frame recognition processing on the video frame after threshold segmentation to obtain an initial endoscopic image of the target part; and determine a target endoscopic image of the target part based on the initial endoscopic image.

[0111] The interference frames include, but are not limited to, video frames containing liquid reflections, artifacts, uneven lighting, blur and smoke.

[0112] In a specific implementation, the image generation module 140 can perform threshold segmentation on each target video frame to obtain at least one threshold-segmented video frame, and input the at least one threshold-segmented video frame into a trained neural network model respectively. The trained neural network model identifies the interference frames in the at least one threshold-segmented video frame, deletes the interference frames, and uses the remaining video frames as the initial endoscopic images of the target part. Thereafter, the target endoscopic image of the target part can be determined based on the initial endoscopic image.

[0113] Figure 4 A schematic diagram of interference frame identification is provided. Figure 4 Since the target video frame may contain interference information, such as liquid reflection, artifacts, uneven lighting, blur, smoke and other interference information, it is not suitable to be archived as a picture in the diagnosis report. Therefore, it is necessary to identify the interference frame of the target video frame. Specifically, after the target video frame is threshold segmented, it can be input into a trained neural network model, such as UNet. The trained neural network model identifies the target video frame after threshold segmentation, and identifies the area where the interference information is located in the target video frame after threshold segmentation. Since the interference information will affect the analysis and diagnosis of the lesion area, the video frame containing the interference information is not suitable as a picture in the diagnosis report, and the video frame containing the interference information can be deleted.

[0114] In this embodiment, threshold segmentation processing is performed on the target video frame to obtain a video frame after threshold segmentation; interference frame recognition processing is performed on the video frame after threshold segmentation to obtain an initial endoscopic image of the target part; based on the initial endoscopic image, a target endoscopic image of the target part is determined, the interference frame can be removed from the target video frame, and the endoscopic image in the diagnostic report is generated based on the remaining initial endoscopic image, which can improve the reliability of the endoscopic image.

[0115] As an optional embodiment of the present application, when there are at least two initial endoscopic images, the above-mentioned image generation module 140 is also used to determine the histogram similarity between different initial endoscopic images; based on the histogram similarity, redundant endoscopic images are removed from the initial endoscopic images to obtain a target endoscopic image of the target part.

[0116] The redundant endoscopic image may be a repeated initial endoscopic image.

[0117] In a specific implementation, when the number of initial endoscopic images is two or more, the image generation module 140 can determine the histogram similarity between any different initial endoscopic images, and compare the histogram similarity with a preset similarity threshold. If the histogram similarity exceeds the similarity threshold, only one of any different initial endoscopic images can be retained, and the other initial endoscopic images can be deleted. Otherwise, if the histogram similarity does not exceed the similarity threshold, there is no need to delete the initial endoscopic image. The above process is repeated until the histogram similarity between any different initial endoscopic images does not exceed the similarity threshold, and the final remaining initial endoscopic image is used as the target endoscopic image of the target part.

[0118] In practical applications, there may be a certain degree of duplication between the initial endoscopic images from which interference information is deleted. Due to the limited space in the diagnostic report, the graphic information should be as concise as possible, and redundant images can be removed from the initial endoscopic images. Specifically, methods such as image-based similarity (e.g., histogram similarity) can be used to perform redundancy removal operations. For example, the cross-correlation between any two different initial endoscopic images can be calculated, and the histogram similarity can be obtained based on the cross-correlation value.

[0119] In this embodiment, by determining the histogram similarity between different initial endoscopic images; based on the histogram similarity, redundant endoscopic images are removed from the initial endoscopic images to obtain a target endoscopic image of the target part, redundant endoscopic images can be removed to improve the efficiency of endoscopic image display.

[0120] The above-mentioned endoscopic image generation system obtains the endoscopic video obtained by photographing the target part with an endoscope through the video acquisition module, the state determination module determines the target video segment in the endoscopic video, and determines the target shooting state corresponding to the target video segment, the target selection module determines the target video frame from the target video segment when the target shooting state meets the preset state, and the image generation module obtains the target endoscopic image of the target part according to the target video frame; during the endoscopic examination process, the target video segment and the shooting state corresponding to the target video segment can be automatically identified from the endoscopic video, the target video frame can be selected from the target video segment according to the shooting state, and the endoscopic image can be directly generated according to the target video frame. Since there is no need for manual filming, the efficiency of endoscopic examination is improved.

[0121] Moreover, the automatic generation of endoscopic images avoids the subjective factors existing in manual filming and improves the reliability of endoscopic image generation.

[0122] In order to facilitate those skilled in the art to have a deeper understanding of the embodiments of the present application, a specific example will be described below.

[0123] Figure 5 A schematic diagram of the process of automatic filming method for gastrointestinal endoscopy is provided. Figure 5 The automatic filming method for digestive tract endoscopy may include the following steps:

[0124] Step S210, data preprocessing. The video signal is read according to a preset duration, for example, the video signal is read in segments of 3 seconds (or 1 second, 2 seconds, or 5 seconds), the original video segment is obtained, and the original video segment is preprocessed to obtain the target video segment, so as to better extract the features of the video segment and improve the calculation speed. The preprocessing operation may include:

[0125] (1) The black border of each video frame in the original video clip is cut off, and only valid information is retained. The black border records some invalid information such as the shooting time, hospital, and patient age, and the black border accounts for a large proportion of the video frame. Cutting off the black border can improve the calculation speed, reduce the calculation memory, and allow the algorithm to extract useful information.

[0126] (2) Removal of non-informative video frames: The original video clips usually contain non-informative video frames such as blur and artifacts, which can be removed by image classification algorithms (for example, convolutional neural networks) to ensure that the subsequent algorithms extract valid information and reduce the false positive results of the lens state recognition results.

[0127] It should be noted that the preprocessing operation may also include data augmentation (cropping, flipping, distortion, Gaussian noise, etc.), and the preprocessing process may not be strictly performed according to the steps of removing borders and removing non-information. Different preprocessing operations may be arranged randomly, and some preprocessing processes may be omitted.

[0128] Step S220, lens state recognition, that is, identifying the original shooting state corresponding to the target video clip. A video classification algorithm is used to identify the lens state, and the lens state is divided into three states: "camera in", "camera out" and "still". Among them, the input of the video classification algorithm can be the pre-processed target video clip, the target video clip is a small section of continuous video frames, a single frame image provides spatial information (color, texture and other features), and continuous multiple frames provide timing information (change information of the same voxel in continuous frames). The output of the video classification algorithm can be the original shooting state corresponding to the target video clip. The video classification algorithm can adopt the following three schemes:

[0129] (1) First, a feature extraction algorithm, such as DT (Dense Trajectories), is used to extract spatial features and temporal features from the preprocessed video clips, and then a classification algorithm such as SVM (Support Vector Machine) is used to classify the extracted spatial features and temporal features to obtain the lens state recognition result. Alternatively, HOG (Histogram of Oriented Gridients) is first used as a video feature extractor to extract features from the preprocessed video clips, and then SVM is used as a classifier to classify the extracted features to obtain the lens state recognition result.

[0130] (2) Using a video classification algorithm based on deep learning. The deep learning method can achieve end-to-end feature extraction and classification. The method of combining CNN (Convolutional Neural Networks) with LSTM (Long Short-Term Memory) can be used. Among them, CNN can well extract the spatial features in each frame of video, and LSTM can model the temporal change information between consecutive frames, and then perform feature classification to obtain the lens state recognition result. Among them, the video classification algorithm based on deep learning can also be implemented by 3D CNN (3D Convolutional Neural Networks).

[0131] (3) Optical flow method: The optical flow method represents the motion information of the same point between the previous and next frames as a vector, which is then represented as an optical flow graph. The optical flow graphs of the previous and next frames during the lens entry and lens exit processes are quite different, and can be easily distinguished through the classification algorithm, thereby obtaining the lens state recognition result.

[0132] Step S230, determine whether it is still. After identifying the original shooting state corresponding to the target video clip, the original shooting state can be recorded as 1 when it is in-shot, recorded as -1 when it is out-shot, and recorded as 0 when it is still. The above process is performed on the original shooting states of all target video clips in turn to obtain an original state list such as [1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, ..., -1, -1, -1, -1, -1, 0, 0, -1, -1]. In order to avoid false positive results in lens state recognition (for example, identifying an in-shot process as a still process, such as Figure 2 As shown in the square block, the original state list can be modified by using a signal processing method (e.g., filtering, time series smoothing, etc.), and the target shooting state corresponding to the target video segment can be obtained based on the modified state list. The time series smoothing method is as follows:

[0133] Plot the original state list as Figure 2 The scatter plot shown in the figure, where the solid circle ● represents the correctly identified camera-in or camera-out state, the hollow circle ○ represents the correctly identified static state, and the square ■ represents the incorrect identification of the camera-in or camera-out state as the static state. In this case, the identification is wrong and a false positive occurs. By determining a threshold value, for example, n=8, taking t consecutive (t≥n, for example, t=10) video frames for lens state identification, if the number of video frames identified as 0 is greater than n, the identification results of the remaining tn video frames are considered to be false positives, and the identification results of the remaining tn video frames are all changed to 0. This method is based on the continuity between video frames, so it is more effective. Where n can also be selected from other numbers such as 2, 3, 4, 5, 6, etc., to produce different correction effects.

[0134] It should be noted that the representation method of advancing the lens, retracting the lens and the static state is not limited to 1, -1 and 0, and the lens state can be divided into two states: "static" and "non-static". The static state can also be represented by -1 and the non-static state by 1. This application does not impose any restrictions on this.

[0135] Step S240, the shooting angle is determined. After determining the target shooting state corresponding to the target video clip, video frames that can be used as endoscopic pictures in the diagnostic report can be found from the target video clip in the static state. The diagnostic report usually selects pictures containing the lesion area for recording. The actual video shooting process is often recorded from near to far. The doctor will choose the video frame in which the lesion area is located in the center of the field of view, without interference or distortion, and can clearly reflect the characteristics of the lesion as the endoscopic picture in the diagnostic report. For the determination of the lesion area, segmentation, detection and other algorithms can be used for automatic selection. For example, a multi-label segmentation network (for example, UNet) can be trained to segment lesions such as polyps and adenomas in the intestine, and then determine whether the angle of the segmented area is reasonable, such as Figure 3As shown, firstly, one or more video frames containing lesions in the target video segment are identified as candidate video frames through a segmentation algorithm, and then each candidate video frame is segmented to determine the contour of the lesion area, and then the coordinates P of the center point of the lesion area (P can be determined by the intersection of the long and short paths) and the center point O of the candidate video frame are determined, and the distance OP between the center of the lesion area and the center point of the candidate video frame is calculated. A smaller threshold t is set, for example, t is 10 pixels, if OP < t, it is considered that the lesion area is located in the center of the field of view, if OP ≥ t, it is considered that the lesion area is not located in the center of the field of view, and finally all candidate video frames with the lesion area located in the center of the field of view are retained as target video frames, that is, the candidate endoscopic images are preliminarily determined. Where t can also be 2, 3, 4, 5, 6 and other numbers, and the target video frames selected by different thresholds have different effects.

[0136] It should be noted that the lesion area can be determined by identifying the contour of the lesion through a segmentation method, or by detecting the minimum bounding rectangle of the lesion area and determining the lesion position based on the minimum bounding rectangle. The detection method of the minimum bounding rectangle of the lesion area can use a deep learning detection network, for example, the YOLO (You Only Look Once) series, Faster-RCNN (Faster Region based CNN, fast regional convolutional neural network) series and other common detection networks. If the detection network is used, the center point P of the lesion area selects the center point of the lesion bounding rectangle, and then the distance OP between the center of the lesion area and the center point of the candidate video frame is also calculated.

[0137] Step S250, interference frame identification. The target video frame may contain interference information (for example, liquid reflection, artifacts, uneven lighting, blur, smoke, etc.), which is not suitable for archiving as an endoscopic image. Figure 4 As shown, the area of ​​interference information can be identified by segmentation. The area of ​​interference information will affect the analysis and diagnosis of the lesion area, so this video frame is not suitable as an endoscopic image. The specific identification method can be: threshold segmentation of the target video frame, inputting the target video frame after threshold segmentation into a trained deep learning model based on a convolutional neural network (e.g., UNet) for identification, and obtaining an identification result of whether the target video frame is an interference frame. If the target video frame is an interference frame, it is deleted. Otherwise, if the target video frame is not an interference frame, it does not need to be deleted.

[0138] It should be noted that the interference area can be identified by a segmentation method or by detecting the minimum bounding rectangle of the interference area. The detection method can use a deep learning detection network, such as the YOLO series, Faster-RCNN series and other common detection networks. Interference frames can also be identified by a classification algorithm, such as SVM and a deep learning algorithm (e.g., CNN).

[0139] Step S260, remove redundant frames and output the diagnosis report. There may be a certain degree of duplication between the target video frames after the interference frames are removed. It is unreasonable to retain all these frames as endoscopic images, because the space of the diagnosis report is limited and the graphic information should be as concise as possible. This problem can be solved by a redundant frame removal algorithm. Specifically, a de-redundancy operation based on image similarity (for example, histogram similarity) can be used, and finally a colonoscopy diagnosis report containing the endoscopic image is output.

[0140] It should be noted that redundant frames can also be removed by hashing methods (e.g., perceptual hashing), and image features in target video frames can be extracted by deep learning networks (e.g., CNN), and then the similarities between the features can be compared, and target video frames with similarities higher than a preset threshold can be filtered out as redundant frames.

[0141] The above-mentioned automatic filming method, based on the results of video data preprocessing, combines algorithms such as lens state recognition, shooting angle determination and interference frame recognition to automatically collect digestive endoscopy report pictures, has the advantages of high picture collection quality, does not need to distract the doctor's attention, and can adapt to various digestive endoscopy scenes. Among them, the data preprocessing operation, by cutting the black border of the endoscopic video and removing the non-information video frame, can help the subsequent automatic filming algorithm to extract more accurate feature information and improve the calculation speed. The lens state recognition algorithm determines whether the lens is in a static state to achieve preliminary screening of candidate video frames, and also combines the post-processing algorithm to remove the false positive of the lens state recognition result, so that the report picture extracted by the automatic filming algorithm is more accurate. The shooting angle determination algorithm determines the distance between the center of the target to be photographed and the center point of the image to select the video frame with the lesion area located in the center of the field of view as the report image, which is more in line with the actual clinical filming scene and avoids the deformation and distortion of the lesion area caused by different shooting angles. The interference frame recognition algorithm identifies the interference area in the video frame and selects the candidate frame with the lesion area located in the center of the field of view, which can make the final report picture quality higher. The final screening is done based on the redundant frame removal algorithm. The lesion is located in the center of the field of view and there is a certain amount of repetition in the high-quality video frame sequence. After the redundant frame removal operation, the final report image with diverse characteristics is determined.

[0142] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0143] Based on the same inventive concept, the embodiments of the present application also provide an endoscopic image generation method and an endoscopic system. The implementation solutions provided by the endoscopic image generation method and the endoscopic system are similar to the implementation solutions recorded in the above-mentioned endoscopic image generation system, so the specific limitations in one or more endoscopic image generation method and endoscopic system embodiments provided below can refer to the limitations of the endoscopic image generation system above, and will not be repeated here.

[0144] In one embodiment, Figure 6 As shown, a method for generating an endoscopic image is provided, the method comprising:

[0145] Step S310, obtaining an endoscopic video obtained by photographing the target part with an endoscope;

[0146] Step S320, determining a target video segment in the endoscopic video, and determining a target shooting state corresponding to the target video segment;

[0147] Step S330, when the target shooting state meets the preset state, determining a target video frame from the target video segment; the center position of the target video frame contains an abnormal area;

[0148] Step S340, obtaining a target endoscopic image of the target part according to the target video frame.

[0149] As an optional embodiment of the present application, the above step S320 includes:

[0150] Determining an original video segment from the endoscopic video based on a preset duration;

[0151] Removing the border area in the original video clip to obtain a video clip after the border is removed;

[0152] Invalid video frames are removed from the video segment after the border is removed to obtain the target video segment.

[0153] As an optional embodiment of the present application, the target video segment is at least one, and the above step S320 further includes:

[0154] Performing state recognition on each of the target video clips to obtain an original shooting state corresponding to each of the target video clips;

[0155] Obtaining an original state list according to at least one of the original shooting states;

[0156] Correcting the original status list to obtain a corrected status list;

[0157] The target shooting state corresponding to each target video segment is obtained according to the corrected state list.

[0158] As an optional embodiment of the present application, the step of identifying the state of each target video segment to obtain the original shooting state corresponding to each target video segment includes:

[0159] Determining at least two adjacent video frames from the target video segment;

[0160] Determine a displacement vector of a target point on the target part in the at least two adjacent video frames;

[0161] The displacement vectors are classified to obtain the original shooting state of the target video segment.

[0162] As an optional embodiment of the present application, the above step S330 includes:

[0163] When the target shooting state meets the preset state, determining a candidate video frame containing an abnormal area from the target video segment;

[0164] Determine the center point of the abnormal area as a first center point, and determine the center point of the candidate video frame as a second center point;

[0165] When the distance between the first center point and the second center point is less than a preset distance threshold, the candidate video frame is determined as the target video frame.

[0166] As an optional embodiment of the present application, the step of determining the center point of the abnormal area as the first center point includes:

[0167] The first center point is determined according to the outline of the abnormal area, or the first center point is determined according to the minimum circumscribed rectangle of the abnormal area.

[0168] As an optional embodiment of the present application, the above step S340 includes:

[0169] Performing threshold segmentation processing on the target video frame to obtain a video frame after threshold segmentation;

[0170] Performing interference frame recognition processing on the video frames after the threshold segmentation to obtain an initial endoscopic image of the target part;

[0171] The target endoscopic image of the target part is determined according to the initial endoscopic image.

[0172] As an optional embodiment of the present application, when there are at least two initial endoscopic images, the step of determining the target endoscopic image of the target part according to the initial endoscopic image includes:

[0173] Determining the histogram similarity between different initial endoscopic images;

[0174] According to the histogram similarity, redundant endoscopic images are removed from the initial endoscopic images to obtain the target endoscopic image of the target part.

[0175] In one embodiment, an endoscope system is provided, comprising: an endoscope, a display, a light source device, and an endoscope image generation system; wherein the endoscope image generation system comprises:

[0176] A video acquisition module is used to acquire an endoscopic video obtained by photographing a target part with an endoscope;

[0177] A state determination module, used to determine a target video segment in the endoscopic video, and determine a target shooting state corresponding to the target video segment;

[0178] A target selection module, used for determining a target video frame from the target video clip when the target shooting state meets a preset state; the center position of the target video frame contains an abnormal area;

[0179] The image generation module is used to obtain a target endoscopic image of the target part according to the target video frame.

[0180] The specific description of the endoscopic image generating system can be found in the relevant description of the endoscopic image generating system described in any of the above embodiments, which will not be described again here.

[0181] The above-mentioned endoscopic image generation method and endoscopic system obtain an endoscopic video obtained by photographing the target part with an endoscope, determine the target video segment in the endoscopic video, and determine the target shooting state corresponding to the target video segment. When the target shooting state meets the preset state, the target video frame is determined from the target video segment, and the target endoscopic image of the target part is obtained according to the target video frame. During the endoscopic examination, the target video segment and the shooting state corresponding to the target video segment can be automatically identified from the endoscopic video, the target video frame can be selected from the target video segment according to the shooting state, and the endoscopic image can be directly generated according to the target video frame. Since manual filming is not required, the efficiency of endoscopic examination is improved.

[0182] Moreover, the automatic generation of endoscopic images avoids the subjective factors existing in manual filming and improves the reliability of endoscopic image generation.

[0183] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an endoscopic image generation method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0184] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0185] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0186] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0187] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0188] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0189] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0190] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0191] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An endoscopic image generation system, characterized in that: The system comprises: A video acquisition module is used to acquire an endoscopic video obtained by photographing a target part with an endoscope; A state determination module, used to determine a target video segment in the endoscopic video, and determine a target shooting state corresponding to the target video segment; A target selection module, used for determining a target video frame from the target video clip when the target shooting state meets a preset state; the center position of the target video frame includes an abnormal area; The image generation module is used to obtain a target endoscopic image of the target part according to the target video frame.

2. The system according to claim 1, characterized in that The state determination module is also used to determine the original video segment from the endoscopic video based on a preset duration; remove the border area in the original video segment to obtain a video segment after the border is removed; remove invalid video frames from the video segment after the border is removed to obtain the target video segment.

3. The system according to claim 1, characterized in that There is at least one target video clip; the state determination module is further used to perform state identification on each of the target video clips to obtain the original shooting state corresponding to each of the target video clips; obtain an original state list based on at least one of the original shooting states; perform correction processing on the original state list to obtain a corrected state list; and obtain the target shooting state corresponding to each of the target video clips based on the corrected state list.

4. The system according to claim 3, characterized in that The state determination module is also used to determine at least two adjacent video frames from the target video segment; determine the displacement vector of the target point on the target part in the at least two adjacent video frames; and classify the displacement vector to obtain the original shooting state of the target video segment.

5. The system according to claim 1, characterized in that The target selection module is also used to determine, when the target shooting state meets the preset state, a candidate video frame containing an abnormal area from the target video segment; determine the center point of the abnormal area as the first center point, and determine the center point of the candidate video frame as the second center point; and determine the candidate video frame as the target video frame when the distance between the first center point and the second center point is less than a preset distance threshold.

6. The system according to claim 5, characterized in that The target selection module is further used to determine the first center point according to the outline of the abnormal area, or to determine the first center point according to the minimum circumscribed rectangle of the abnormal area.

7. The system according to claim 1, characterized in that The image generation module is also used to perform threshold segmentation processing on the target video frame to obtain a video frame after threshold segmentation; perform interference frame recognition processing on the video frame after threshold segmentation to obtain an initial endoscopic image of the target part; and determine the target endoscopic image of the target part based on the initial endoscopic image.

8. The system according to claim 7, characterized in that When there are at least two initial endoscopic images, the image generation module is further used to determine the histogram similarity between different initial endoscopic images; based on the histogram similarity, redundant endoscopic images are removed from the initial endoscopic images to obtain the target endoscopic image of the target part.

9. A method for generating an endoscopic image, characterized in that: The method comprises: Acquire an endoscopic video obtained by photographing the target area with an endoscope; Determining a target video segment in the endoscopic video, and determining a target shooting state corresponding to the target video segment; In the case where the target shooting state meets the preset state, determining a target video frame from the target video segment; the center position of the target video frame includes an abnormal area; A target endoscopic image of the target part is obtained according to the target video frame.

10. The method according to claim 9, characterized in that When the target shooting state meets the preset state, determining the target video frame from the target video segment includes: When the target shooting state meets the preset state, determining a candidate video frame containing an abnormal area from the target video segment; Determine the center point of the abnormal area as a first center point, and determine the center point of the candidate video frame as a second center point; When the distance between the first center point and the second center point is less than a preset distance threshold, the candidate video frame is determined as the target video frame.

11. An endoscope system, characterized in that: The invention comprises an endoscope, a display, a light source device and an endoscopic image generating system as described in any one of claims 1 to 8.

Images