Video processing method, device, equipment and endoscope system
Through video processing methods, clinical sites are determined and fuzzy are performed for irrelevant parts taken in vitro during endoscopy, which solves the privacy risk problem and achieves a balance between privacy protection and examination and analysis.
Patent Information
- Application Number
- CN202311099967.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-29
- Publication Date
- 2025-03-04
AI Technical Summary
During the endoscopy, the endoscopy may capture irrelevant parts outside the subject, resulting in an increase in user privacy risks.
Through the video processing method, the clinical site is determined according to the type of the endoscope, the in vitro video frame is detected in the to-process video, and the target area associated with the clinical site is determined in the in vitro video frame, and the area outside the target area is fuzzy.
It effectively reduces the excessive exposure of the private parts of the examined subject in vitro, protects the privacy of the examined subject, and does not affect the examination and analysis of medical staff.
Smart Images
Figure CN119559089A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of privacy protection for endoscopic examinations, and in particular to a video processing method, apparatus, computer equipment, endoscopic system, storage medium, and computer program product. Background Art
[0002] Endoscopic examination generally uses an endoscope to photograph the internal conditions of the subject to be examined, so as to determine whether there is any abnormality in the subject's body. During the process of endoscopic examination, the endoscope sometimes photographs the external parts of the subject that are not related to the examination, which may lead to user privacy risks.
[0003] Therefore, a solution is needed to reduce user privacy risks during endoscopic examination. Summary of the invention
[0004] Based on this, it is necessary to provide a video processing method, device, computer equipment, endoscope system, storage medium and computer program product to address the above technical problems.
[0005] The present application provides a video processing method, the method comprising:
[0006] Based on the type of endoscope, determine the clinical site to be tested;
[0007] Determining in the video to be processed an in vitro video frame shot of the object to be inspected in vitro; the video to be processed is a video shot during the process of inspecting the object to be inspected using the endoscope;
[0008] determining a target region associated with the clinical site in the in vitro video frame;
[0009] The area outside the target area in the in vitro video frame is blurred.
[0010] In one embodiment, determining the target area associated with the clinical site in the in vitro video frame includes: obtaining a detection scheme corresponding to the clinical site; different clinical sites have different corresponding detection schemes; and determining the target area associated with the clinical site in the in vitro video frame using the detection scheme.
[0011] In one embodiment, the determining of the clinical site to be detected based on the type of endoscope includes: determining the physiological site involved in the endoscopic operation process based on the type of endoscope; and determining the site of the physiological site exposed to the body as the clinical site.
[0012] In one embodiment, determining the target area associated with the clinical site in the in vitro video frame includes: determining the area where the clinical site is located in the in vitro video frame; determining a region expansion ratio based on the privacy level of the clinical site; the region expansion ratio is negatively correlated with the privacy level; and expanding the area based on the region expansion ratio to obtain the target area.
[0013] In one embodiment, the privacy level of the clinical site is positively correlated with the amount of private information contained in the clinical site.
[0014] In one embodiment, the determining of the clinical site to be inspected based on the type of endoscope includes: when the endoscope is a hard endoscope, determining that the clinical site includes a surgical incision;
[0015] Determining the target area associated with the clinical site in the in vitro video frame includes: identifying the incision area of the surgical incision and the area covered by iodine solution contained in the in vitro video frame; and determining the target area based on the incision area and the area covered by iodine solution.
[0016] In one embodiment, the method further comprises: if the in vitro video frame does not include the target area associated with the clinical site, blurring the entire area of the in vitro video frame.
[0017] In one embodiment, the method is applied in a process of inspecting the object under inspection using the endoscope.
[0018] In one embodiment, determining the in vitro video frames shot of the inspected object outside the body in the video to be processed includes: extracting video frames at intervals from the video to be processed to obtain multiple pre-inspection video frames and performing in vitro detection; when two adjacent pre-inspection video frames are detected to be an in vitro video frame and an in vivo video frame, respectively, determining the video frames in the video to be processed that are located between the two adjacent pre-inspection video frames to obtain multiple post-inspection video frames; performing in vitro detection on the multiple post-inspection video frames to determine the in vitro video frames.
[0019] The present application provides a video processing device, the device comprising:
[0020] A clinical site determination module is used to determine the clinical site to be detected based on the type of endoscope;
[0021] An in vitro frame detection module, used to determine in the video to be processed an in vitro video frame shot of the object to be inspected in vitro; the video to be processed is a video shot during the inspection of the object to be inspected using the endoscope;
[0022] a region detection module, for determining a target region associated with the clinical site in the in vitro video frame;
[0023] A blur processing module is used to blur the area outside the target area in the in vitro video frame.
[0024] The present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the above method.
[0025] The present application provides an endoscope system, comprising: a display, an endoscope, a light source device, and the computer device as described in the above embodiment.
[0026] The present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to execute the above method.
[0027] The present application provides a computer program product, on which a computer program is stored, and the computer program is executed by a processor to perform the above method.
[0028] The present application can determine the clinical part to be detected this time according to the type of endoscope. After obtaining the video to be processed formed by the endoscope inspecting the object to be inspected, the video frame of the video to be processed can be detected in vitro to determine whether the video frame is taken when the endoscope is outside the body. If so, it is used as an in vitro video frame. Then, it can be detected whether the in vitro video frame includes a target area associated with the clinical part. If so, the area except the target area in the in vitro video frame is blurred, which can avoid excessive exposure of the private parts of the object to be inspected outside the body and protect the privacy of the object to be inspected. In addition, when the target area associated with the clinical part is not blurred, the solution provided by the present application does not affect the inspection and analysis of medical staff, and guarantees the normal endoscopic inspection needs of medical staff. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 is a schematic flow chart of a video processing method in one embodiment;
[0030] Figure 2 is a schematic diagram of in vitro detection in one embodiment;
[0031] Figure 3 is a flow chart of a video processing method in another embodiment;
[0032] Figure 4 is a structural block diagram of a video processing device in one embodiment;
[0033] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0034] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0035] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.
[0036] The video processing method provided in this application can be used for privacy protection in endoscopic examinations to avoid excessive exposure of the private parts of the subject's body. In the process of using an endoscope to examine the subject, the video processing method provided in this application can be used. Compared with the desensitization method after the examination is completed, the risk of leakage of the subject's privacy information can be further reduced. The method can be executed by a computer device, including Figure 1 Steps shown.
[0037] Step S101, based on the type of endoscope, determine the clinical site to be detected this time.
[0038] Endoscopes can be divided into soft endoscopes and hard endoscopes according to the material of the endoscope and whether it can be bent or twisted. Furthermore, according to the different parts to be examined by the soft endoscope, the soft endoscope can be further divided into colonoscopes and gastroscopes.
[0039] An endoscope can enter the body of the subject through the natural cavity or surgical incision of the subject, thereby reaching the desired site (which can be called the target site). In the process of the endoscope reaching the target site inside the body from the outside of the body, the natural cavity or surgical incision passed through can be called the physiological site involved in the endoscopic operation process. The physiological sites involved in the endoscopic operation process can be different for different types of endoscopes; therefore, based on the type of endoscope, the physiological sites involved in the endoscopic operation process can be determined.
[0040] For example, in a colonoscopy, it is generally necessary to enter from the subject's anus to the rectum, and then pass through the sigmoid colon, descending colon, transverse colon, ascending colon to reach the ileocecal region, enter the terminal small intestine at the ileocecal valve, and then gradually advance to observe the above-mentioned colorectal mucosa in turn; the physiological parts involved in this operation include the anus, rectum, sigmoid colon, descending colon, transverse colon, ascending colon, ileocecal region, terminal small intestine and colorectum.
[0041] Among them, among the physiological parts involved in the endoscopic operation process, in addition to the parts inside the body of the subject that can be used for analysis by medical staff, the parts exposed outside the body of the subject can also be used by medical staff for inspection and analysis. For example, in the above-mentioned colonoscopy, the anus is a part exposed outside the body of the subject, and this part can also be used by medical staff for inspection and analysis.
[0042] When the endoscope is outside the body, it will not only capture the above-mentioned exposed parts that can be used for medical staff to examine and analyze, but it may also capture other parts of the subject that are exposed outside the body and are not related to the medical staff's examination and analysis, resulting in the privacy of the subject being leaked. Therefore, according to the type of endoscope, the exposed parts can be determined from the physiological parts involved in the endoscope operation process to obtain the clinical parts to be tested this time; for example, the clinical parts of a colonoscope can be the anus, the clinical parts of a gastroscope and oral scope can be the mouth, the clinical parts of a sinusoscope can be the nostrils, and the clinical parts of a urethra cystoscope can be the genitals.
[0043] Step S102: determining in vitro video frames shot of the subject in vitro in the video to be processed.
[0044] In the process of using an endoscope to inspect an object to be inspected, the video captured by the endoscope can be called a to-be-processed video. The present application can perform in vitro detection on the video frames in the to-be-processed video, that is, determine whether the video frame is a video frame captured when the endoscope is outside the body of the object to be inspected, and if so, use the video frame as an in vitro video frame.
[0045] Determining whether a video frame is a video frame taken when the endoscope is outside the body of the object to be inspected can be achieved using an algorithm. The algorithm is mainly used to identify whether a video frame is an in vitro video frame, which belongs to the classification of video frames, so the algorithm can be called a classification algorithm. In some embodiments, the classification algorithm can be constructed based on a machine learning method, and the machine learning method can include but is not limited to a deep learning method and a classifier learning method based on image features. The classifier can be a support vector machine (SVM), Adabooting, etc. In other embodiments, other methods can also be used to implement the classification algorithm, such as analyzing the human body features and environmental features of the image, and implementing image classification based on these features.
[0046] Regarding the deep learning method, colonoscopy is used as an example: the video taken during the colonoscopy can be used as a video sample, and the video frames in the video sample can be used as video frame samples. The video frame samples taken by the endoscope outside the body are labeled as in vitro video frames, and the video frame samples taken by the endoscope inside the body are labeled as in vivo videos. Therefore, the video frame samples carry the labeled categories, and the video frame samples carrying the labeled categories are used for deep learning to obtain a classification algorithm.
[0047] Regarding the classifier learning method based on image features, colonoscopy is used as an example to introduce: after obtaining video frame samples carrying labeled categories in the above method, feature extraction is performed on the video frame samples to obtain the image features of each video frame sample, and the image features are associated with the labeled categories carried by the video frame samples, and classifier learning is performed in sequence to obtain a classification algorithm.
[0048] Whether the video frame is a video frame taken when the endoscope is outside the body of the object to be inspected can also be determined based on the time when the computer device receives the endoscope entry instruction, which can be issued by the endoscope operator. Specifically, when the endoscope operator determines that the endoscope has entered the body during the operation of the endoscope, the endoscope operator can input the endoscope entry instruction into the computer device by clicking a preset button or the like; when the computer device receives the endoscope entry instruction, the receiving time is recorded, and the video frame taken before the receiving time is used as the external video frame in the video to be processed.
[0049] Step S103, determining a target area associated with a clinical site in the in vitro video frame.
[0050] After the in vitro video frame is determined, the clinical site detection can be performed on the in vitro video frame to determine whether the in vitro video frame contains the area where the clinical site is located. If so, based on the area where the clinical site is located, a target area associated with the clinical site can be determined in the in vitro video frame. For example, the area where the clinical site is located is used as the target area, and for another example, the area where the clinical site is located is fine-tuned to obtain the target area.
[0051] Step S104, performing blur processing on the area outside the target area in the in vitro video frame.
[0052] After determining the target area of the in-vitro video frame, the area other than the target area in the in-vitro video frame can be blurred, while the target area is not blurred. The embodiment of the present application does not make too many restrictions on the blurring method, which can be determined according to the actual application. For example, blurring can be achieved by coding or the content of the area outside the target area can be removed or replaced with other content to achieve blurring.
[0053] The above-mentioned video processing method can determine the clinical part to be detected this time according to the type of endoscope. After obtaining the video to be processed formed by the endoscope inspecting the object to be inspected, the video frame of the video to be processed can be subjected to in vitro detection to determine whether the video frame is taken when the endoscope is outside the body. If so, it is used as an in vitro video frame. Then, it can be detected whether the in vitro video frame includes a target area associated with the clinical part. If so, the area except the target area in the in vitro video frame is blurred, which can avoid excessive exposure of the private parts of the object to be inspected outside the body and protect the privacy of the object to be inspected. Moreover, when the target area associated with the clinical part is not blurred, the solution provided by the present application does not affect the inspection and analysis of medical staff, and guarantees the normal endoscopic inspection needs of medical staff.
[0054] In one embodiment, determining the target region associated with the clinical site in the in vitro video frame in step S103 includes: obtaining a detection scheme corresponding to the clinical site; and determining the target region associated with the clinical site in the in vitro video frame using the detection scheme.
[0055] Different clinical sites have different detection schemes. The detection scheme can be used to detect whether the image includes the area where the clinical site is located. If so, the target area associated with the clinical site can be obtained based on the area where the detection scheme output is located, for example, the area where the clinical site is located is used as the target area, and for example, the area where the clinical site is located is fine-tuned to obtain the target area. In a possible way, the detection scheme can also be used to fine-tune the area where the clinical site is located to obtain the target area. Among them, the embodiment of the present application does not make too many restrictions on the specific detection scheme corresponding to each clinical site, and can be set according to actual needs. For example, in some optional embodiments, the detection scheme can be a recognition model constructed based on a machine learning method, and the machine learning method includes but is not limited to: a deep learning method, a classifier learning method based on image features. At this time, for different clinical sites, it corresponds to different recognition models. In other optional embodiments, the detection scheme can also be a specific detection algorithm. That is, the corresponding detection algorithm can be constructed for the features of different clinical sites in advance, so as to realize the feature analysis and recognition of each different clinical site. In some optional embodiments, the recognition model can also be set for some clinical sites, and the detection algorithm can be set for some clinical sites.
[0056] When the detection scheme includes a recognition model, for common clinical sites such as the anus and mouth, deep learning can be used because of the sufficient training samples. For uncommon sites, there are fewer training samples, so feature design can be performed on the image to obtain image features, such as the color and texture of the clinical site, and then combined with a classifier for classification and recognition. The classifier can be a support vector machine (SVM), Adabooting, etc. As the number of training samples for uncommon sites increases, deep learning can be used for recognition.
[0057] Regarding the training samples used in the deep learning method, a colonoscopy and a detection scheme for detecting whether an image includes the area where the clinical site is located are used as an example: the video taken during the colonoscopy examination can be used as a video sample, and the video frames in the video sample can be used as video frame samples. The video frame samples are marked with the area where the clinical site is located. Thus, the video frame samples carry the annotation boxes, and deep learning is performed based on the video frame samples carrying the annotation boxes to obtain a detection scheme.
[0058] As an embodiment of the present application, after the area where the clinical site is located is determined in the in vitro video frame, the area where the clinical site is located can be expanded according to the set area expansion ratio in consideration of the detection accuracy, so as to avoid affecting the inspection and analysis of medical staff. That is, based on the identified clinical site area, the boundary of the area is expanded outward until the size of the expanded area reaches the set area expansion ratio. Among them, the set area expansion ratio can be set according to actual needs. For example, in some embodiments, it can be set to 10%. At this time, the boundary of the identified clinical site area can be expanded outward until the size of the expanded area reaches 10% of the size of the identified clinical site area.
[0059] In different types of endoscopic examinations, the privacy of clinical sites varies. For example, the privacy of the anus in a colonoscopy may be higher than that of the oral cavity in a gastroscopy. Therefore, if the same set area expansion ratio is always used for expansion, the effect of protecting the privacy of the subject may be limited to a certain extent.
[0060] Based on this, in order to improve the privacy protection effect, the present application also provides an embodiment, in which the target area associated with the clinical site is determined in the in vitro video frame in step S103, including: determining the area where the clinical site is located in the in vitro video frame; determining the area expansion ratio based on the privacy degree of the clinical site; and expanding the area based on the area expansion ratio to obtain the target area.
[0061] In this embodiment, after the area where the clinical site is located is determined in the in vitro video frame, the area where the clinical site is located may not be directly used as the target area, but the area where the clinical site is located may be fine-tuned. During the fine-tuning process, the area where the clinical site is located is expanded according to the regional expansion ratio determined according to the privacy of the clinical site to obtain the target area. Among them, the regional expansion ratio is negatively correlated with the privacy. That is, the higher the privacy of the clinical site, the smaller the expansion degree of the target area relative to the area where the clinical site is located, and the lower the privacy of the clinical site, the greater the expansion degree of the target area relative to the area where the clinical site is located.
[0062] Among them, the privacy degree of the clinical site is positively correlated with the amount of privacy information contained in the clinical site. For example, compared with the oral cavity, the anus contains a larger amount of privacy information, and the privacy degree of the anus is greater than that of the oral cavity.
[0063] In one embodiment, the step S101 determines the clinical site to be inspected based on the type of endoscope, including: when the endoscope is a hard endoscope, determining that the clinical site includes the surgical incision; the step S103 determines the target area associated with the clinical site in the in vitro video frame, including: identifying the area covered by iodine solution in the in vitro video frame; and determining the target area based on the area covered by iodine solution.
[0064] Hard endoscopes are generally used for minimally invasive surgery, and enter the body through the surgical incision of the object being inspected. Therefore, when the endoscope is a hard endoscope, the clinical site includes the surgical incision. The characteristics of the surgical incision are sometimes difficult to identify, but considering that the object being inspected is generally disinfected with iodine solution during surgery, there is generally iodine solution around the surgical incision. The color of the iodine solution is quite different from the color of other things in the video frame (such as human skin), so the characteristics of the iodine solution are easier to identify than the characteristics of the surgical incision.
[0065] Therefore, when iodine solution is applied around the surgical blade, it is possible to identify whether the in vitro video frame includes the area covered by iodine solution, when the area covered by iodine solution is identified. Since iodine solution is generally applied around the surgical blade, therefore, the blade area can be defaulted to be within the iodine solution covered area. Then, the area covered by iodine solution can be used as the area of the clinical site, thereby determining the target area.
[0066] In this embodiment, when iodine liquid is applied around the surgical incision, the area covered by the iodine liquid is preferentially identified to improve the recognition accuracy; and when the area covered by the iodine liquid is identified, the area covered by the iodine liquid can be directly used as the area where the clinical site is located, without the need to identify the incision area again, thereby improving the recognition efficiency.
[0067] In addition, when iodine solution is applied around the surgical incision, the incision area and the area covered by the iodine solution can also be identified together to obtain the incision area and the area covered by the iodine solution, thereby determining the target area. Therefore, the present application also provides an embodiment, in step S101, based on the type of endoscope, determining the clinical site to be detected this time, including: when the endoscope is a hard endoscope, determining that the clinical site includes the surgical incision; in step S103, determining the target area associated with the clinical site in the in vitro video frame, including: identifying the incision area of the surgical incision contained in the in vitro video frame and the area covered by the iodine solution; determining the target area according to the incision area and the area covered by the iodine solution.
[0068] In step S102 of the present application, the in vitro frequency frames shot outside the body of the inspected object can be determined in the video to be processed, and each video frame of the video to be processed can be subjected to in vitro detection. Considering that the video to be processed is a video of the process of the endoscope entering the body from outside the body of the inspected object, it has a certain continuity. Therefore, the video to be processed can be subjected to in vitro detection frame by frame in chronological order. When an in vivo video frame is detected, it can be determined that the endoscope begins to enter the body, and the subsequent video frames are also likely to be in vivo video frames. At this time, the in vitro detection can be stopped.
[0069] In order to further improve the efficiency of in vitro detection, the present application also provides an embodiment, in which step S102 determines in vitro video frames shot of the object to be inspected in vitro in the video to be processed, including: extracting video frames at intervals from the video to be processed to obtain multiple pre-inspection video frames and performing in vitro detection; when two adjacent pre-inspection video frames are detected to be an in vitro video frame and an in vivo video frame, respectively, determining a video frame in the video to be processed that is located between the two adjacent pre-inspection video frames to obtain multiple post-inspection video frames; performing in vitro detection on the multiple post-inspection video frames to determine the in vitro video frames.
[0070] Reference Figure 2 , exemplarily, the video to be processed includes N video frames, which are named as the first video frame, the second video frame, the third video frame, ..., the Nth video frame in chronological order. The N video frames can be sorted in chronological order to obtain a video frame sequence, and the video frame sequence can be extracted at intervals. Specifically, a video frame can be extracted every 10 frames, thereby extracting the i-th video frame, the i+10-th video frame, the i+20-th video frame, ..., the i+10×k-th video frame at intervals. Among them, i is the first extracted video frame in the interval extraction, and k is an integer.
[0071] First, perform in vitro detection on the video frames extracted at intervals to determine whether each video frame is an in vitro video frame or an in vivo video frame. If the in vitro detection result indicates that the i+50th video frame is an in vitro video frame and the i+60th video frame is an in vivo video frame, it means that during the process from the i+50th video frame to the i+60th video frame, the endoscope enters the body from the outside of the body. At this time, each video frame between the i+50th video frame and the i+60th video frame can be obtained, and each video frame can be tested in vitro to determine the in vitro video frame between the i+50th video frame and the i+60th video frame.
[0072] It can be seen that the i-th video frame, the i+10th video frame, the i+20th video frame, ..., the i+10×kth video frame extracted at intervals are first subjected to in vitro detection compared to each video frame between the i+50th video frame to the i+60th video frame. Therefore, the i-th video frame, the i+10th video frame, the i+20th video frame, ..., the i+10×kth video frame extracted at intervals can be called pre-inspection video frames, and each video frame between the i+50th video frame to the i+60th video frame can be called post-inspection video frames.
[0073] In one embodiment, the method provided by the present application further includes: if the in vitro video frame does not include the target area associated with the clinical site, blurring the entire area of the in vitro video frame.
[0074] In order to better understand the video processing method of the present application, an embodiment is provided below for introduction:
[0075] Reference Figure 3 After the endoscope is inserted into a specific machine, the type of the endoscope can be identified (step S301); based on the type of the endoscope, the clinical site to be detected this time and the detection algorithm corresponding to the clinical site are determined.
[0076] After obtaining the video to be processed captured during the process of endoscope inspection of the object to be inspected, the process may proceed to step S302 to perform an in vitro detection on the video frame to be processed to determine whether the video frame is an in vitro video frame.
[0077] If the video frame is an in vitro video frame, the process proceeds to step S303 to identify whether the in vitro video frame includes a target area associated with a clinical site.
[0078] If the target area is not identified, the process proceeds to step S304 to perform blur processing on the entire area of the in vitro video frame.
[0079] If the target area is identified, the process proceeds to step S305 to perform blur processing on the area outside the target area of the in vitro video frame.
[0080] Among them, the specific process of identifying the target area can be: using the detection scheme corresponding to the clinical site, determining the area where the clinical site is located in the in vitro video frame, and expanding the area where the clinical site is located based on the regional expansion ratio determined based on the privacy of the clinical site to obtain the target area.
[0081] If the video frame is an in-vivo video frame, the process enters the in-vivo detection mode in step S306 .
[0082] In this embodiment, after determining the in vitro video frame, it can be detected whether the in vitro video frame includes a target area associated with the clinical site. If so, the area except the target area in the in vitro video frame is blurred, so as to avoid excessive exposure of the in vitro private parts of the subject and protect the privacy of the subject. In addition, the target area associated with the clinical site is not blurred, which does not affect the inspection and analysis of medical staff and ensures the normal endoscopic inspection needs of medical staff.
[0083] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0084] In one embodiment, Figure 4 As shown, a video processing device is provided, comprising:
[0085] A clinical site determination module 401 is used to determine the clinical site to be detected based on the type of endoscope;
[0086] The in-vitro frame detection module 402 is used to determine in the video to be processed an in-vitro video frame shot of the subject in-vitro; the video to be processed is a video shot during the inspection of the subject using the endoscope;
[0087] A region detection module 403, for determining a target region associated with the clinical site in the in vitro video frame;
[0088] The blur processing module 404 is used to perform blur processing on the area outside the target area in the in vitro video frame.
[0089] In one embodiment, the region detection module 403 is further used to obtain a detection scheme corresponding to the clinical site; different clinical sites have different corresponding detection schemes; and using the detection scheme, a target region associated with the clinical site is determined in the in vitro video frame.
[0090] In one embodiment, the clinical site determination module 401 is further used to determine the physiological site involved in the endoscopic operation process based on the type of endoscope; and determine the site of the physiological site exposed to the body as the clinical site.
[0091] In one embodiment, the region detection module 403 is further used to determine the region where the clinical site is located in the in vitro video frame; determine the region expansion ratio based on the privacy level of the clinical site; the region expansion ratio is negatively correlated with the privacy level; based on the region expansion ratio, expand the region to obtain the target region.
[0092] In one embodiment, the privacy level of the clinical site is positively correlated with the amount of private information contained in the clinical site.
[0093] In one embodiment, the clinical site determination module 401 is also used to determine that the clinical site includes a surgical incision when the endoscope is a hard endoscope; the area detection module 403 is also used to identify the incision area of the surgical incision contained in the in vitro video frame and the area covered by iodine solution; and determine the target area based on the incision area and the area covered by iodine solution.
[0094] In one embodiment, the blur processing module 404 is further configured to perform blur processing on the entire area of the in vitro video frame if the in vitro video frame does not include the target area associated with the clinical site.
[0095] In one embodiment, the device is used in a process of inspecting the object under inspection using the endoscope.
[0096] In one embodiment, the in vitro frame detection module 402 is also used to extract video frames from the video to be processed at intervals to obtain multiple pre-inspection video frames and perform in vitro detection; when it is detected that two adjacent pre-inspection video frames are respectively an in vitro video frame and an in vivo video frame, determine the video frame in the video to be processed that is located between the two adjacent pre-inspection video frames to obtain multiple post-inspection video frames; perform in vitro detection on the multiple post-inspection video frames to determine the in vitro video frame.
[0097] For the specific definition of the video processing device, please refer to the definition of the video processing method above, which will not be repeated here. Each module in the above video processing device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.
[0098] In one embodiment, a computer device is provided, whose internal structure diagram can be as follows: Figure 5 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video processing data. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer device also includes an input and output interface, which is a connection circuit for exchanging information between the processor and the external device. They are connected to the processor through a bus, referred to as an I / O interface. When the computer program is executed by the processor, a video processing method is implemented.
[0099] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0100] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned various method embodiments when executing the computer program. In some embodiments, the computer device may be an image processing device or an AI box or other device with image or video processing capabilities.
[0101] In one embodiment, an endoscope system is provided. The endoscope system includes: a display, an endoscope, a light source device, and a computer device as described in the above embodiment.
[0102] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0103] In one embodiment, a computer program product is provided, on which a computer program is stored, and the computer program is used by a processor to execute the steps in the above-mentioned various method embodiments.
[0104] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0105] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0106] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0107] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A video processing method, characterized in that: The method comprises: Based on the type of endoscope, determine the clinical site to be tested; Determining in the video to be processed an in vitro video frame shot of the object to be inspected in vitro; the video to be processed is a video shot during the process of inspecting the object to be inspected using the endoscope; determining a target region associated with the clinical site in the in vitro video frame; The area outside the target area in the in vitro video frame is blurred.
2. The method according to claim 1, characterized in that Determining the target area associated with the clinical site in the in vitro video frame includes: Obtaining a detection scheme corresponding to the clinical site; different detection schemes correspond to different clinical sites; Using the detection scheme, a target region associated with the clinical site is determined in the in vitro video frame.
3. The method according to claim 1, characterized in that Based on the type of endoscope, the clinical site to be tested is determined, including: Based on the type of endoscope, determining the physiological part involved in the endoscopic operation process; Among the physiological sites, the site exposed to the outside of the body is determined as a clinical site.
4. The method according to claim 1, characterized in that: Determining the target area associated with the clinical site in the in vitro video frame includes: Determining the region where the clinical site is located in the in vitro video frame; Based on the privacy degree of the clinical site, determining a regional expansion ratio; the regional expansion ratio is negatively correlated with the privacy degree; Based on the area expansion ratio, the area is expanded to obtain the target area.
5. The method according to claim 4, characterized in that The privacy degree of the clinical site is positively correlated with the amount of privacy information contained in the clinical site.
6. The method according to any one of claims 1 to 5, characterized in that: Based on the type of endoscope, the clinical site to be tested is determined, including: When the endoscope is a hard endoscope, determining that the clinical site includes a surgical incision; Determining the target area associated with the clinical site in the in vitro video frame includes: Identifying the surgical incision area and the area covered by iodine solution contained in the in vitro video frame; The target area is determined according to the incision area and the area covered by the iodine solution.
7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: If the in vitro video frame does not include the target area associated with the clinical site, blurring is performed on the entire area of the in vitro video frame.
8. The method according to any one of claims 1 to 5, characterized in that: The method is applied in the process of inspecting the object by using the endoscope.
9. The method according to any one of claims 1 to 5, characterized in that: The step of determining, in the video to be processed, an in vitro video frame shot of the subject in vitro includes: Extracting video frames from the video to be processed at intervals to obtain a plurality of pre-test video frames and performing in vitro testing; When it is detected that two adjacent first-checked video frames are respectively an in-vitro video frame and an in-vitro video frame, a video frame located between the two adjacent first-checked video frames in the video to be processed is determined to obtain a plurality of post-checked video frames; An in vitro detection is performed on the multiple post-detection video frames to determine the in vitro video frame.
10. A video processing device, characterized in that: The device comprises: A clinical site determination module is used to determine the clinical site to be detected based on the type of endoscope; An in vitro frame detection module, used to determine in the video to be processed an in vitro video frame shot of the object to be inspected in vitro; the video to be processed is a video shot during the inspection of the object to be inspected using the endoscope; a region detection module, for determining a target region associated with the clinical site in the in vitro video frame; A blur processing module is used to blur the area outside the target area in the in vitro video frame.
11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.
12. An endoscope system, characterized in that: include: A display, an endoscope, a light source device, and a computer device as claimed in claim 11.