A quality control method, device and computer equipment for medical endoscopy

Abstract

The application relates to a quality control method, device and computer equipment for medical endoscopy. The method comprises the following steps: acquiring a video frame image of endoscopy, extracting a predicted skeleton image of at least one target region from the video frame image, matching the predicted skeleton image with a pre-stored template skeleton image, calculating an overlap rate of the predicted skeleton image and the template skeleton image, determining the number of complete predicted skeleton images according to the overlap rate, obtaining the relative displacement of the predicted skeleton image from continuous video frame images, comparing the relative displacement with a pre-set relative displacement template, determining the number of specific motions, and judging whether the endoscopy process is complete according to the number of complete predicted skeleton images and the number of specific motions. The method can realize quality monitoring of the completeness of endoscopy of a detected motion part by double statistics of the number of complete predicted skeleton images and the number of specific motions.

Description

Technical Field

[0001] This application relates to the field of video image processing technology, and in particular to a quality control method, apparatus and computer equipment for medical endoscopic examination. Background Technology

[0002] A medical endoscope is a diagnostic instrument composed of an image sensor, a light source, optical lenses, and other physical components. It can enter the human body through various openings, such as the nose and mouth, to photograph internal tissues and organs. Doctors can use the endoscope to penetrate deep into different tissues and record images, preserving information about lesions. Therefore, medical endoscopes play a crucial role in pathological diagnosis. However, when using a medical endoscope to examine the human body, due to the complexity of the internal structure and the numerous areas requiring examination, some areas may be missed in the image, leading to incomplete diagnosis. This can result in undiagnosed lesions and hinder patient treatment.

[0003] Most existing endoscopic quality control technologies classify the current frame to determine the location of the current frame, and then determine whether all image recognition results cover the preset location to be identified. This method performs overall location recognition on a single frame image, which cannot handle scenarios where it is necessary to check whether the movement of the same part of the patient is normal. Summary of the Invention

[0004] Therefore, it is necessary to provide a quality control method, device, and computer equipment for medical endoscopy to address the aforementioned technical problems, which can monitor the examination results of endoscopy on moving parts.

[0005] In a first aspect, this application provides a quality control method for medical endoscopy, the quality control method for medical endoscopy comprising:

[0006] Acquire video frame images of an endoscopic examination, and extract a predicted skeleton image of at least one target region from the video frame images;

[0007] The predicted skeleton image is matched with a pre-stored template skeleton image, the overlap rate between the predicted skeleton image and the template skeleton image is calculated, and the number of complete predicted skeleton images is determined based on the overlap rate.

[0008] The relative displacement of the predicted skeleton image is obtained from the consecutive video frame images, and the relative displacement is compared with a preset relative displacement template to determine the number of specific movements;

[0009] The completeness of the endoscopic examination process is determined based on the number of complete predicted skeleton images and the number of specific movements.

[0010] In one embodiment, matching the predicted skeleton image with a pre-stored template skeleton image includes:

[0011] The predicted skeleton image is normalized at the scale level.

[0012] The processed predicted skeleton image is matched with the pre-stored template skeleton image.

[0013] In one embodiment, matching the processed predicted skeleton image with a pre-stored template skeleton image includes:

[0014] Calculate the matching similarity between the processed predicted skeleton image and the pre-stored template skeleton image;

[0015] If the matching similarity is greater than the matching threshold, the match is considered successful.

[0016] In one embodiment, calculating the overlap rate between the predicted skeleton image and the template skeleton image, and determining the number of complete predicted skeleton images based on the overlap rate, includes:

[0017] Calculate the weighted Euclidean distance between the predicted points of the predicted skeleton image and the template keypoints of the template skeleton image;

[0018] If the weighted Euclidean distance is less than the prediction threshold, then the predicted point is determined to be a correct prediction point;

[0019] The overlap rate between the predicted skeleton image and the template skeleton image is calculated based on the number of correctly predicted points and the total number of predicted points.

[0020] If the overlap rate is greater than the integrity threshold, the predicted skeleton image is determined to be complete;

[0021] Count the number of complete predicted skeleton images.

[0022] In one embodiment, obtaining the relative displacement of the predicted skeleton image from consecutive video frame images, comparing the relative displacement with a preset relative displacement template, and determining the number of specific movements includes:

[0023] Obtain the current predicted skeleton image and the historical predicted skeleton images from the consecutive video frame images;

[0024] The relative displacement of the predicted skeleton image is obtained based on the current predicted skeleton image and the historical predicted skeleton images;

[0025] The displacement similarity is obtained by comparing the relative displacement with a preset relative displacement template;

[0026] If the displacement similarity is greater than the displacement threshold, it is determined that a specific motion of the target area has been captured;

[0027] Count the number of the specific movements.

[0028] In one embodiment, determining whether the endoscopic examination process is complete based on the number of complete predicted skeleton images and the number of specific movements includes:

[0029] If the number of complete predicted skeleton images is equal to the preset number of skeleton images, and the number of specific movements is equal to the preset number of movements, then the endoscopic examination is complete.

[0030] If the number of complete predicted skeleton images is not equal to the preset number of skeleton images, and the number of specific movements is not equal to the preset number of movements, then the endoscopic examination is incomplete.

[0031] In one embodiment, after determining whether the endoscopic examination process is complete based on the number of complete predicted skeleton images and the number of specific movements, the quality control method further includes:

[0032] An examination reminder is generated based on the results of the endoscopic examination.

[0033] Secondly, this application also provides a quality control device for medical endoscopic examinations, the device comprising:

[0034] The extraction module is used to acquire video frame images of an endoscopic examination and extract a predicted skeleton image of at least one target region from the video frame images;

[0035] The matching calculation module is used to match the predicted skeleton image with the pre-stored template skeleton image, calculate the overlap rate between the predicted skeleton image and the template skeleton image, and determine the number of complete predicted skeleton images based on the overlap rate.

[0036] The motion recognition module is used to obtain the relative displacement of the predicted skeleton image from the continuous video frame images, compare the relative displacement with the preset relative displacement template, and determine the number of specific movements.

[0037] The integrity judgment module is used to determine whether the endoscopic examination process is complete based on the number of complete predicted skeleton images and the number of specific movements.

[0038] In one embodiment, the device further includes:

[0039] The reminder generation module is used to generate examination reminders based on the judgment results of the endoscopic examination.

[0040] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0041] Acquire video frame images of an endoscopic examination, and extract a predicted skeleton image of at least one target region from the video frame images;

[0042] The predicted skeleton image is matched with a pre-stored template skeleton image, the overlap rate between the predicted skeleton image and the template skeleton image is calculated, and the number of complete predicted skeleton images is determined based on the overlap rate.

[0043] The relative displacement of the predicted skeleton image is obtained from the consecutive video frame images, and the relative displacement is compared with a preset relative displacement template to determine the number of specific movements;

[0044] The completeness of the endoscopic examination process is determined based on the number of complete predicted skeleton images and the number of specific movements.

[0045] The aforementioned quality control method, apparatus, and computer equipment for medical endoscopy extracts a predicted skeleton image of at least one target region from a video frame image, determines the inspection completeness of the target region based on the overlap between the predicted skeleton image and a pre-stored template skeleton image, captures specific movements to determine the movement status of the target region, and achieves quality monitoring of the integrity of the endoscopic examination of the detected moving parts by dual statistics of the number of complete predicted skeleton images and the number of specific movements. Attached Figure Description

[0046] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0047] Figure 1 This is an application environment diagram of a quality control method for medical endoscopy in one embodiment;

[0048] Figure 2 This is a flowchart illustrating a quality control method for medical endoscopy in one embodiment.

[0049] Figure 3 This is a flowchart illustrating a quality control method for medical endoscopy in a preferred embodiment.

[0050] Figure 4 This is a structural block diagram of a quality control device for medical endoscopy in one embodiment;

[0051] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0053] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0054] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0055] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0056] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the quality control method of medical endoscopy in this embodiment. Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0057] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the quality control method for medical endoscopy in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the aforementioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0058] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0059] This embodiment provides a quality control method for medical endoscopic examinations. Figure 2 This is a flowchart of the quality control method for medical endoscopy in this embodiment, as shown below. Figure 2 As shown, the process includes the following steps:

[0060] Step S210: Acquire video frame images of the endoscopic examination, and extract a predicted skeleton image of at least one target region from the video frame images.

[0061] Endoscopic examination refers to the examination of human organs using a medical endoscope. A medical endoscope includes an image sensor, illumination source, optical lenses, and other physical devices. It can enter through various parts of the body, such as the nose and mouth, to photograph the internal tissues and organs. When a doctor uses an endoscope, video images of the human organs are captured, and the video frames are obtained from the video in a frame-by-frame or frame-skipping manner. Each video frame includes at least one target region. For example, in an endoscopic video of the larynx, at least one tissue region of the larynx and the connecting regions between these tissue regions can all be considered as target regions. A predicted skeleton image of the target region is extracted. For example, the predicted skeleton image can be extracted by using a trained skeleton recognition model to recognize the video frame image, or by identifying key points in the video frame image and determining the predicted skeleton image based on the location and distribution information of the key points.

[0062] Step S220: Match the predicted skeleton image with the pre-stored template skeleton image, calculate the overlap rate between the predicted skeleton image and the template skeleton image, and determine the number of complete predicted skeleton images based on the overlap rate.

[0063] The similarity between the predicted points of the predicted skeleton image and the template keypoints of the pre-stored template skeleton image in the skeleton structure library is calculated. If the similarity is greater than a preset matching threshold, the predicted skeleton image and the template skeleton image are successfully matched. The weighted Euclidean distance between the predicted points of the predicted skeleton image and the template keypoints of the template skeleton image is calculated. The weighted Euclidean distance is used to determine whether the predicted points are correct, and the number of correct predicted points is used to determine the completeness of the predicted skeleton image.

[0064] For cases where there are at least two tissue regions in a single video frame, the integrity of each tissue region can be detected by extracting the predicted skeleton image of the tissue region. When a tissue region in a single video frame is incomplete, the predicted skeleton images of the tissue region in multiple video frames can be used to complete the prediction skeleton image of the complete tissue region. The overlap rate can then be calculated based on the predicted skeleton image to more accurately complete the subsequent integrity judgment.

[0065] Step S230: Obtain the relative displacement of the predicted skeleton image from the continuous video frame images, compare the relative displacement with the preset relative displacement template, and determine the number of specific movements.

[0066] When an endoscopic examination involves a specific body part with motion function, the relative displacement of the predicted skeleton image is obtained from consecutive video frame images and compared with a preset relative displacement template to calculate the motion of the tissue area of ​​that part. This can be applied to scenarios where endoscopic examination is used to examine movable parts and to determine the completeness of the examination results for the movable parts.

[0067] Step S240: Determine whether the endoscopic examination process is complete based on the number of complete predicted skeleton images and the number of specific movements.

[0068] Compared with the prior art, this embodiment extracts a predicted skeleton image of at least one target region from a video frame image, determines the inspection completeness of the target region based on the overlap between the predicted skeleton image and a pre-stored template skeleton image, captures specific motions to determine the motion status of the target region, and achieves quality monitoring of the endoscopic inspection completeness of the detected moving parts by dual statistics of the number of complete predicted skeleton images and the number of specific motions.

[0069] In one embodiment, based on step S220 above, the predicted skeleton image is matched with a pre-stored template skeleton image, the overlap rate between the predicted skeleton image and the template skeleton image is calculated, and the number of complete predicted skeleton images is determined based on the overlap rate. Specifically, this may include the following steps:

[0070] Step S221: Normalize the predicted skeleton image at the scale level.

[0071] The scale-level normalization process aims to eliminate the interference of inconsistent scales between the predicted skeleton image and the template skeleton image on the matching, thereby improving the matching accuracy. The scale-level normalization process adopts a linear function normalization method.

[0072] Step S222: Match the processed predicted skeleton image with the pre-stored template skeleton image.

[0073] In one embodiment, step S222 specifically includes the following steps:

[0074] Step S2221: Calculate the matching similarity between the processed predicted skeleton image and the pre-stored template skeleton image.

[0075] Specifically, the predicted points of the scale-normalized predicted skeleton image are extracted, along with the template keypoints of the pre-stored template skeleton image. The matching similarity OKS of the keypoints between the two is calculated, and the OKS of the p-th predicted skeleton image is calculated. P The calculation formula is as follows:

[0076]

[0077] Among them, the number of prediction points and template key points are the same and correspond one-to-one, and i represents the i-th prediction point and the i-th template key point;

[0078] d pi Let represent the Euclidean distance between the i-th predicted point in the p-th predicted skeleton image and the i-th template keypoint in the template skeleton image;

[0079] S p represents the scale factor of the p-th predicted skeleton image, and its value is the square root of the area of ​​the bounding rectangle of the predicted skeleton image.

[0080] σ i σ represents the normalization factor between the i-th predicted point and the i-th template keypoint. This factor is the standard deviation between the template keypoint values ​​and the true values ​​of manually annotated template skeleton images in all sample sets. The larger σ is, the more difficult it is to annotate this type of keypoint.

[0081] v pi v represents the visibility of the i-th keypoint in the p-th predicted skeleton image. pi =0 indicates that the key point is not marked (it does not exist in the diagram or its location is uncertain), v pi =1 indicates that the key points are unobstructed and have been labeled, v pi =2 indicates that the keypoint is occluded but has been labeled. For predicting keypoints, v p ′ i =0 indicates that no prediction was made, v p ′ i =1 indicates that it was predicted;

[0082] δ represents 1 if the condition is true, and 0 otherwise;

[0083] Step S2222: If the matching similarity is greater than the matching threshold, the matching is considered successful.

[0084] If the matching similarity OKS is greater than the matching threshold, it is determined that the predicted skeleton image in the current video frame image has the same structure as the template skeleton image, and the match is successful.

[0085] Step S223: Calculate the weighted Euclidean distance between the predicted points of the predicted skeleton image and the template keypoints of the template skeleton image.

[0086] The formula for calculating the weighted Euclidean distance is as follows:

[0087]

[0088] Where i represents the number of predicted points in the predicted skeleton image, j represents the number of template keypoints in the template skeleton image, x and y represent the position coordinates, and λi The weight represents the i-th prediction point. Different regions have different levels of difficulty, and this weight can be used to weight the results to make the prediction performance more stable.

[0089] Step S224: If the weighted Euclidean distance is less than the prediction threshold, then the predicted point is determined to be a correct predicted point.

[0090] If the weighted Euclidean distance corresponding to a prediction point is less than the preset prediction threshold, then the prediction point is a correct prediction point; otherwise, the prediction point is an outlier.

[0091] Step S225: Based on the number of correctly predicted points and the total number of predicted points, calculate the overlap rate between the predicted skeleton image and the template skeleton image. The specific calculation formula is as follows:

[0092]

[0093] Among them, C i T represents the overlap rate of the i-th predicted skeleton image. i N represents the number of correctly predicted points in the i-th predicted skeleton image. i This represents the total number of prediction points in the i-th predicted skeleton image.

[0094] Step S226: If the overlap rate is greater than the aforementioned completeness threshold, the predicted skeleton image is determined to be complete.

[0095] If the overlap rate is greater than the preset complete threshold, the predicted skeleton image is considered complete.

[0096] Step S227: Count the number of complete predicted skeleton images.

[0097] In cases where multiple tissue regions in a single frame image cannot be accurately identified, steps S221 to S227 extract a predicted skeleton image from the video frame image and use the predicted points in the predicted skeleton image to perform similarity calculation with the template key points in the template skeleton image. The weighted Euclidean distance is then calculated to obtain the correct predicted points more accurately. The overlap rate of the predicted skeleton image is then calculated based on the number of correct predicted points, thereby improving the accuracy of quality control in endoscopic examinations.

[0098] In one embodiment, based on step S230, the relative displacement of the predicted skeleton image is obtained from consecutive video frame images, and the relative displacement is compared with a preset relative displacement template to determine the number of specific movements. This specifically includes the following steps:

[0099] Step S231: Obtain the current predicted skeleton image and the historical predicted skeleton image from consecutive video frame images.

[0100] Among them, the historical predicted skeleton image is at least two video frames preceding the current predicted skeleton image.

[0101] Step S232: Obtain the relative displacement of the predicted skeleton image based on the current predicted skeleton image and the historical predicted skeleton images.

[0102] Specifically, the relative displacement combination between the corresponding prediction points in the current predicted skeleton image and the historical predicted skeleton images is calculated, and the relative displacement of the predicted skeleton image is obtained based on the relative displacement combination of the prediction points.

[0103] Step S233: Compare the relative displacement with the preset relative displacement template to obtain the displacement similarity.

[0104] Specifically, the relative displacement combination of the predicted points is compared with the relative displacement combination of each predicted point in the relative displacement template, and the magnitude of displacement similarity is calculated.

[0105] Step S234: If the displacement similarity is greater than the displacement threshold, then it is determined that a specific motion of the target area has been captured.

[0106] If the displacement similarity is greater than the preset displacement threshold, it is determined that the target area corresponding to the tissue area of ​​the detected part has completed a specific movement.

[0107] Step S235: Count the number of specific movements.

[0108] Specifically, the number of specific movements refers to the number of target areas to complete the specific movement.

[0109] When an endoscopic examination involves a specific area with motion function, the relative displacement combination of the predicted points of the predicted skeleton image is obtained from consecutive video frame images. By introducing the predicted points for calculation, the relative displacement can be obtained more accurately, thus revealing a more precise motion of the tissue area in that area.

[0110] In one embodiment, based on step S240 above, determining whether the endoscopic examination process is complete based on the number of complete predicted skeleton images and the number of specific movements may specifically include the following steps:

[0111] Step S241: If the number of complete predicted skeleton images is equal to the preset number of skeleton images, and the number of specific movements is equal to the preset number of movements, then the endoscopic examination is complete.

[0112] Step S242: If the number of complete predicted skeleton images is not equal to the preset number of skeleton images, and the number of specific movements is not equal to the preset number of movements, then the endoscopic examination is incomplete.

[0113] The completeness of endoscopic examination is judged by both the number of complete predicted skeleton images and the number of specific movements, thus achieving the effect of quality control.

[0114] In one embodiment, in step S240, based on the number of complete predicted skeleton images and the number of specific movements, it is determined whether the endoscopic examination process is complete, and then the following steps are included:

[0115] Step S250: Generate an examination reminder based on the results of the endoscopic examination.

[0116] The results of endoscopic examinations can be visualized, such as by displaying the current endoscopic examination results via an app. Alternatively, based on the number of complete predicted skeletal images, the number of specific movements, and the results, the results can be played back via voice to inform the doctor of the areas that have been covered and those that have not.

[0117] The present embodiment will now be described and illustrated through preferred embodiments.

[0118] Figure 3 This is a flowchart of a quality control method for medical endoscopy according to a preferred embodiment.

[0119] Step S310: Acquire video frame images of the endoscopic examination, and extract a predicted skeleton image of at least one target region from the video frame images.

[0120] Step S320: Normalize the predicted skeleton image at the scale level, calculate the matching similarity between the processed predicted skeleton image and the pre-stored template skeleton image, and determine that the matching is successful if the matching similarity is greater than the matching threshold.

[0121] Step S330: Calculate the weighted Euclidean distance between the predicted points of the predicted skeleton image and the template key points of the template skeleton image. If the weighted Euclidean distance is less than the prediction threshold, the predicted points are determined to be correct prediction points. Based on the number of correct prediction points and the total number of prediction points, calculate the overlap rate between the predicted skeleton image and the template skeleton image. If the overlap rate is greater than the completeness threshold, the predicted skeleton image is determined to be complete, and the number of complete predicted skeleton images is counted.

[0122] Step S340: Obtain the current predicted skeleton image and the historical predicted skeleton image from the continuous video frame images; obtain the relative displacement of the predicted skeleton image based on the current predicted skeleton image and the historical predicted skeleton image; compare the relative displacement with the preset relative displacement template to obtain the displacement similarity; if the displacement similarity is greater than the displacement threshold, it is determined that a specific motion of the target area has been captured, and the number of specific motions is counted.

[0123] Step S350: If the number of complete predicted skeleton images is equal to the preset number of skeleton images, and the number of specific movements is equal to the preset number of movements, then the endoscopic examination is complete, and an examination reminder is generated based on the judgment result of the endoscopic examination.

[0124] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0125] Based on the same inventive concept, this embodiment also provides a quality control device for medical endoscopy. This system is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0126] In one embodiment, such as Figure 4 As shown, a quality control device 40 for medical endoscopy is provided, comprising: an extraction module 41, a matching calculation module 42, a motion recognition module 43, and an integrity judgment module 44, wherein:

[0127] Extraction module 41 is used to acquire video frame images of endoscopic examination and extract a predicted skeleton image of at least one target region from the video frame images.

[0128] Endoscopic examination refers to the examination of human organs using a medical endoscope. A medical endoscope includes an image sensor, illumination source, optical lenses, and other physical devices. It can enter through various parts of the body, such as the nose and mouth, to photograph the internal tissues and organs. When a doctor uses an endoscope, video images of the human organs are captured, and the video frames are obtained from the video in a frame-by-frame or frame-skipping manner. Each video frame includes at least one target region. For example, in an endoscopic video of the larynx, at least one tissue region of the larynx and the connecting regions between these tissue regions can all be considered as target regions. A predicted skeleton image of the target region is extracted. For example, the predicted skeleton image can be extracted by using a trained skeleton recognition model to recognize the video frame image, or by identifying key points in the video frame image and determining the predicted skeleton image based on the location and distribution information of the key points.

[0129] The matching calculation module 42 is used to match the predicted skeleton image with the pre-stored template skeleton image, calculate the overlap rate between the predicted skeleton image and the template skeleton image, and determine the number of complete predicted skeleton images based on the overlap rate.

[0130] The similarity between the predicted points of the predicted skeleton image and the template keypoints of the pre-stored template skeleton image in the skeleton structure library is calculated. If the similarity is greater than a preset matching threshold, the predicted skeleton image and the template skeleton image are successfully matched. The weighted Euclidean distance between the predicted points of the predicted skeleton image and the template keypoints of the template skeleton image is calculated. The weighted Euclidean distance is used to determine whether the predicted points are correct, and the number of correct predicted points is used to determine the completeness of the predicted skeleton image.

[0131] For cases where there are at least two tissue regions in a single video frame, the integrity of each tissue region can be detected by extracting the predicted skeleton image of the tissue region. When a tissue region in a single video frame is incomplete, the predicted skeleton images of the tissue region in multiple video frames can be used to complete the prediction skeleton image of the complete tissue region. The overlap rate can then be calculated based on the predicted skeleton image to more accurately complete the subsequent integrity judgment.

[0132] The motion recognition module 43 is used to obtain the relative displacement of the predicted skeleton image from continuous video frame images, compare the relative displacement with a preset relative displacement template, and determine the number of specific movements.

[0133] When an endoscopic examination involves a specific body part with motion function, the relative displacement of the predicted skeleton image is obtained from consecutive video frame images and compared with a preset relative displacement template to calculate the motion of the tissue area of ​​that part. This can be applied to scenarios where endoscopic examination is used to examine movable parts and to determine the completeness of the examination results for the movable parts.

[0134] The integrity judgment module 44 is used to determine whether the endoscopic examination process is complete based on the number of complete predicted skeleton images and the number of specific movements. Specifically, if the number of complete predicted skeleton images equals the preset number of skeleton images and the number of specific movements equals the preset number of movements, the endoscopic examination is complete; if the number of complete predicted skeleton images does not equal the preset number of skeleton images and the number of specific movements does not equal the preset number of movements, the endoscopic examination is incomplete.

[0135] This embodiment extracts a predicted skeleton image of at least one target region from a video frame image. Based on the overlap between the predicted skeleton image and a pre-stored template skeleton image, the inspection completeness of the target region is determined. Specific movements are captured to determine the movement status of the target region. By statistically analyzing both the number of complete predicted skeleton images and the number of specific movements, the quality monitoring of the endoscopic inspection completeness of the detected moving parts is achieved.

[0136] In one embodiment, the quality control device 40 for medical endoscopy further includes a reminder generation module 45.

[0137] The reminder generation module 45 is used to generate examination reminders based on the judgment results of the endoscopic examination.

[0138] The modules in the aforementioned quality control device for medical endoscopy can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0139] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a quality control method for medical endoscopy. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0140] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0141] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0142] Step S210: Acquire video frame images of the endoscopic examination, and extract a predicted skeleton image of at least one target region from the video frame images.

[0143] Step S220: Match the predicted skeleton image with the pre-stored template skeleton image, calculate the overlap rate between the predicted skeleton image and the template skeleton image, and determine the number of complete predicted skeleton images based on the overlap rate.

[0144] Step S230: Obtain the relative displacement of the predicted skeleton image from the continuous video frame images, compare the relative displacement with the preset relative displacement template, and determine the number of specific movements.

[0145] Step S240: Determine whether the endoscopic examination process is complete based on the number of complete predicted skeleton images and the number of specific movements.

[0146] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0147] Step S250: Generate an examination reminder based on the results of the endoscopic examination.

[0148] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0149] Step S210: Acquire video frame images of the endoscopic examination, and extract a predicted skeleton image of at least one target region from the video frame images.

[0150] Step S220: Match the predicted skeleton image with the pre-stored template skeleton image, calculate the overlap rate between the predicted skeleton image and the template skeleton image, and determine the number of complete predicted skeleton images based on the overlap rate.

[0151] Step S230: Obtain the relative displacement of the predicted skeleton image from the continuous video frame images, compare the relative displacement with the preset relative displacement template, and determine the number of specific movements.

[0152] Step S240: Determine whether the endoscopic examination process is complete based on the number of complete predicted skeleton images and the number of specific movements.

[0153] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0154] Step S250: Generate an examination reminder based on the results of the endoscopic examination.

[0155] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0156] Step S210: Acquire video frame images of the endoscopic examination, and extract a predicted skeleton image of at least one target region from the video frame images.

[0157] Step S220: Match the predicted skeleton image with the pre-stored template skeleton image, calculate the overlap rate between the predicted skeleton image and the template skeleton image, and determine the number of complete predicted skeleton images based on the overlap rate.

[0158] Step S230: Obtain the relative displacement of the predicted skeleton image from the continuous video frame images, compare the relative displacement with the preset relative displacement template, and determine the number of specific movements.

[0159] Step S240: Determine whether the endoscopic examination process is complete based on the number of complete predicted skeleton images and the number of specific movements.

[0160] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0161] Step S250: Generate an examination reminder based on the results of the endoscopic examination.

[0162] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0163] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0164] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0165] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A quality control method for medical endoscopic examinations, characterized in that, The quality control method includes: Acquire video frame images of an endoscopic examination, and extract a predicted skeleton image of at least one target region from the video frame images; The predicted skeleton image is matched with a pre-stored template skeleton image, the overlap rate between the predicted skeleton image and the template skeleton image is calculated, and the number of complete predicted skeleton images is determined based on the overlap rate. The relative displacement of the predicted skeleton image is obtained from the consecutive video frame images, and the relative displacement is compared with a preset relative displacement template to determine the number of specific movements. The completeness of the endoscopic examination process is determined based on the number of complete predicted skeleton images and the number of specific movements.

2. The quality control method for medical endoscopy according to claim 1, characterized in that, The step of matching the predicted skeleton image with the pre-stored template skeleton image includes: The predicted skeleton image is normalized at the scale level. The processed predicted skeleton image is matched with the pre-stored template skeleton image.

3. The quality control method for medical endoscopy according to claim 2, characterized in that, The step of matching the processed predicted skeleton image with the pre-stored template skeleton image includes: Calculate the matching similarity between the processed predicted skeleton image and the pre-stored template skeleton image; If the matching similarity is greater than the matching threshold, the match is considered successful.

4. The quality control method for medical endoscopy according to claim 1, characterized in that, The step of calculating the overlap rate between the predicted skeleton image and the template skeleton image, and determining the number of complete predicted skeleton images based on the overlap rate, includes: Calculate the weighted Euclidean distance between the predicted points of the predicted skeleton image and the template keypoints of the template skeleton image; If the weighted Euclidean distance is less than the prediction threshold, then the predicted point is determined to be a correct prediction point; The overlap rate between the predicted skeleton image and the template skeleton image is calculated based on the number of correctly predicted points and the total number of predicted points. If the overlap rate is greater than the integrity threshold, the predicted skeleton image is determined to be complete; Count the number of complete predicted skeleton images.

5. The quality control method for medical endoscopy according to claim 1, characterized in that, The step of obtaining the relative displacement of the predicted skeleton image from consecutive video frame images, comparing the relative displacement with a preset relative displacement template, and determining the number of specific movements includes: Obtain the current predicted skeleton image and the historical predicted skeleton images from the consecutive video frame images; The relative displacement of the predicted skeleton image is obtained based on the current predicted skeleton image and the historical predicted skeleton images; The displacement similarity is obtained by comparing the relative displacement with a preset relative displacement template; If the displacement similarity is greater than the displacement threshold, it is determined that a specific motion of the target area has been captured; Count the number of the specific movements.

6. The quality control method for medical endoscopy according to claim 1, characterized in that, The step of determining whether the endoscopic examination process is complete based on the number of complete predicted skeleton images and the number of specific movements includes: If the number of complete predicted skeleton images is equal to the preset number of skeleton images, and the number of specific movements is equal to the preset number of movements, then the endoscopic examination is complete. If the number of complete predicted skeleton images is not equal to the preset number of skeleton images, and the number of specific movements is not equal to the preset number of movements, then the endoscopic examination is incomplete.

7. The quality control method for medical endoscopy according to claim 1, characterized in that, After determining whether the endoscopic examination process is complete based on the number of complete predicted skeleton images and the number of specific movements, the quality control method further includes: An examination reminder is generated based on the results of the endoscopic examination.

8. A quality control device for medical endoscopic examination, characterized in that, The device includes: The extraction module is used to acquire video frame images of endoscopic examinations and extract a predicted skeleton image of at least one target region from the video frame images; The matching calculation module is used to match the predicted skeleton image with the pre-stored template skeleton image, calculate the overlap rate between the predicted skeleton image and the template skeleton image, and determine the number of complete predicted skeleton images based on the overlap rate. The motion recognition module is used to obtain the relative displacement of the predicted skeleton image from the continuous video frame images, compare the relative displacement with the preset relative displacement template, and determine the number of specific movements. The integrity judgment module is used to determine whether the endoscopic examination process is complete based on the number of complete predicted skeleton images and the number of specific movements.

9. The quality control device for medical endoscopy according to claim 8, characterized in that, The device further includes: The reminder generation module is used to generate examination reminders based on the judgment results of the endoscopic examination.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the quality control method for medical endoscopy as claimed in any one of claims 1 to 7.

Images