Method and system for automatically intercepting pictures of key positions in electronic nasopharyngoscopy

Abstract

The application provides a key position picture automatic interception method and system in electronic nasopharyngoscopy, acquires a current frame of a video stream, carries out gray scale on the current frame to obtain a gray scale picture, extracts the contour of the gray scale picture after preprocessing the gray scale picture, judges the category to which the current frame belongs and the probability of the category until all frames are processed; determines the first N video frames with the highest probability in each category, takes each of the first N video frames as a center frame, obtains the weight of the center frame according to the probability of each M frame adjacent to the front and back of the center frame and the probability of the center frame, filters the first N video frames according to the weight to obtain a plurality of candidate frames corresponding to each category; determines a key frame according to the definition of the candidate frame and the weight of the candidate frame, takes the key frame as a key position picture, and saves the key position picture. The application improves the automation degree of the screenshot in electronic nasopharyngoscopy, has high accuracy of interception, and the image is clear.

Description

Technical Field

[0001] This invention relates to the field of electronic nasopharyngoscopy, and more particularly to a method and system for automatically capturing images of key locations during electronic nasopharyngoscopy. Background Technology

[0002] Nasopharyngeal diseases severely disrupt patients' daily lives. The structure of the nasopharynx is complex and difficult to observe. Nasopharyngoscopy allows direct observation of the nasopharynx. Due to its minimally invasive and non-invasive nature, it is highly acceptable to patients, and electronic nasopharyngoscopy has become an important tool for nasopharyngeal examinations. Electronic nasopharyngoscopy is a type of endoscope. Flexible endoscopes are divided into fiber optic endoscopes and electronic endoscopes. Compared to fiber optic endoscopes, electronic endoscopes have a wider range of applications. Furthermore, with the development of photosensitive components and image processing technology, electronic endoscopes provide increasingly clear images. Another advantage of electronic endoscopes is that they can store the observation process for later review. When using electronic nasopharyngoscopy, doctors often need to take screenshots of lesions or suspected lesions. This allows patients to see the results in the examination report and helps doctors further confirm the lesions or suspected lesions. However, during electronic nasopharyngoscopy, taking screenshots is always done by the doctor, which can often distract the doctor. Automatic image capture during the electronic nasopharyngoscopy examination is therefore very useful. Summary of the Invention

[0003] In order to enable automatic image capture of key locations during electronic nasopharyngoscopy, in a first aspect, the present invention provides a method for automatic image capture of key locations during electronic nasopharyngoscopy, the method comprising the following steps:

[0004] S1: Read the video stream from the electronic nasopharyngoscope, obtain the current frame of the video stream, convert the current frame to grayscale to obtain a grayscale image, extract the contour of the grayscale image after preprocessing, determine the category to which the current frame belongs and the probability of belonging to the category based on the contour, and continue to execute S1 for the next frame of the video stream until all frames in the video stream have been processed.

[0005] S2, determine the top N video frames with the highest probability in each category, take each of the top N video frames as the center frame, obtain the weight of the center frame based on the probability of each of the M frames adjacent to the center frame before and after it and the probability of the center frame, and filter the top N video frames according to the weight to obtain multiple candidate frames corresponding to each category; where N and M are positive integers greater than 1.

[0006] S3, determine the key frame of the category according to the clarity and weight of the candidate frame, and save the image corresponding to the key frame as the key position image.

[0007] Preferably, the step of extracting the outline of the grayscale image after preprocessing specifically involves:

[0008] The grayscale image is convolved using a first convolution kernel to obtain a first convolution result, and the first convolution result is convolved using a second convolution kernel to obtain a second convolution result;

[0009] Pixels with pixel values ​​greater than a preset value in the first convolution result are obtained, and a first region set for the erosion operation is determined based on the position of the pixel and the first receptive field; pixels with pixel values ​​greater than a preset value in the second convolution result are obtained, and a second region set for the erosion operation is determined based on the position of the pixel and the second receptive field; if the regions in the first region set and the regions in the second region set intersect, the intersecting first region is deleted from the first region set.

[0010] A first erosion convolution kernel is used to perform an erosion operation on regions in a first region set; a second erosion convolution kernel is used to perform an erosion operation on regions in a second region set; wherein, the first receptive field and the first erosion convolution kernel are determined based on the first convolution kernel; the second receptive field and the second erosion convolution kernel are determined based on the first convolution kernel and the second convolution kernel;

[0011] The grayscale image after the erosion operation is filtered, and the edge contours of the filtered grayscale image are extracted.

[0012] Preferably, the step of obtaining the weight of the center frame based on the probabilities of the M frames adjacent to the center frame before and after it, and the probability of the center frame itself, specifically involves:

[0013] For each of the M frames adjacent to the center frame, determine whether the category of the frame is the same as that of the center frame. If they are the same, accumulate the probability. If they are different, skip the frame. Obtain the total probability based on the accumulated result and the probability corresponding to the center frame, and use the total probability as the weight.

[0014] Preferably, determining the keyframe belonging to a category based on the sharpness and weight of the candidate frames specifically involves:

[0015] Calculate the sharpness of each candidate frame, sort the candidate frames according to the product of the sharpness and the weight, and take the first ranked candidate frame as the key frame.

[0016] Preferably, the calculation of the sharpness of each candidate frame specifically involves:

[0017] Candidate frames are convolved using a Laplacian kernel, and the variance of the convolved result is calculated. This variance is then used as the sharpness.

[0018] In another aspect, the present invention provides an automatic image capture system for key locations during electronic nasopharyngoscopy, the system comprising the following modules:

[0019] The video frame classification module is used to read the video stream of the electronic nasopharyngoscope, obtain the current frame of the video stream, convert the current frame to grayscale to obtain a grayscale image, extract the contour of the grayscale image after preprocessing, determine the category to which the current frame belongs and the probability of belonging to the category based on the contour, and continue to execute the video frame classification module for the next frame of the video stream until all frames in the video stream have been processed.

[0020] The candidate frame selection module is used to determine the top N video frames with the highest probability in each category, take each of the top N video frames as the center frame, obtain the weight of the center frame based on the probability of each of the M adjacent frames before and after the center frame and the probability of the center frame, and filter the top N video frames according to the weight to obtain multiple candidate frames corresponding to each category; where N and M are positive integers greater than 1.

[0021] The key position image capture module is used to determine the key frame of the category based on the clarity and weight of the candidate frame, and save the image corresponding to the key frame as the key position image.

[0022] Preferably, the step of extracting the outline of the grayscale image after preprocessing specifically involves:

[0023] The grayscale image is convolved using a first convolution kernel to obtain a first convolution result, and the first convolution result is convolved using a second convolution kernel to obtain a second convolution result;

[0024] Pixels with pixel values ​​greater than a preset value in the first convolution result are obtained, and a first region set for the erosion operation is determined based on the position of the pixel and the first receptive field; pixels with pixel values ​​greater than a preset value in the second convolution result are obtained, and a second region set for the erosion operation is determined based on the position of the pixel and the second receptive field; if the regions in the first region set and the regions in the second region set intersect, the intersecting first region is deleted from the first region set.

[0025] A first erosion convolution kernel is used to perform an erosion operation on regions in a first region set; a second erosion convolution kernel is used to perform an erosion operation on regions in a second region set; wherein, the first receptive field and the first erosion convolution kernel are determined based on the first convolution kernel; the second receptive field and the second erosion convolution kernel are determined based on the first convolution kernel and the second convolution kernel;

[0026] The grayscale image after the erosion operation is filtered, and the edge contours of the filtered grayscale image are extracted.

[0027] Preferably, the step of obtaining the weight of the center frame based on the probabilities of the M frames adjacent to the center frame before and after it, and the probability of the center frame itself, specifically involves:

[0028] For each of the M frames adjacent to the center frame, determine whether the category of the frame is the same as that of the center frame. If they are the same, accumulate the probability. If they are different, skip the frame. Obtain the total probability based on the accumulated result and the probability corresponding to the center frame, and use the total probability as the weight.

[0029] Preferably, determining the keyframe belonging to a category based on the sharpness and weight of the candidate frames specifically involves:

[0030] Calculate the sharpness of each candidate frame, sort the candidate frames according to the product of the sharpness and the weight, and take the first ranked candidate frame as the key frame.

[0031] Preferably, the calculation of the sharpness of each candidate frame specifically involves:

[0032] Candidate frames are convolved using a Laplacian kernel, and the variance of the convolved result is calculated. This variance is then used as the sharpness.

[0033] In another aspect, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0034] To improve the automation level of nasopharyngoscopy, this invention provides a method and system for automatically capturing images of key locations during electronic nasopharyngoscopy. First, the category and probability of a video frame belonging to that category are determined based on its outline. The N video frames with the highest probability in each category are selected and used as the center frame. Candidate frames are then selected based on the category and probability of the center frame and adjacent video frames. Finally, key frames are selected from the candidate frames and used as the automatically captured images. This invention improves the automation level of nasopharyngoscopy and can accurately extract clear images that the doctor wants to capture. Attached Figure Description

[0035] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0036] Figure 1This is a flowchart of Example 1;

[0037] Figure 2 A schematic diagram of the sensory field;

[0038] Figure 3 The effect of eliminating light spots in the video frame of an electronic nasopharyngoscope;

[0039] Figure 4 This is a module diagram of Example 2. Detailed Implementation

[0040] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Existing screenshot methods require doctors to manually capture images at key locations such as the pharynx and larynx during the examination. Based on this, the present invention provides a method and system for automatically capturing images of key locations during electronic nasopharyngoscopy.

[0043] Example 1

[0044] In the first aspect, the present invention provides a method for automatically extracting images of key locations during electronic nasopharyngoscopy, such as... Figure 1 As shown, the method includes the following steps:

[0045] S1: Read the video stream from the electronic nasopharyngoscope, obtain the current frame of the video stream, convert the current frame to grayscale to obtain a grayscale image, extract the contour of the grayscale image after preprocessing, determine the category to which the current frame belongs and the probability of belonging to the category based on the contour, and continue to execute S1 for the next frame of the video stream until all frames in the video stream have been processed.

[0046] This invention allows for the reading of video from an electronic nasopharyngoscope after a nasopharyngoscopy examination, or it can be done during the examination itself. The video is actually composed of many video frames, each of which is an image. Analysis involves processing each frame of the video. After obtaining the current video frame to be processed, the frame is converted to grayscale to obtain a grayscale image. Then, the grayscale image undergoes preprocessing. In one embodiment, the preprocessing includes, but is not limited to, noise reduction and smoothing.

[0047] Due to the influence of the light source in the electronic nasopharyngoscope, light spots will be generated. The size and unknown nature of the light spots are uncertain, which will affect subsequent analysis. In another embodiment, the preprocessing also includes removing the light spots.

[0048] After preprocessing, the outline of the grayscale image is extracted, and then the category to which the current frame belongs is determined based on the outline. In one specific embodiment, a deep learning model or SVM is used to determine the category to which the current frame belongs. The number and content of categories are pre-selected by the program, or can be set by the doctor before the examination. In an exemplary example, there are two categories: pharynx and larynx.

[0049] Subsequent steps will further determine the key locations in the image, without requiring high precision. The deep learning model or SVM mentioned is a lightweight model. In a more specific embodiment, the deep learning model includes an encoding module and a decoding module. The encoding module uses 4-6 convolutions combined with activation and pooling, while the decoding module uses one or two fully connected layers.

[0050] For example, if the 10th frame is the current frame, the classification model determines that the probability of the current frame being the pharynx is 0.7 and the probability of it being the larynx is 0.3. Therefore, the current frame is classified as the pharynx with a probability of 0.7. After processing the 10th frame, the category and probability of the 11th frame are determined, and so on, until all frames in the video stream have been processed.

[0051] S2, determine the top N video frames with the highest probability in each category, take each of the top N video frames as the center frame, obtain the weight of the center frame based on the probability of each of the M frames adjacent to the center frame before and after it and the probability of the center frame, and filter the top N video frames according to the weight to obtain multiple candidate frames corresponding to each category; where N and M are positive integers greater than 1.

[0052] During the examination, the electronic nasopharyngoscope moves slowly inside the body, and adjacent video frames display similar content. For example, during a pharyngeal examination, video frames will show the pharynx for a short period. If a nasal image appears during this period, it's highly likely a misclassification in step 1 or a movement from the nose to the pharynx, which is obviously not the optimal time for a screenshot. After determining the category and probability of each frame in the video, the N video frames with the highest corresponding probabilities are selected. Each of these N video frames is a center frame. For each center frame, the probabilities of the M video frames closest to and before the center frame, and the probabilities of the M video frames closest to and after the center frame are further determined. Including the center frame, there are a total of 2M+1 video frames, meaning 2M+1 video frames are obtained from one center frame. The weight of the center frame is determined based on the category and probability of these 2M+1 video frames, and then candidate frames are selected from the N video frames. After the above process, candidate frames for each category are selected. Step 2 allows for more precise selection of images at key locations.

[0053] S3, determine the key frame of the category according to the clarity and weight of the candidate frame, and save the image corresponding to the key frame as the key position image.

[0054] During the examination, the nasopharyngoscope or head movement may cause out-of-focus or motion blur. If a blurry image is selected, the key image will fail to be selected. To avoid this, the present invention further calculates the sharpness of the candidate frames, and then determines the key frames of each category based on the sharpness and weight, thereby selecting images with high sharpness and the nasopharyngoscope being in the key position.

[0055] During electronic nasopharyngoscopy, light spots are generated, and the size and location of these spots are uncertain, affecting subsequent feature recognition. In a specific embodiment, the extraction of the grayscale image contour after grayscale image preprocessing specifically involves:

[0056] The grayscale image is convolved using a first convolution kernel to obtain a first convolution result, and the first convolution result is convolved using a second convolution kernel to obtain a second convolution result;

[0057] That is, a first convolutional kernel and a second convolutional kernel are used to perform cascaded convolution on the grayscale image. The first convolutional kernel can be the same as the second convolutional kernel. Preferably, the first and second convolutional kernels are dilated convolutional kernels. After passing through the first and second convolutional kernels, pooling can be performed to further reduce the size of the grayscale image. The receptive fields of the first and second convolutional results are different, and the resulting feature maps are also different. If there is a small spot, there will be a bright spot in the first convolutional result, but the bright spot will disappear in the second convolutional result. Conversely, a large spot will also have a bright spot in the second convolutional result. A schematic diagram of the receptive field is shown below. Figure 2 As shown.

[0058] Pixels with pixel values ​​greater than a preset value in the first convolution result are obtained, and a first region set for the erosion operation is determined based on the position of the pixel and the first receptive field; pixels with pixel values ​​greater than a preset value in the second convolution result are obtained, and a second region set for the erosion operation is determined based on the position of the pixel and the second receptive field; if the regions in the first region set and the regions in the second region set intersect, the intersecting first region is deleted from the first region set.

[0059] In one embodiment, the method further includes clustering pixels with values ​​greater than a preset value. Specifically, the pixels are clustered based on their positions in the first convolution result, and the pixels located at or closest to the cluster center are considered as pixels with values ​​greater than the preset value. Similarly, pixels with values ​​greater than the preset value in the second convolution result are clustered.

[0060] In the first convolution result, each pixel corresponds to multiple pixels in the grayscale image. The bright spots in the first convolution result correspond to large light spots in the grayscale image. Similarly, the bright spots in the second convolution result correspond to even larger light spots in the grayscale image. This invention uses two convolution kernels, namely the first convolution kernel and the second convolution kernel, to avoid the problem of using a single convolution kernel. If the kernel is too large, small light spots in the grayscale image will be missed. If the kernel is too small, a large light spot will be divided into multiple small light spots.

[0061] In the first and second convolution results, each bright spot corresponds to a light spot. Deduplication is also required. Specifically, if regions in the first region set and regions in the second region set intersect, i.e., they overlap, the region will be removed from the first region set. In one specific embodiment, the region refers to a region in the grayscale image.

[0062] The first erosion convolution kernel is used to perform erosion operation on the regions in the first region set; the second erosion convolution kernel is used to perform erosion operation on the regions in the second region set;

[0063] In one specific embodiment, if the convolution stride is 1, then the first receptive field and the first erosion convolution kernel are determined based on the first convolution kernel; the second receptive field and the second erosion convolution kernel are determined based on the first convolution kernel and the second convolution kernel.

[0064] In another specific embodiment, if the convolution stride is not 1, the first receptive field and the first erosion convolution kernel are determined based on the first convolution kernel and the stride of the first convolution kernel; the second receptive field and the second erosion convolution kernel are determined based on the first convolution kernel, the second convolution kernel, and the stride of the first convolution kernel and the stride of the second convolution kernel.

[0065] Assuming the first convolutional kernel is 3×3, padding is 1×1, and stride is 2×2, then the range of each pixel in the resulting feature map (i.e., the grayscale image before convolution) is 3×3, and the receptive field is 3×3. If the second convolutional kernel is the same size as the first, with padding of 1×1 and stride of 2×2, then the second receptive field is 7×7. Preferably, the erosion convolutional kernel uses a kernel larger than the receptive field; this erosion convolutional kernel is the one used for the erosion operation.

[0066] The sizes of the regions in the first and second region sets are different, which is related to the receptive field in the convolution process described above. The erosion kernels used in eliminating light spots are also different, resulting in different effects on light spot elimination. Figure 3 As shown.

[0067] The above process can eliminate light spots, but it cannot eliminate noise. Further, the grayscale image after the erosion operation is filtered to extract the edge contour of the filtered grayscale image.

[0068] If the electronic nasopharyngoscope is located in a critical position, this short period of time or this sequence of video frames will be classified as the same category. To improve recognition accuracy, in a specific embodiment, the weight of the center frame is obtained based on the probabilities of the M frames adjacent to the center frame before and after it, and the probability of the center frame itself. Specifically:

[0069] For each of the M frames adjacent to the center frame, determine whether the category of the frame is the same as that of the center frame. If they are the same, accumulate the probability. If they are different, skip the frame. Obtain the total probability based on the accumulated result and the probability corresponding to the center frame, and use the total probability as the weight.

[0070] Assuming M=2, the center frame is frame 10, and the categories and probabilities of frames 8, 9, 10, 11, and 12 are (A, 0.6), (A, 0.8), (A, 0.9), (A, 0.6), and (B, 0.6) respectively, where A and B represent categories, the total probability is 2.9. Frame 12 is not included here because its category is different from that of the center frame; therefore, the weight corresponding to the center frame is 2.9. It should be understood that the assumption of M=2 here is only for the sake of explanation in this embodiment.

[0071] It is obviously inappropriate to automatically capture a blurry video frame as a screenshot of a key location. To avoid this, in one embodiment, the key frame of a certain category is determined based on the sharpness and weight of the candidate frames, specifically as follows:

[0072] Calculate the sharpness of each candidate frame, sort the candidate frames according to the product of the sharpness and the weight, and take the first ranked candidate frame as the key frame.

[0073] The calculation of the sharpness of each candidate frame is specifically as follows:

[0074] Candidate frames are convolved using a Laplacian kernel, and the variance of the convolved result is calculated. This variance is then used as the sharpness.

[0075] In another aspect, the present invention provides an automatic image capture system 100 for key locations during electronic nasopharyngoscopy, the system comprising the following modules:

[0076] The video frame classification module 201 is used to read the video stream of the electronic nasopharyngoscope, obtain the current frame of the video stream, convert the current frame to grayscale to obtain a grayscale image, extract the contour of the grayscale image after preprocessing, determine the category to which the current frame belongs and the probability of belonging to the category based on the contour, and continue to execute the video frame classification module for the next frame of the video stream until all frames in the video stream have been processed.

[0077] The candidate frame selection module 202 is used to determine the top N video frames with the highest probability in each category, take each of the top N video frames as the center frame, obtain the weight of the center frame based on the probability of each of the M frames adjacent to the center frame before and after it and the probability of the center frame, and filter the top N video frames according to the weight to obtain multiple candidate frames corresponding to each category; where N and M are positive integers greater than 1.

[0078] The key position image capture module 203 is used to determine the key frame of the category according to the clarity and weight of the candidate frame, and save the image corresponding to the key frame as the key position image.

[0079] Preferably, the step of extracting the outline of the grayscale image after preprocessing specifically involves:

[0080] The grayscale image is convolved using a first convolution kernel to obtain a first convolution result, and the first convolution result is convolved using a second convolution kernel to obtain a second convolution result;

[0081] Pixels with pixel values ​​greater than a preset value in the first convolution result are obtained, and a first region set for the erosion operation is determined based on the position of the pixel and the first receptive field; pixels with pixel values ​​greater than a preset value in the second convolution result are obtained, and a second region set for the erosion operation is determined based on the position of the pixel and the second receptive field; if the regions in the first region set and the regions in the second region set intersect, the intersecting first region is deleted from the first region set.

[0082] A first erosion convolution kernel is used to perform an erosion operation on regions in a first region set; a second erosion convolution kernel is used to perform an erosion operation on regions in a second region set; wherein, the first receptive field and the first erosion convolution kernel are determined based on the first convolution kernel; the second receptive field and the second erosion convolution kernel are determined based on the first convolution kernel and the second convolution kernel;

[0083] The grayscale image after the erosion operation is filtered, and the edge contours of the filtered grayscale image are extracted.

[0084] Preferably, the step of obtaining the weight of the center frame based on the probabilities of the M frames adjacent to the center frame before and after it, and the probability of the center frame itself, specifically involves:

[0085] For each of the M frames adjacent to the center frame, determine whether the category of the frame is the same as that of the center frame. If they are the same, accumulate the probability. If they are different, skip the frame. Obtain the total probability based on the accumulated result and the probability corresponding to the center frame, and use the total probability as the weight.

[0086] Preferably, determining the keyframe belonging to a category based on the sharpness and weight of the candidate frames specifically involves:

[0087] Calculate the sharpness of each candidate frame, sort the candidate frames according to the product of the sharpness and the weight, and take the first ranked candidate frame as the key frame.

[0088] Preferably, the calculation of the sharpness of each candidate frame specifically involves:

[0089] Candidate frames are convolved using a Laplacian kernel, and the variance of the convolved result is calculated. This variance is then used as the sharpness.

[0090] Example 3

[0091] The present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method described in Embodiment 1 of the present invention.

[0092] Example 4

[0093] The present invention provides a computer device, which includes at least a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the method described in Embodiment 1 of the present invention.

[0094] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically extracting images of key locations during electronic nasopharyngoscopy, characterized in that, The method includes using Next steps: S1: Read the video stream from the electronic nasopharyngoscope, obtain the current frame of the video stream, convert the current frame to grayscale to obtain a grayscale image, extract the contour of the grayscale image after preprocessing, determine the category to which the current frame belongs and the probability of belonging to the category based on the contour, and continue to execute S1 for the next frame of the video stream until all frames in the video stream have been processed. S2, determine the top N video frames with the highest probability in each category, take each of the top N video frames as the center frame, obtain the weight of the center frame based on the probability of each of the M frames adjacent to the center frame before and after it and the probability of the center frame, and filter the top N video frames according to the weight to obtain multiple candidate frames corresponding to each category; where N and M are positive integers greater than 1. S3, determine the key frame of the category according to the clarity and weight of the candidate frame, and save the image corresponding to the key frame as the key position image. The extraction of grayscale image contours after grayscale image preprocessing specifically involves: convolving the grayscale image with a first convolution kernel to obtain a first convolution result; convolving the first convolution result with a second convolution kernel to obtain a second convolution result; identifying pixels with pixel values ​​greater than a preset value in the first convolution result; determining a first set of regions for erosion operation based on the position of the pixels and a first receptive field; identifying pixels with pixel values ​​greater than the preset value in the second convolution result; determining a second set of regions for erosion operation based on the position of the pixels and a second receptive field; if regions in the first set and regions in the second set intersect, deleting the intersecting first region from the first set; performing erosion operation on regions in the first set using a first erosion convolution kernel; performing erosion operation on regions in the second set using a second erosion convolution kernel; wherein the first receptive field and the first erosion convolution kernel are determined based on the first convolution kernel; the second receptive field and the second erosion convolution kernel are determined based on the first and second convolution kernels; filtering the grayscale image after erosion operation to extract the edge contours of the filtered grayscale image. The step of obtaining the weight of the center frame based on the probabilities of the M frames adjacent to the center frame and the probability of the center frame is as follows: For each of the M frames adjacent to the center frame, it is determined whether the category to which the frame belongs is the same as the category to which the center frame belongs. If they are the same, the probabilities are accumulated. If they are different, they are skipped. The total probability is obtained based on the accumulated result and the probability corresponding to the center frame, and the total probability is used as the weight. The step of determining the keyframe of a category based on the clarity and weight of the candidate frames specifically involves: calculating the clarity of each candidate frame, sorting the candidate frames according to the product of the clarity and the weight, and taking the candidate frame ranked first as the keyframe.

2. The method as described in claim 1, characterized in that, The calculation of the sharpness of each candidate frame is specifically as follows: Candidate frames are convolved using a Laplacian kernel, and the variance of the convolved result is calculated. This variance is then used as the sharpness.

3. A system for automatically capturing images of key locations during electronic nasopharyngoscopy, characterized in that, The system includes the following modules: The video frame classification module is used to read the video stream of the electronic nasopharyngoscope, obtain the current frame of the video stream, convert the current frame to grayscale to obtain a grayscale image, extract the contour of the grayscale image after preprocessing, determine the category to which the current frame belongs and the probability of belonging to the category based on the contour, and continue to execute the video frame classification module for the next frame of the video stream until all frames in the video stream have been processed. The candidate frame selection module is used to determine the top N video frames with the highest probability in each category, take each of the top N video frames as the center frame, obtain the weight of the center frame based on the probability of each of the M adjacent frames before and after the center frame and the probability of the center frame, and filter the top N video frames according to the weight to obtain multiple candidate frames corresponding to each category; where N and M are positive integers greater than 1. The key position image capture module is used to determine the key frame of the category based on the clarity and weight of the candidate frame, and save the image corresponding to the key frame as the key position image. The extraction of grayscale image contours after grayscale image preprocessing specifically involves: convolving the grayscale image with a first convolution kernel to obtain a first convolution result; convolving the first convolution result with a second convolution kernel to obtain a second convolution result; identifying pixels with pixel values ​​greater than a preset value in the first convolution result; determining a first set of regions for erosion operation based on the position of the pixels and a first receptive field; identifying pixels with pixel values ​​greater than the preset value in the second convolution result; determining a second set of regions for erosion operation based on the position of the pixels and a second receptive field; if regions in the first set and regions in the second set intersect, deleting the intersecting first region from the first set; performing erosion operation on regions in the first set using a first erosion convolution kernel; performing erosion operation on regions in the second set using a second erosion convolution kernel; wherein the first receptive field and the first erosion convolution kernel are determined based on the first convolution kernel; the second receptive field and the second erosion convolution kernel are determined based on the first and second convolution kernels; filtering the grayscale image after erosion operation to extract the edge contours of the filtered grayscale image. The step of obtaining the weight of the center frame based on the probabilities of the M frames adjacent to the center frame and the probability of the center frame is as follows: For each of the M frames adjacent to the center frame, it is determined whether the category to which the frame belongs is the same as the category to which the center frame belongs. If they are the same, the probabilities are accumulated. If they are different, they are skipped. The total probability is obtained based on the accumulated result and the probability corresponding to the center frame, and the total probability is used as the weight. The step of determining the keyframe of a category based on the clarity and weight of the candidate frames specifically involves: calculating the clarity of each candidate frame, sorting the candidate frames according to the product of the clarity and the weight, and taking the candidate frame ranked first as the keyframe.

4. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-2.

Images