Method and apparatus for identifying abnormal nasopharyngolaryngoscope images

Abstract

The present application provides a method and device for identifying abnormal nasopharyngolaryngoscope images. The method for identifying abnormal nasopharyngolaryngoscope images includes: obtaining a first nasopharyngolaryngoscope image when withdrawing the nasopharyngolaryngoscope; obtaining a first part category of the first nasopharyngolaryngoscope image; determining a corresponding second part classification model based on the first part category; inputting the first nasopharyngolaryngoscope image into the second part classification model to obtain a second part category of the first nasopharyngolaryngoscope image; obtaining multiple nasopharyngolaryngoscope retention images of the second part category; obtaining at least one feature quantization value of each nasopharyngolaryngoscope retention image; obtaining a target feature quantization value of each nasopharyngolaryngoscope retention image; determining the nasopharyngolaryngoscope retention image with the largest target feature quantization value as the target retention image of the second part category; and determining an abnormal recognition result based on the target retention image of the second part category. The present application can improve the recognition accuracy of abnormal nasopharyngolaryngoscope images.

Description

Technical Field

[0001] This application mainly relates to the field of image processing technology, and particularly relates to a method and device for identifying abnormal nasopharyngolaryngoscope images. Background Art

[0002] Otolaryngology - Head and Neck Surgery is a highly specialized discipline. The organs are deep - seated and hidden, mostly small cavities, with delicate, complex anatomical structures and diverse functions. To recognize their normal morphology and pathological phenomena, special examination equipment must be used. The electronic laryngoscope examination technology is relatively easy to master, with high patient comfort and good tolerance, and can directly observe the mucosa of hollow organs. However, in the existing technology, affected by the doctor's image - saving level, visual blind spots may occur, and due to the complex anatomical structure, it is impossible to accurately identify image abnormalities.

[0003] That is, the recognition accuracy of abnormal nasopharyngolaryngoscope images in the existing technology is not high. Summary of the Invention

[0004] This application provides a method and device for identifying abnormal nasopharyngolaryngoscope images, aiming to solve the problem of low recognition accuracy of abnormal nasopharyngolaryngoscope images in the existing technology.

[0005] In the first aspect, this application provides a method for identifying abnormal nasopharyngolaryngoscope images. The method for identifying abnormal nasopharyngolaryngoscope images includes:

[0006] Obtain a first nasopharyngolaryngoscope image when the nasopharyngolaryngoscope is withdrawn;

[0007] Input the first nasopharyngolaryngoscope image into a first part classification model to obtain a first part category of the first nasopharyngolaryngoscope image;

[0008] Determine a corresponding second part classification model based on the first part category, where different first part categories correspond to different second part classification models, and the parts of each category output by the second part classification model are all located in the parts of the first part category;

[0009] Input the first nasopharyngolaryngoscope image into the second part classification model to obtain a second part category of the first nasopharyngolaryngoscope image;

[0010] Obtain multiple nasopharyngolaryngoscope saved - image images of the second part category, where the multiple nasopharyngolaryngoscope saved - image images are obtained by taking pictures of the parts of the second part category at different angles and saving the images;

[0011] Obtain at least one feature quantization value for each nasopharyngolaryngoscope saved - image image;

[0012] For any one of the endoscopic images of the nasopharynx, larynx and pharynx, determine the target feature quantization value of the endoscopic image of the nasopharynx, larynx and pharynx according to at least one feature quantization value of the endoscopic image of the nasopharynx, larynx and pharynx, and obtain the target feature quantization values of each endoscopic image of the nasopharynx, larynx and pharynx;

[0013] Determine the target retained image of the second part category as the endoscopic image of the nasopharynx, larynx and pharynx with the largest target feature quantization value;

[0014] Determine the abnormal recognition result based on the target retained image of the second part category.

[0015] Optionally, the feature quantization value includes at least one of a color feature quantization value, a texture feature quantization value, an image entropy quantization value, and an image quality quantization value. The obtaining of at least one feature quantization value of each endoscopic image of the nasopharynx, larynx and pharynx includes:

[0016] Remove the black pixel points in the endoscopic image of the nasopharynx, larynx and pharynx to obtain a plurality of pixel points after removal; obtain the average pixel values of the three channels of the plurality of pixel points after removal on the RGB three channels; determine the median of the average pixel values of the three channels as the color feature quantization value;

[0017] And / or, obtain the LBP value of each pixel point in the endoscopic image of the nasopharynx, larynx and pharynx; determine the sum of the LBP values of each pixel point as the texture feature quantization value;

[0018] And / or, obtain the gray value and the neighborhood gray mean value of each pixel point in the endoscopic image of the nasopharynx, larynx and pharynx, regard the pixel points with the same gray value and neighborhood gray mean value as the same category to obtain a plurality of categories of pixel points; determine the image entropy quantization value according to the frequency of each category of pixel points and the total number of pixel points in the endoscopic image of the nasopharynx, larynx and pharynx;

[0019] And / or, input the endoscopic image of the nasopharynx, larynx and pharynx into a pre-trained image quality scoring model to obtain the image quality quantization value.

[0020] Optionally, before obtaining at least one feature quantization value of each endoscopic image of the nasopharynx, larynx and pharynx, it includes:

[0021] Obtain a plurality of second endoscopic images of the nasopharynx, larynx and pharynx taken at a preset frequency when the endoscope is withdrawn;

[0022] Obtain the shooting time interval of multiple endoscopic retained images of the second part category;

[0023] Obtain a plurality of third endoscopic images of the nasopharynx, larynx and pharynx whose shooting time is within the shooting time interval from the plurality of second endoscopic images of the nasopharynx, larynx and pharynx;

[0024] Put each of the multiple third nasopharyngolaryngoscope images into the multiple nasopharyngolaryngoscope retained images of the second part category in sequence without taking them back and perform three-dimensional reconstruction to obtain multiple reconstructed three-dimensional models;

[0025] If the multiple three-dimensional models are the same, it is determined that the retention of the multiple nasopharyngolaryngoscope retained images is complete, and at least one feature quantization value of each nasopharyngolaryngoscope retained image is obtained.

[0026] Optionally, determining the abnormal recognition result based on the target retained image of the second part category includes:

[0027] Judge whether the second part category belongs to a preset part category, where the parts of the preset part category have symmetric parts;

[0028] If the second part category belongs to the preset part category, obtain the symmetric part category corresponding to the second part category, where the parts of the second part category and the corresponding symmetric part category are symmetric left and right;

[0029] Obtain the target retained image of the symmetric part category;

[0030] Flip the target retained image of the symmetric part category left and right to obtain a flipped image;

[0031] Input the flipped image and the target retained image of the second part category into a part segmentation model to obtain the first part segmentation region of the flipped image and the second part segmentation region of the symmetric part category;

[0032] Align the flipped image with the target retained image of the second part category, and calculate the intersection over union of the first part segmentation region and the second part segmentation region;

[0033] Determine the first symmetry coefficient between the flipped image and the target retained image of the second part category based on the intersection over union of the first part segmentation region and the second part segmentation region;

[0034] Determine the abnormal recognition result according to the first symmetry coefficient.

[0035] Optionally, the determining the abnormal recognition result according to the first symmetry coefficient includes:

[0036] Binarize each pixel point in the flipped image to 0 or 1;

[0037] Obtain the first row alternating occurrence times of the pixel values of 0 and 1 alternating in each row of the flipped image and the first column alternating occurrence times of the pixel values of 0 and 1 alternating in each column of the flipped image;

[0038] Determine a first image feature parameter according to the number of alternations in the first row and the number of alternations in the first column;

[0039] Binarize each pixel point in the target retained image of the second part category to 0 or 1;

[0040] Obtain a second row alternation count of pixel values of 0 and 1 alternating in each row of the target retained image of the second part category and a second column alternation count of pixel values of 0 and 1 alternating in each column of the target retained image of the second part category;

[0041] Determine a second image feature parameter according to the second row alternation count and the second alternation count;

[0042] Determine a second symmetry coefficient according to the first image feature parameter and the second image feature parameter;

[0043] Determine a target symmetry coefficient according to the first symmetry coefficient and the second symmetry coefficient;

[0044] Determine an abnormality recognition result according to the target symmetry coefficient.

[0045] Optionally, the determining the abnormality recognition result according to the target symmetry coefficient includes:

[0046] If the target symmetry coefficient is less than a preset value, determine that the first nasopharyngolaryngoscope image is abnormal.

[0047] Optionally, the method for recognizing an abnormal nasopharyngolaryngoscope image includes:

[0048] Perform three-dimensional reconstruction based on the plurality of second nasopharyngolaryngoscope images to obtain a nasopharynx and larynx three-dimensional model;

[0049] Unfold the nasopharynx and larynx three-dimensional model into a two-dimensional unfolded diagram;

[0050] Input the two-dimensional unfolded diagram into a lesion segmentation model and a part segmentation model respectively to obtain a lesion segmentation area and each part segmentation area in the two-dimensional unfolded diagram;

[0051] Calculate the centroid distance between the lesion segmentation area and each part segmentation area;

[0052] Determine the part segmentation area with the largest centroid distance as the part to which the lesion segmentation area belongs.

[0053] In a second aspect, the present application provides a device for recognizing an abnormal nasopharyngolaryngoscope image, and the device for recognizing an abnormal nasopharyngolaryngoscope image includes:

[0054] A first acquisition unit, configured to acquire a first nasopharyngolaryngoscope image when the nasopharyngolaryngoscope is withdrawn;

[0055] A first classification unit for inputting the first nasopharyngolaryngoscope image into a first part classification model to obtain a first part category of the first nasopharyngolaryngoscope image;

[0056] A model determination unit for determining a corresponding second part classification model based on the first part category, where different first part categories correspond to different second part classification models, and the parts of each category output by the second part classification model are all located in the parts of the first part category;

[0057] A second classification unit for inputting the first nasopharyngolaryngoscope image into the second part classification model to obtain a second part category of the first nasopharyngolaryngoscope image;

[0058] A second acquisition unit for acquiring multiple nasopharyngolaryngoscope retained images of the second part category, where the multiple nasopharyngolaryngoscope retained images are obtained by taking pictures of the parts of the second part category from different angles with a nasopharyngolaryngoscope and retaining the images;

[0059] A third acquisition unit for acquiring at least one feature quantization value of each nasopharyngolaryngoscope retained image;

[0060] A first determination unit for determining a target feature quantization value of any one of the nasopharyngolaryngoscope retained images according to the at least one feature quantization value of the nasopharyngolaryngoscope retained image, and obtaining the target feature quantization values of each nasopharyngolaryngoscope retained image;

[0061] A second determination unit for determining the nasopharyngolaryngoscope retained image with the largest target feature quantization value as the target retained image of the second part category;

[0062] An abnormality recognition unit for determining an abnormality recognition result based on the target retained image of the second part category.

[0063] In a third aspect, the present application provides a computer device, where the computer device includes:

[0064] One or more processors;

[0065] A memory; and

[0066] One or more applications, where the one or more applications are stored in the memory and configured to be executed by the processor to implement the method for recognizing abnormalities in nasopharyngolaryngoscope images described in any one of the first aspects.

[0067] Fourthly, the present application provides a computer-readable storage medium storing multiple instructions, which are suitable for being loaded by a processor to execute the steps in the method for identifying abnormal nasopharyngolaryngoscope images according to any one of the first aspects.

[0068] The present application provides a method and apparatus for identifying abnormal nasopharyngolaryngoscope images. The method for identifying abnormal nasopharyngolaryngoscope images includes: obtaining a first nasopharyngolaryngoscope image when the nasopharyngolaryngoscope is withdrawn; inputting the first nasopharyngolaryngoscope image into a first part classification model to obtain a first part category of the first nasopharyngolaryngoscope image; determining a corresponding second part classification model based on the first part category, where different first part categories correspond to different second part classification models, and the parts of each category output by the second part classification model are all located in the parts of the first part category; inputting the first nasopharyngolaryngoscope image into the second part classification model to obtain a second part category of the first nasopharyngolaryngoscope image; obtaining multiple nasopharyngolaryngoscope retention images of the second part category, where the multiple nasopharyngolaryngoscope retention images are obtained by the nasopharyngolaryngoscope taking pictures of the parts of the second part category at different angles and retaining the pictures; obtaining at least one feature quantization value of each nasopharyngolaryngoscope retention image; for any one of the nasopharyngolaryngoscope retention images, determining a target feature quantization value of the nasopharyngolaryngoscope retention image according to at least one feature quantization value of the nasopharyngolaryngoscope retention image to obtain the target feature quantization values of each nasopharyngolaryngoscope retention image; determining the nasopharyngolaryngoscope retention image with the largest target feature quantization value as the target retention image of the second part category; and determining an abnormal recognition result based on the target retention image of the second part category. The present application can improve the recognition accuracy of abnormal nasopharyngolaryngoscope images. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained without creative efforts based on these drawings.

[0070] Figure 1 is a schematic diagram of the scenario of the system for identifying abnormal nasopharyngolaryngoscope images provided by the embodiments of the present application;

[0071] Figure 2 is a schematic flowchart of an embodiment of the method for identifying abnormal nasopharyngolaryngoscope images provided by the embodiments of the present application;

[0072] Figure 3 is a schematic structural diagram of an embodiment of the apparatus for identifying abnormal nasopharyngolaryngoscope images provided by the embodiments of the present application;

[0073] Figure 4 This is a schematic structural diagram of an embodiment of a computer device provided in an embodiment of the present application. Detailed implementation manners

[0074] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present application.

[0075] In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present application, "a plurality" means two or more unless otherwise specifically defined.

[0076] In the present application, the term "exemplary" is used to mean "serving as an example, illustration, or description". Any embodiment described as "exemplary" in the present application is not necessarily to be construed as more preferred or more advantageous than other embodiments. In order for any person skilled in the art to implement and use the present application, the following description is given. In the following description, details are set forth for the purpose of explanation. It should be understood that those skilled in the art can realize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid unnecessary details from obscuring the description of the present application. Therefore, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed in the present application.

[0077] The embodiments of the present application provide a method and device for identifying abnormal nasopharyngolaryngoscope images, which will be described in detail below.

[0078] Please refer to Figure 1 , Figure 1This is a schematic diagram of the scenario of the system for identifying abnormal nasopharyngolaryngoscope images provided by an embodiment of the present application. The system for identifying abnormal nasopharyngolaryngoscope images may include a computer device 100, and an apparatus for identifying abnormal nasopharyngolaryngoscope images is integrated in the computer device 100.

[0079] In an embodiment of the present application, the computer device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiment of the present application includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. Among them, the cloud server is composed of a large number of computers or network servers based on cloud computing.

[0080] In an embodiment of the present application, the above-mentioned computer device 100 may be a general computer device or a dedicated computer device. In a specific implementation, the computer device 100 may be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, etc. The type of the computer device 100 is not limited in this embodiment.

[0081] Those skilled in the art can understand that Figure 1 the application environment shown in Figure 1 is only an application scenario of the solution of the present application, and does not constitute a limitation on the application scenario of the solution of the present application. Other application environments may also include more or fewer computer devices than Figure 1 shown in

[0082] For example, Figure 1 only 1 computer device is shown in

[0083] It should be noted that Figure 1 the schematic diagram of the scenario of the system for identifying abnormal nasopharyngolaryngoscope images shown in

[0084] First, an identification method for abnormal images of a nasopharyngolaryngoscope is provided in an embodiment of the present application. The identification method for abnormal images of a nasopharyngolaryngoscope includes: obtaining a first nasopharyngolaryngoscope image when the nasopharyngolaryngoscope is withdrawn; inputting the first nasopharyngolaryngoscope image into a first part classification model to obtain a first part category of the first nasopharyngolaryngoscope image; determining a corresponding second part classification model based on the first part category, where different first part categories correspond to different second part classification models, and the parts of each category output by the second part classification model are all located in the part of the first part category; inputting the first nasopharyngolaryngoscope image into the second part classification model to obtain a second part category of the first nasopharyngolaryngoscope image; obtaining multiple retained nasopharyngolaryngoscope images of the second part category, where the multiple retained nasopharyngolaryngoscope images are obtained by taking pictures of the part of the second part category with the nasopharyngolaryngoscope at different angles and retaining the images; obtaining at least one feature quantization value for each retained nasopharyngolaryngoscope image; for any one of the retained nasopharyngolaryngoscope images, determining a target feature quantization value of the retained nasopharyngolaryngoscope image according to at least one feature quantization value of the retained nasopharyngolaryngoscope image, and obtaining the target feature quantization values of each retained nasopharyngolaryngoscope image; determining the retained nasopharyngolaryngoscope image with the largest target feature quantization value as the target retained image of the second part category; and determining an abnormal identification result based on the target retained image of the second part category.

[0085] As Figure 2 shown, Figure 2 is a schematic flowchart of an embodiment of the identification method for abnormal images of a nasopharyngolaryngoscope in an embodiment of the present application. The identification method for abnormal images of a nasopharyngolaryngoscope includes the following steps S201 to S209:

[0086] S201. Obtain a first nasopharyngolaryngoscope image when the nasopharyngolaryngoscope is withdrawn.

[0087] S202. Input the first nasopharyngolaryngoscope image into a first part classification model to obtain a first part category of the first nasopharyngolaryngoscope image.

[0088] In an embodiment of the present application, a first part classification model that can classify 7 large parts is trained. Unet++ is preferably selected. The determination method for the boundaries of each large part: The boundary of each large part is determined according to the regional boundary surrounded by the small parts included in each large part. The boundary of the posterior naris is outlined by a professional nasopharyngolaryngoscope endoscopist. The 7 large parts are respectively the nasal cavity; the posterior naris; the nasopharynx; the oropharynx; the hypopharynx; the larynx; and the oral cavity. Then the first part category of the first nasopharyngolaryngoscope image can be any one of the nasal cavity category; the posterior naris category; the nasopharynx category; the oropharynx category; the hypopharynx category; the larynx category; and the oral cavity category. Of course, the first part classification model is not limited to classifying 7 large parts, and can also classify more or fewer large parts.

[0089] For example, the first part category of the first nasopharyngolaryngoscope image is the nasal cavity category.

[0090] S203. Determine the corresponding second part classification model based on the first part category.

[0091] Among them, different first part categories correspond to different second part classification models, and the parts of each category output by the second part classification model are all located in the parts of the first part category.

[0092] For example, when the first part category is the nasal cavity, the corresponding second part classification model is used to classify parts such as the nasal concha and nasal septum under the nasal cavity. Specifically, when training the second part classification models corresponding to each first part category, Unet++ is preferably selected, and the boundaries of each small part are outlined by professional nasopharyngolaryngoscope physicians.

[0093] Specifically, the 7 second part classification models corresponding to 7 first part categories can classify 26 small parts. The 26 small parts are: nasal concha, nasal septum; torus tubarius, pharyngeal recess, pharyngotympanic tube opening, roof wall, posterior wall, nasopharyngeal surface of soft palate; soft palate, tonsil, posterior wall of oropharynx, pharyngoepiglottic fold, root of tongue; piriform fossa, posterior wall, postcricoid area; aryepiglottic fold, arytenoid area, laryngeal surface of epiglottis, lingual surface, vallecula epiglottica, vocal cord; hard palate, buccal mucosa, gingiva, retromolar area. Of course, the 7 second part classification models are not limited to classifying 26 small parts, and can also classify more or fewer small parts.

[0094] S204. Input the first nasopharyngolaryngoscope image into the second part classification model to obtain the second part category of the first nasopharyngolaryngoscope image.

[0095] For example, the second part category of the second nasopharyngolaryngoscope image is the nasal concha category.

[0096] S205. Obtain multiple nasopharyngolaryngoscope retained images of the second part category.

[0097] Among them, the multiple nasopharyngolaryngoscope retained images are obtained by taking pictures of the parts of the second part category from different angles with a nasopharyngolaryngoscope and retaining the images.

[0098] Specifically, when the nasopharyngolaryngoscope is withdrawn, multiple second nasopharyngolaryngoscope images are taken at a preset frequency. The doctor will observe multiple second nasopharyngolaryngoscope images and select some for retention. For example, when the second part category of the first nasopharyngolaryngoscope image is obtained, it indicates that the nasopharyngolaryngoscope has entered the nasal concha part, and a retention instruction is issued. After receiving the retention instruction, the doctor retains the images of the parts of the second part category, that is, multiple nasopharyngolaryngoscope retained images of the second part category are obtained.

[0099] S206. Obtain at least one feature quantization value for each nasopharyngolaryngoscope retained image.

[0100] In the embodiment of the present application, the feature quantization value includes at least one of a color feature quantization value, a texture feature quantization value, an image entropy quantization value, and an image quality quantization value, and obtaining at least one feature quantization value of each nasopharyngeal laryngoscope image includes:

[0101] (1) Remove the black pixels in the image of the nasopharyngeal laryngoscope to obtain a plurality of removed pixels.

[0102] Specifically, the getcolors() method provided by PIL is used to obtain all color features in the nasopharyngeal laryngoscope images at various angles, which are recorded as the primary color feature set. , remove the black pixels in the primary color feature set, obtain multiple removed pixels, and obtain the intermediate color feature set of multiple removed pixels .

[0103] (2) Obtain the average pixel values ​​of the three RGB channels of multiple eliminated pixels.

[0104] Find the intermediate color feature set The average value of the three channel pixels in the RGB channels.

[0105] (3) The median of the average values ​​of the pixels in the three channels is determined as the color feature quantization value.

[0106] The median of the average values ​​of the three channel pixels is used as the color feature quantization value .

[0107] Further, at least one characteristic quantization value of each nasopharyngeal laryngoscope image is obtained, including:

[0108] (1) Obtain the LBP value of each pixel in the nasopharyngeal laryngoscope image.

[0109] (2) The sum of the LBP values ​​of each pixel is determined as the texture feature quantization value.

[0110] Texture feature quantification: The circular LBP operator in the LBP (Local Binary Pattern) local binary pattern is used to extract image texture features, and the LBP value of each pixel in the nasopharyngeal laryngoscope image is obtained. The calculation method of the texture feature quantization value LBP is as follows:

[0111]

[0112] For a given pixel Its neighborhood pixel position is ,

[0113]

[0114] , where R is the sampling radius, p is the p-th sampling point, and P is the number of samplings. .

[0115] Furthermore, at least one feature quantization value of each laryngoscope retained image is obtained, including:

[0116] (1) Obtain the gray values and neighborhood gray means of each pixel point in the laryngoscope retained image, and regard the pixel points with the same gray value and neighborhood gray mean as the same category to obtain pixel points of multiple categories.

[0117] Select the neighborhood gray mean of the laryngoscope retained image as the spatial feature quantity of the gray distribution, and form a feature binary group with the pixel gray value of the laryngoscope retained image, denoted as , where i represents the gray value of the pixel point and j represents the neighborhood gray mean of the pixel point. Pixel points of multiple categories can be obtained.

[0118] (2) Determine the image entropy quantization value according to the pixel point frequency of each category and the total number of pixel points in the laryngoscope retained image.

[0119] Image entropy quantization value satisfies the following formula:

[0120]

[0121]

[0122] where the pixel point frequency is the frequency of the feature binary group appearing, and N is the pixel points of the laryngoscope retained image.

[0123] Further, at least one feature quantization value of each laryngoscope retained image is obtained, including: inputting the laryngoscope retained image into a pre-trained image quality scoring model to obtain an image quality quantization value .

[0124] Further, in order to determine whether the image left by the doctor can observe all parts and avoid blind spots, before obtaining at least one feature quantization value of each laryngoscope retained image, it includes:

[0125] (1) Obtain multiple second laryngoscope images taken at a preset frequency when the laryngoscope is withdrawn.

[0126] (2) Obtain the shooting time interval of multiple laryngoscope retained images of the second part category.

[0127] Multiple laryngoscope retained images are selected by the doctor from multiple second laryngoscope images for later display and recognition.

[0128] (3) Obtain multiple third nasopharyngolaryngoscope images whose shooting times are within the shooting time interval from multiple second nasopharyngolaryngoscope images.

[0129] (4) Sequentially place each of the multiple third nasopharyngolaryngoscope images into multiple retained nasopharyngolaryngoscope images of the second part category without taking them back, and perform three-dimensional reconstruction to obtain multiple reconstructed three-dimensional models.

[0130] Specifically, obtain one of the multiple third nasopharyngolaryngoscope images, place the selected third nasopharyngolaryngoscope image into multiple retained nasopharyngolaryngoscope images to obtain multiple retained nasopharyngolaryngoscope images and one third nasopharyngolaryngoscope image, and perform three-dimensional reconstruction based on the multiple retained nasopharyngolaryngoscope images and one third nasopharyngolaryngoscope image to obtain one three-dimensional model; then obtain one of the remaining multiple third nasopharyngolaryngoscope images, place the selected third nasopharyngolaryngoscope image into multiple retained nasopharyngolaryngoscope images and one third nasopharyngolaryngoscope image to obtain multiple retained nasopharyngolaryngoscope images and two third nasopharyngolaryngoscope images, and perform three-dimensional reconstruction based on the multiple retained nasopharyngolaryngoscope images and two third nasopharyngolaryngoscope images to obtain one three-dimensional model; multiple three-dimensional reconstruction models can be obtained through multiple placements without taking them back.

[0131] (5) If the multiple three-dimensional models are the same, it is determined that the multiple retained nasopharyngolaryngoscope images are completely retained, and at least one feature quantization value of each retained nasopharyngolaryngoscope image is obtained.

[0132] If the multiple three-dimensional models are the same, it indicates that adding the third nasopharyngolaryngoscope image will not affect the three-dimensional model reconstructed from the multiple retained nasopharyngolaryngoscope images. The three-dimensional model reconstructed from the multiple retained nasopharyngolaryngoscope images is a complete model, and all positions of the part can be observed on the multiple retained nasopharyngolaryngoscope images without blind spots. It is determined that the multiple retained nasopharyngolaryngoscope images are completely retained, and at least one feature quantization value of each retained nasopharyngolaryngoscope image is obtained.

[0133] S207. For any one of the retained nasopharyngolaryngoscope images, determine the target feature quantization value of the retained nasopharyngolaryngoscope image according to at least one feature quantization value of the retained nasopharyngolaryngoscope image, and obtain the target feature quantization values of each retained nasopharyngolaryngoscope image.

[0134] Specifically, perform weighted summation on at least one feature quantization value according to a preset weight coefficient to obtain the target feature quantization value of the retained nasopharyngolaryngoscope image .

[0135] Target feature quantization value The calculation formula is as follows:

[0136] .

[0137] S208. Determine the endoscopic image of the nasopharynx, larynx, and pharynx with the largest target feature quantization value as the target image of the second part category.

[0138] The largest target feature quantization value indicates that this captured image is the best. Determining the endoscopic image of the nasopharynx, larynx, and pharynx with the largest target feature quantization value as the target image of the second part category can facilitate better display and recognition.

[0139] S209. Determine the abnormal recognition result based on the target image of the second part category.

[0140] In the embodiments of this application, determining the abnormal recognition result based on the target image of the second part category may include:

[0141] (1) Determine whether the second part category belongs to a preset part category, where the parts of the preset part category are symmetric.

[0142] The preset part category is set in advance. For example, the preset part category may be the left tonsil part category, the right tonsil part category, the left pharyngoepiglottic fold category, the right pharyngoepiglottic fold category, the left pyriform sinus category, and the right pyriform sinus category.

[0143] (2) If the second part category belongs to the preset part category, obtain the corresponding symmetric part category of the second part category, where the parts of the second part category and the corresponding symmetric part category are symmetric left and right.

[0144] For example, if the second part category is the left tonsil part category and the second part category belongs to the preset part category, the corresponding symmetric part category of the second part category is the right tonsil part category.

[0145] (3) Obtain the target image of the symmetric part category.

[0146] (4) Flip the target image of the symmetric part category left and right to obtain a flipped image.

[0147] (5) Input the flipped image and the target image of the second part category into the part segmentation model to obtain the first part segmentation region of the flipped image and the second part segmentation region of the symmetric part category.

[0148] Specifically, when training the part segmentation model, Unet++ is preferably selected, and the labels are outlined by professional nasopharyngeal and laryngeal endoscopic physicians for the boundaries of each small part.

[0149] (6) Align the flipped image with the target image of the second part category, and calculate the intersection over union of the first part segmentation region and the second part segmentation region.

[0150] (7) Determine the first symmetry coefficient between the flipped image and the target retained image of the second part category based on the intersection over union (IoU) of the first part segmentation region and the second part segmentation region.

[0151] In a specific embodiment, the intersection over union is determined as the first symmetry coefficient.

[0152] (8) Determine the abnormal recognition result according to the first symmetry coefficient.

[0153] In a specific embodiment, if the first symmetry coefficient is not greater than a preset value, indicating a high degree of asymmetry between the two parts, it is determined that the first nasopharyngolaryngoscope image is abnormal.

[0154] In another specific embodiment, determining the abnormal recognition result according to the first symmetry coefficient includes:

[0155] (1) Binarize each pixel point in the flipped image to 0 or 1.

[0156] (2) Obtain the first row alternating occurrence times of the pixel values of 0 and 1 alternating in each row of the flipped image and the first column alternating occurrence times of the pixel values of 0 and 1 alternating in each column of the flipped image.

[0157] For example, if the pixel values of a certain row in the flipped image are 000000010 respectively, the first row alternating occurrence times is 2.

[0158] (3) Determine the first image feature parameter according to the first row alternating occurrence times and the first column alternating occurrence times.

[0159] In a specific embodiment, the first row alternating occurrence times is Wi, and the first column alternating occurrence times is Hj. The first image feature parameter is as shown in the following formula

[0160]

[0161] (4) Binarize each pixel point in the target retained image of the second part category to 0 or 1.

[0162] (5) Obtain the second row alternating occurrence times of the pixel values of 0 and 1 alternating in each row of the target retained image of the second part category and the second column alternating occurrence times of the pixel values of 0 and 1 alternating in each column of the target retained image of the second part category.

[0163] (6) Determine the second image feature parameter according to the second row alternating occurrence times and the second alternating occurrence times.

[0164] Similar to the calculation method of the first image feature parameter Calculate the second image feature parameter 。

[0165] (7) Determine the second symmetry coefficient according to the first image feature parameter and the second image feature parameter.

[0166] Specifically, the second symmetry coefficient and the first image feature parameter and the second image feature parameter are related as shown in the following formula:

[0167] 。

[0168] (8) Determine the target symmetry coefficient according to the first symmetry coefficient and the second symmetry coefficient.

[0169] Specifically, the target symmetry coefficient is calculated according to the following formula:

[0170]

[0171] (9) Determine the abnormal recognition result according to the target symmetry coefficient.

[0172] Specifically, when the target symmetry coefficient is not less than the preset value, it indicates left - right symmetry and the first nasopharyngolaryngoscope image is normal; if the target symmetry coefficient is less than the preset value, it is determined that the first nasopharyngolaryngoscope image is abnormal. The target symmetry coefficient satisfying indicates left - right symmetry and the first nasopharyngolaryngoscope image is normal, otherwise it is asymmetrical and the first nasopharyngolaryngoscope image is abnormal.

[0173] Furthermore, the method for recognizing abnormal nasopharyngolaryngoscope images further includes:

[0174] (1) Perform three - dimensional reconstruction on multiple second nasopharyngolaryngoscope images to obtain a nasopharyngolaryngeal three - dimensional model;

[0175] (2) Unfold the nasopharyngolaryngeal three - dimensional model into a two - dimensional unfolded diagram;

[0176] (3) Input the two - dimensional unfolded diagram into a lesion segmentation model and a part segmentation model respectively to obtain the lesion segmentation region and each part segmentation region in the two - dimensional unfolded diagram;

[0177] (4) Calculate the centroid distance between the lesion segmentation region and each part segmentation region;

[0178] (5) Determine the part to which the lesion segmentation region belongs as the part segmentation region with the largest centroid distance.

[0179] Furthermore, output the part to which the lesion segmentation region belongs, the target retained images of each second part category, and the target retained images of each lesion segmentation region.

[0180] When the otorhinolaryngology endoscopist starts the examination, load the trained large-part segmentation model, that is, the first-part classification model. When the nasal cavity is recognized:

[0181] Load the trained small-part segmentation model. When the nasal concha is recognized, load the trained recognition model for whether the nasal concha is swollen. If it prompts "swollen", the endoscopist needs to observe carefully whether the endoscope can pass smoothly. If not, exit; otherwise, it means that the endoscope can continue to be inserted. According to the methods of S201 - S209, select the target image for retention corresponding to the second-part category of the nasal concha for retention;

[0182] When the nasal septum is recognized, select the target image for retention corresponding to the second-part category of the nasal septum for retention;

[0183] During the process of inserting the endoscope, for the trained lesion segmentation model, when a lesion is recognized:

[0184] According to the methods of S201 - S209, select the target image for retention of the lesion for retention;

[0185] Describe the lesion site.

[0186] Score the observation of the nasal cavity part: The full score is 10 points.

[0187] After the posterior naris is recognized, select the target image for retention corresponding to the second-part category of the posterior naris for retention and determine the lesion position. The full score for the part observation is 5 points.

[0188] After the nasopharynx is recognized, automatically prompt the patient to close the mouth and inhale. Select the target image for retention corresponding to the second-part category of the posterior nasopharynx for retention and determine the lesion position. The full score for the part observation is 20 points.

[0189] After the oropharynx is recognized,

[0190] Insert the endoscope through the oral cavity, prompt the patient to say "ee", examine the soft palate and tonsils, select the target image for retention corresponding to the second-part category of the posterior soft palate and tonsils for retention and determine the lesion position, and judge whether the left and right tonsils are symmetric.

[0191] Insert the endoscope through the nasal cavity, prompt the patient to say "ee", examine the posterior wall of the oropharynx, pharyngoepiglottic fold, and root of the tongue, select the target image for retention corresponding to the second-part category of the posterior wall of the oropharynx, pharyngoepiglottic fold, and root of the tongue for retention and determine the lesion position, and judge whether the left and right pharyngoepiglottic folds are symmetric; The full score for the oropharynx is 20 points.

[0192] After the hypopharynx is recognized, prompt the patient to say "ee", select the target image for retention corresponding to the second-part category of the hypopharynx for retention and determine the lesion position. The full score for the part observation is 10 points, and judge whether the left and right pyriform sinuses are symmetric.

[0193] After the larynx is recognized, select the target retention image of the second part category corresponding to the larynx for retention and determine the lesion location. The full score for the part observation score is 20 points.

[0194] After the oral cavity is recognized, perform endoscopy through the oral cavity, select the target retention image of the second part category corresponding to the oral cavity for retention and determine the lesion location. The full score for the part observation score is 15 points.

[0195] To better implement the method for identifying abnormal nasopharyngolaryngoscope images in the embodiments of the present application, based on the method for identifying abnormal nasopharyngolaryngoscope images, an apparatus for identifying abnormal nasopharyngolaryngoscope images is further provided in the embodiments of the present application. As Figure 3 shown, the apparatus 300 for identifying abnormal nasopharyngolaryngoscope images includes:

[0196] A first acquisition unit 301, configured to acquire a first nasopharyngolaryngoscope image when the nasopharyngolaryngoscope is withdrawn;

[0197] A first classification unit 302, configured to input the first nasopharyngolaryngoscope image into a first part classification model to obtain a first part category of the first nasopharyngolaryngoscope image;

[0198] A model determination unit 303, configured to determine a corresponding second part classification model based on the first part category, where different first part categories correspond to different second part classification models, and the parts of each category output by the second part classification model are all located in the part of the first part category;

[0199] A second classification unit 304, configured to input the first nasopharyngolaryngoscope image into the second part classification model to obtain a second part category of the first nasopharyngolaryngoscope image;

[0200] A second acquisition unit 305, configured to acquire multiple nasopharyngolaryngoscope retention images of the second part category, where the multiple nasopharyngolaryngoscope retention images are obtained by taking pictures of the part of the second part category of the nasopharyngolaryngoscope from different angles and retaining the images;

[0201] A third acquisition unit 306, configured to acquire at least one feature quantization value of each nasopharyngolaryngoscope retention image;

[0202] A first determination unit 307, configured to, for any one of the nasopharyngolaryngoscope retention images, determine a target feature quantization value of the nasopharyngolaryngoscope retention image according to at least one feature quantization value of the nasopharyngolaryngoscope retention image, and obtain the target feature quantization values of each nasopharyngolaryngoscope retention image;

[0203] A second determination unit 308, configured to determine the nasopharyngolaryngoscope retention image with the largest target feature quantization value as the target retention image of the second part category;

[0204] Anomaly recognition unit 309 is configured to determine an anomaly recognition result based on the target retained image of the second part category.

[0205] An embodiment of the present application further provides a computer device, which integrates any one of the recognition devices for nasopharyngolaryngoscope image anomalies provided in the embodiments of the present application. The computer device includes:

[0206] One or more processors;

[0207] A memory; and

[0208] One or more applications, where one or more applications are stored in the memory and are configured to be executed by the processor to perform the steps in the method for recognizing nasopharyngolaryngoscope image anomalies in any one of the embodiments of the method for recognizing nasopharyngolaryngoscope image anomalies described above.

[0209] As Figure 4 shown, it shows a schematic structural diagram of the computer device involved in the embodiments of the present application. Specifically:

[0210] The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 of one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art can understand that Figure 4 the structure of the computer device shown in Figure 4 does not constitute a limitation on the computer device, and it may include more or fewer components than

[0211] The processor 401 is the control center of the computer device, connecting various parts of the entire computer device through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 402, and by invoking the data stored in the memory 402, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 401 may include one or more processing cores; the processor 401 may be a central processing unit (CPU), or it may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Preferably, the processor 401 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 401 either.

[0212] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, image playback function, etc.), etc.; the data storage area can store the data created according to the use of the computer device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0213] The computer device also includes a power supply 403 that powers each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 403 may also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator, etc.

[0214] The computer device may further include an input unit 404, which may be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0215] Although not shown, the computer device may further include a display unit, etc., which will not be elaborated here. Specifically, in this embodiment, the processor 401 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402 to implement various functions as follows:

[0216] Obtain a first nasopharyngolaryngoscope image when the nasopharyngolaryngoscope is withdrawn; input the first nasopharyngolaryngoscope image into a first part classification model to obtain a first part category of the first nasopharyngolaryngoscope image; determine a corresponding second part classification model based on the first part category, where different first part categories correspond to different second part classification models, and the parts of each category output by the second part classification model are all located in the parts of the first part category; input the first nasopharyngolaryngoscope image into the second part classification model to obtain a second part category of the first nasopharyngolaryngoscope image; obtain multiple nasopharyngolaryngoscope retention images of the second part category, where the multiple nasopharyngolaryngoscope retention images are obtained by the nasopharyngolaryngoscope taking pictures and retaining images of the parts of the second part category at different angles; obtain at least one feature quantization value for each nasopharyngolaryngoscope retention image; for any one of the nasopharyngolaryngoscope retention images, determine the target feature quantization value of the nasopharyngolaryngoscope retention image according to the at least one feature quantization value of the nasopharyngolaryngoscope retention image, and obtain the target feature quantization values of each nasopharyngolaryngoscope retention image; determine the nasopharyngolaryngoscope retention image with the largest target feature quantization value as the target retention image of the second part category; determine an abnormality recognition result based on the target retention image of the second part category.

[0217] Those of ordinary skill in the art can understand that all or part of the steps in the above-mentioned various methods can be completed by instructions, or by controlling relevant hardware through instructions. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0218] To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), a magnetic disk, an optical disc, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the methods for identifying abnormal nasopharyngolaryngoscope images provided by the embodiments of the present application. For example, when the computer program is loaded by the processor, the following steps may be executed:

[0219] Obtain a first nasopharyngolaryngoscope image when the nasopharyngolaryngoscope is withdrawn; input the first nasopharyngolaryngoscope image into a first part classification model to obtain a first part category of the first nasopharyngolaryngoscope image; determine a corresponding second part classification model based on the first part category, where different first part categories correspond to different second part classification models, and the parts of each category output by the second part classification model are all located in the parts of the first part category; input the first nasopharyngolaryngoscope image into the second part classification model to obtain a second part category of the first nasopharyngolaryngoscope image; obtain multiple nasopharyngolaryngoscope retained images of the second part category, where the multiple nasopharyngolaryngoscope retained images are obtained by taking pictures of the parts of the second part category at different angles with the nasopharyngolaryngoscope and retaining the images; obtain at least one feature quantization value of each nasopharyngolaryngoscope retained image; for any one of the nasopharyngolaryngoscope retained images, determine a target feature quantization value of the nasopharyngolaryngoscope retained image according to the at least one feature quantization value of the nasopharyngolaryngoscope retained image, and obtain the target feature quantization values of each nasopharyngolaryngoscope retained image; determine the nasopharyngolaryngoscope retained image with the largest target feature quantization value as the target retained image of the second part category; determine an abnormal recognition result based on the target retained image of the second part category.

[0220] In the above embodiments, the descriptions of the various embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference may be made to the detailed descriptions of other embodiments above, and details will not be repeated here.

[0221] In specific implementation, the above-mentioned units or structures may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities. For the specific implementation of the above-mentioned units or structures, reference may be made to the method embodiments above, and details will not be repeated here.

[0222] For the specific implementation of each of the above operations, reference may be made to the foregoing embodiments, and details will not be repeated here.

[0223] The above has introduced in detail a method and device for identifying abnormal images of a nasopharyngolaryngoscope. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for identifying abnormal nasopharyngolaryngoscope images, characterized in that, The method for identifying abnormal nasopharyngolaryngoscope images includes: Obtaining a first nasopharyngolaryngoscope image when the nasopharyngolaryngoscope is withdrawn; Inputting the first nasopharyngolaryngoscope image into a first part classification model to obtain a first part category of the first nasopharyngolaryngoscope image; Determining a corresponding second part classification model based on the first part category, where different first part categories correspond to different second part classification models, and the parts of each category output by the second part classification model are all located in the parts of the first part category; Inputting the first nasopharyngolaryngoscope image into the second part classification model to obtain a second part category of the first nasopharyngolaryngoscope image; Obtaining multiple nasopharyngolaryngoscope retained images of the second part category, where the multiple nasopharyngolaryngoscope retained images are obtained by taking pictures of the parts of the second part category at different angles and retaining the images; Obtaining at least one feature quantization value for each nasopharyngolaryngoscope retained image; For any one of the nasopharyngolaryngoscope retained images, determining the target feature quantization value of the nasopharyngolaryngoscope retained image according to at least one feature quantization value of the nasopharyngolaryngoscope retained image, and obtaining the target feature quantization values of each nasopharyngolaryngoscope retained image; Determining the nasopharyngolaryngoscope retained image with the largest target feature quantization value as the target retained image of the second part category; Determining an abnormal recognition result based on the target retained image of the second part category; Among them, the determining the abnormal recognition result based on the target retained image of the second part category includes: Judging whether the second part category belongs to a preset part category, where the parts of the preset part category have symmetrical parts; If the second part category belongs to the preset part category, obtaining the corresponding symmetrical part category of the second part category, where the parts of the second part category and the corresponding symmetrical part category are left-right symmetrical; Obtaining the target retained image of the symmetrical part category; Flipping the target retained image of the symmetrical part category left and right to obtain a flipped image; Inputting the flipped image and the target retained image of the second part category into a part segmentation model to obtain a first part segmentation region of the flipped image and a second part segmentation region of the symmetrical part category; Aligning the flipped image with the target retained image of the second part category, and calculating the intersection over union of the first part segmentation region and the second part segmentation region; Determining a first symmetry coefficient between the flipped image and the target retained image of the second part category based on the intersection over union of the first part segmentation region and the second part segmentation region; Determining the abnormal recognition result according to the first symmetry coefficient.

2. The method for identifying abnormal nasopharyngolaryngoscope images according to claim 1, characterized in that, The feature quantization value includes at least one of a color feature quantization value, a texture feature quantization value, an image entropy quantization value, and an image quality quantization value. The obtaining at least one feature quantization value for each nasopharyngolaryngoscope retained image includes: Removing the black pixel points in the nasopharyngolaryngoscope retained image to obtain multiple pixel points after removal; obtaining the average pixel values of the three channels of the multiple pixel points after removal on the RGB three channels; and determining the median of the average pixel values of the three channels as the color feature quantization value; And / or, obtain the LBP value of each pixel point in the retained image of the nasopharyngolaryngoscope; determine the sum of the LBP values of each pixel point as the texture feature quantization value; And / or, obtain the gray value of each pixel point and the neighborhood gray mean value in the retained image of the nasopharyngolaryngoscope, take the pixel points with the same gray value and neighborhood gray mean value as the same category to obtain pixel points of multiple categories; determine the image entropy quantization value according to the frequency of each category of pixel points and the total number of pixel points in the retained image of the nasopharyngolaryngoscope; And / or, input the retained image of the nasopharyngolaryngoscope into a pre-trained image quality scoring model to obtain an image quality quantization value.

3. The method for identifying abnormal nasopharyngolaryngoscope images according to claim 1, characterized in that, Before obtaining at least one feature quantization value of each retained image of the nasopharyngolaryngoscope, it includes: Obtain multiple second nasopharyngolaryngoscope images taken at a preset frequency when withdrawing the nasopharyngolaryngoscope; Obtain the shooting time interval of multiple retained images of the nasopharyngolaryngoscope of the second part category; Obtain multiple third nasopharyngolaryngoscope images whose shooting time is within the shooting time interval from the multiple second nasopharyngolaryngoscope images; Put each of the multiple third nasopharyngolaryngoscope images into the multiple retained images of the nasopharyngolaryngoscope of the second part category in turn without taking them back and perform three-dimensional reconstruction to obtain multiple reconstructed three-dimensional models; If the multiple three-dimensional models are the same, it is determined that the retained images of the multiple nasopharyngolaryngoscopes are complete, and at least one feature quantization value of each retained image of the nasopharyngolaryngoscope is obtained.

4. The method for identifying abnormal nasopharyngolaryngoscope images according to claim 1, characterized in that, The determining the abnormal recognition result according to the first symmetry coefficient includes: Binarize each pixel point in the flipped image to 0 or 1; Obtain the first row alternating occurrence times of the pixel values of 0 and 1 alternating in each row of the flipped image and the first column alternating occurrence times of the pixel values of 0 and 1 alternating in each column of the flipped image; Determine the first image feature parameter according to the first row alternating occurrence times and the first column alternating occurrence times; Binarize each pixel point in the target retained image of the second part category to 0 or 1; Obtain the second row alternating occurrence times of the pixel values of 0 and 1 alternating in each row of the target retained image of the second part category and the second column alternating occurrence times of the pixel values of 0 and 1 alternating in each column of the target retained image of the second part category; Determine the second image feature parameter according to the second row alternating occurrence times and the second alternating occurrence times; Determine the second symmetry coefficient according to the first image feature parameter and the second image feature parameter; Determine the target symmetry coefficient according to the first symmetry coefficient and the second symmetry coefficient; Determine the abnormal recognition result according to the target symmetry coefficient.

5. The method for identifying abnormal nasopharyngolaryngoscope images according to claim 4, wherein, The determining the abnormal recognition result according to the target symmetry coefficient includes: If the target symmetry coefficient is less than the preset value, determine that the first nasopharyngolaryngoscope image is abnormal.

6. The method for identifying abnormal nasopharyngolaryngoscope images according to claim 3, wherein, The method for recognizing abnormal nasopharyngolaryngoscope images includes: Perform three-dimensional reconstruction based on the multiple second nasopharyngolaryngoscope images to obtain a nasopharynx and larynx three-dimensional model; Unfold the nasopharynx and larynx three-dimensional model into a two-dimensional unfolded diagram; Input the two-dimensional unfolded diagram into the lesion segmentation model and the part segmentation model respectively to obtain the lesion segmentation area and each part segmentation area in the two-dimensional unfolded diagram; Calculate the centroid distance between the lesion segmentation area and each part segmentation area; Determine the part to which the lesion segmentation area belongs as the part segmentation area with the largest centroid distance.

7. An apparatus for identifying abnormal nasopharyngolaryngoscope images, wherein, The recognition device for abnormal nasopharyngolaryngoscope images includes: A first acquisition unit for acquiring a first nasopharyngolaryngoscope image when the nasopharyngolaryngoscope is withdrawn; A first classification unit for inputting the first nasopharyngolaryngoscope image into a first part classification model to obtain a first part category of the first nasopharyngolaryngoscope image; A model determination unit for determining a corresponding second part classification model based on the first part category, where different first part categories correspond to different second part classification models, and the parts of each category output by the second part classification model are all located in the part of the first part category; A second classification unit for inputting the first nasopharyngolaryngoscope image into the second part classification model to obtain a second part category of the first nasopharyngolaryngoscope image; A second acquisition unit for acquiring multiple nasopharyngolaryngoscope retained images of the second part category, where the multiple nasopharyngolaryngoscope retained images are obtained by taking pictures of the part of the second part category at different angles and retaining the images; A third acquisition unit for acquiring at least one feature quantization value of each nasopharyngolaryngoscope retained image; A first determination unit for determining a target feature quantization value of any one of the nasopharyngolaryngoscope retained images according to at least one feature quantization value of the nasopharyngolaryngoscope retained image, and obtaining the target feature quantization values of each nasopharyngolaryngoscope retained image; A second determination unit for determining the nasopharyngolaryngoscope retained image with the largest target feature quantization value as the target retained image of the second part category; An abnormality recognition unit for determining an abnormality recognition result based on the target retained image of the second part category; Among them, determining the abnormality recognition result based on the target retained image of the second part category includes: Judging whether the second part category belongs to a preset part category, where the parts of the preset part category have symmetrical parts; If the second part category belongs to the preset part category, then obtain the corresponding symmetrical part category of the second part category, where the parts of the second part category and the corresponding symmetrical part category are symmetrical left and right; Obtain the target retained image of the symmetrical part category; Flip the target retained image of the symmetrical part category left and right to obtain a flipped image; Input the flipped image and the target retained image of the second part category into a part segmentation model to obtain a first part segmentation area of the flipped image and a second part segmentation area of the symmetrical part category; Align the flipped image with the target retained image of the second part category, and calculate the intersection over union of the first part segmentation area and the second part segmentation area; Determine a first symmetry coefficient between the flipped image and the target retained image of the second part category based on the intersection over union of the first part segmentation area and the second part segmentation area; Determine the abnormal recognition result according to the first symmetry coefficient.

8. A computer device, wherein, The computer device includes: One or more processors; A memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method for recognizing abnormalities in nasopharyngolaryngoscope images according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and the computer program is loaded by the processor to execute the steps in the method for recognizing abnormalities in nasopharyngolaryngoscope images according to any one of claims 1 to 6.

Images