Otoscopic examination monitoring method and device, computer equipment and readable storage medium

Abstract

An embodiment of the present application provides a monitoring method and device for otoscope examination, a computer device and a readable storage medium. The monitoring method for otoscope examination records the first moment and the area of the external auditory canal entrance, the second moment and the effective area inside the ear when the external auditory canal, light cone and / or malleus are identified in sequence, and different types of segmentation models are used to segment the abnormal area in the otoscope examination image, and the third moment when the abnormal area appears, the area of the abnormal area and the abnormal image data are determined. The mirror advancement parameters can be determined according to the first moment, the second moment and the mirror advancement speed, and the abnormal parameters inside the ear can be determined according to the first moment, the third moment, the mirror advancement speed, the area of the external auditory canal entrance, the area of the abnormal area and the effective area inside the ear. Prompts can be given based on the abnormal image data, the mirror advancement parameters and the abnormal parameters inside the ear, thereby realizing the monitoring and prompting of the mirror advancement operation and the abnormality inside the ear during the otoscope examination process, and avoiding the problem of missed detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a monitoring method and apparatus for otoscope examination, a computer device, and a readable storage medium. Background Art

[0002] An otoscope is a medical instrument that is inserted into the ear to observe the condition inside. When using an otoscope to examine the human body, the doctor must understand the anatomy of the ear canal and gently insert the otoscope along the curvature of the external auditory canal deep into the ear canal to check for impurities such as hair and perforations. This makes the otoscope examination completely dependent on the doctor, and during this process, there is a risk of missed or false detections due to poor doctor condition or excessive speed.

[0003] Therefore, the existing otoscopy examination method has a technical problem of being too dependent on doctors, which may lead to missed detections. Summary of the Invention

[0004] The embodiments of the present application provide a monitoring method and apparatus for otoscope examination, a computer device, and a readable storage medium, to alleviate the technical problem that existing otoscope examination methods are too dependent on doctors, resulting in possible missed detections.

[0005] The present invention provides a method for monitoring otoscopy, which includes:

[0006] receiving an inspection instruction carrying an inspection identifier, and acquiring otoscope inspection data when the otoscope is inspecting inside the ear based on the inspection identifier;

[0007] extracting an otoscopic examination image based on the otoscopic examination data;

[0008] Recognize the otoscope examination image, and when the external auditory canal is identified, monitor the speed of the otoscope advancement in real time, and determine the first moment when the external auditory canal is identified and the area of the external auditory canal entrance;

[0009] Identifying the otoscope image, and when the tympanic membrane is identified, identifying the light cone and / or the malleus, and when the light cone and / or the malleus are identified, determining a second moment at which the light cone and / or the malleus are identified and an effective area within the ear;

[0010] Segmenting the abnormal region in the otoscope examination image using different types of segmentation models, and determining a third moment of occurrence of the abnormal region, an area of the abnormal region, and abnormal image data;

[0011] determining a mirror advancement parameter according to the first moment, the second moment, and the mirror advancement speed;

[0012] Determining an abnormality parameter in the ear according to the first moment, the third moment, the scope advancement speed, the area of the external auditory canal entrance, the area of the abnormal region, and the effective area in the ear;

[0013] Prompts are given based on the abnormal image data, the scope insertion parameters and the abnormal parameters inside the ear.

[0014] At the same time, an embodiment of the present application provides a monitoring device for otoscope examination, the monitoring device for otoscope examination comprising:

[0015] a receiving module, configured to receive an inspection instruction carrying an inspection identifier, and obtain otoscope inspection data when the otoscope is inspecting inside the ear based on the inspection identifier;

[0016] an extraction module, configured to extract an otoscope examination image based on the otoscope examination data;

[0017] a first recognition module, configured to recognize the otoscope examination image, and when the external auditory canal is recognized, monitor the speed of the otoscope advancement in real time, and determine a first moment and an area of the external auditory canal entrance;

[0018] a second recognition module for recognizing the otoscope examination image, and when the tympanic membrane is recognized, recognizing the light cone and / or the malleus, and determining a second moment and an effective area in the ear when the light cone and / or the malleus are recognized;

[0019] a segmentation module for segmenting the abnormal region in the otoscope examination image using different types of segmentation models, and determining a third moment of appearance of the abnormal region, an area of the abnormal region, and abnormal image data;

[0020] a first determining module, configured to determine a mirror advancing parameter according to the first moment, the second moment, and the mirror advancing speed;

[0021] a second determining module, configured to determine an abnormality parameter in the ear according to the first moment, the third moment, the scope advancement speed, the area of the external auditory canal entrance, the area of the abnormal region, and the effective area in the ear;

[0022] A prompt module is used to provide prompts based on the abnormal image data, the scope insertion parameters and the abnormal parameters in the ear.

[0023] At the same time, an embodiment of the present application provides a computer device, which includes:

[0024] one or more processors;

[0025] Memory; and

[0026] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method for monitoring otoscopy as described in any of the above embodiments.

[0027] At the same time, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is loaded by a processor to execute the steps in the otoscope examination monitoring method described in any of the above embodiments.

[0028] Beneficial effect: The embodiment of the present application provides a monitoring method and device for otoscope examination, a computer device and a readable storage medium. After receiving an inspection instruction carrying an inspection identifier, the monitoring method for otoscope examination obtains otoscope examination data when the otoscope is inspecting in the ear based on the inspection identifier, and extracts an otoscope examination image based on the otoscope examination data, and then recognizes the otoscope examination image, and when the external auditory canal is recognized, monitors the speed of the mirror advancement in real time, and determines the first moment of recognition of the external auditory canal and the area of the entrance of the external auditory canal, and then recognizes the otoscope examination image, and when the tympanic membrane is recognized, recognizes the light cone and / or the malleus, and When the light cone and / or malleus is identified, the second moment at which the light cone and / or malleus are identified and the effective area in the ear are determined. At the same time, different types of segmentation models are used to segment the abnormal area in the otoscope examination image, and the third moment at which the abnormal area appears, the area of the abnormal area and the abnormal image data are determined. The mirror advancement parameters can be determined based on the first moment, the second moment and the mirror advancement speed. The abnormal parameters in the ear can be determined based on the first moment, the third moment, the mirror advancement speed, the area of the external auditory canal entrance, the area of the abnormal area and the effective area in the ear. Prompts can be given based on the abnormal image data, the mirror advancement parameters and the abnormal parameters in the ear. The present application records the first moment and the area of the external auditory canal entrance, the second moment and the effective area inside the ear when the external auditory canal, light cone and / or malleus are identified in sequence, and uses different types of segmentation models to segment the abnormal area in the otoscope examination image, and determines the third moment when the abnormal area appears, the area of the abnormal area and the abnormal image data. The mirror advancement parameters can be determined according to the first moment, the second moment and the mirror advancement speed, and the abnormal parameters inside the ear can be determined according to the first moment, the third moment, the mirror advancement speed, the area of the external auditory canal entrance, the area of the abnormal area and the effective area inside the ear. Prompts can be given based on the abnormal image data, the mirror advancement parameters and the abnormal parameters inside the ear, thereby realizing the monitoring and prompts of the mirror advancement operation and abnormalities inside the ear during the otoscope examination process, and avoiding the problem of missed detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The following detailed description of the specific embodiments of the present application in conjunction with the accompanying drawings will make the technical solutions and other beneficial effects of the present application apparent.

[0030] Figure 1 A schematic diagram of the structure of the monitoring system for otoscope examination provided in an embodiment of the present application.

[0031] Figure 2 A flowchart of a method for monitoring otoscopy provided in an embodiment of the present application.

[0032] Figure 3 A schematic diagram of the structure of an ear provided in an embodiment of the present application.

[0033] Figure 4 A comparison diagram of a normal tympanic membrane and a perforated tympanic membrane provided in an embodiment of the present application.

[0034] Figure 5 This is a structural diagram of the eardrum provided in an embodiment of the present application.

[0035] Figure 6 This is a structural diagram of the monitoring device for otoscope examination provided in an embodiment of the present application.

[0036] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0038] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0039] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that one of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0040] The embodiments of the present application provide a monitoring method and apparatus for otoscope examination, a computer device, and a readable storage medium, which are described in detail below.

[0041] See also Figure 1 , Figure 1 This is a scene diagram of the otoscope examination monitoring system provided in an embodiment of the present application. The otoscope examination monitoring system may include a computer device 100, in which an otoscope examination monitoring device is integrated.

[0042] In the embodiments of the present application, the computer device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiments of the present application includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. A cloud server is composed of a large number of computers or network servers based on cloud computing.

[0043] In the embodiment of the present application, the computer device 100 can be a general-purpose computer device or a dedicated computer device. In a specific implementation, the computer device 100 can be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, etc. This embodiment does not limit the type of the computer device 100.

[0044] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer computer devices as shown in Figure 1Only one computer device is shown in the figure. It can be understood that the otoscope examination monitoring system can also include one or more other computer devices that can process data, which are not limited here.

[0045] In addition, if Figure 1 As shown, the otoscopy monitoring system may further include a storage unit 200 for storing data.

[0046] It should be noted that Figure 1 The scenario diagram of the otoscope examination monitoring system shown is merely an example. The otoscope examination monitoring system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art will appreciate that, with the evolution of the otoscope examination monitoring system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0047] First, an embodiment of the present application provides a method for monitoring otoscope examination, which includes: receiving an inspection instruction carrying an inspection identifier, and obtaining otoscope examination data when the otoscope is inspecting in the ear based on the inspection identifier; extracting an otoscope examination image based on the otoscope examination data; identifying the otoscope examination image, and when the external auditory canal is identified, monitoring the mirror advancement speed in real time, and determining the first moment of identifying the external auditory canal and the entrance area of the external auditory canal; identifying the otoscope examination image, and when the tympanic membrane is identified, identifying the light cone and / or the malleus, and when the light cone and / or the malleus are identified, determining the identification time. to the second moment of the light cone and / or the malleus and the effective area in the ear; use different types of segmentation models to segment the abnormal area in the otoscope examination image, and determine the third moment when the abnormal area appears, the area of the abnormal area and the abnormal image data; determine the mirror advancement parameters according to the first moment, the second moment and the mirror advancement speed; determine the abnormal parameters in the ear according to the first moment, the third moment, the mirror advancement speed, the entrance area of the external auditory canal, the area of the abnormal area and the effective area in the ear; and provide prompts based on the abnormal image data, the mirror advancement parameters and the abnormal parameters in the ear.

[0048] like Figure 2 As shown, Figure 2 1 is a flow chart of an embodiment of a method for monitoring otoscopy in an embodiment of the present application. The method for monitoring otoscopy includes the following steps S201 to S208:

[0049] S201: receiving an inspection instruction carrying an inspection identifier, and acquiring otoscope inspection data when the otoscope is inspecting inside the ear based on the inspection identifier.

[0050] Specifically, the inspection instruction refers to an instruction for performing an otoscope examination or an instruction for monitoring an otoscope examination. The inspection instruction includes instructions input in a variety of ways. For example, the otoscope can communicate with or physically connect the otoscope to a computer device to achieve information interaction. The inspection instruction can be input by clicking on the otoscope or entering it into the computer device, including but not limited to voice input, fingerprint input, and key input, so that the otoscope examination process can be monitored according to the inspection instruction.

[0051] Specifically, the inspection identifier refers to the identifier carried by the inspection instruction. This identifier can determine the inspection to be performed by the inspection instruction. Accordingly, it can directly or indirectly determine the processing method and data to be processed, so that corresponding processing can be performed according to the inspection identifier. The inspection identifier can also be determined by parsing the inspection instruction.

[0052] Specifically, the inspection identifier may include content such as "otoscope", "otoscope examination", "otoscope examination monitoring", etc., so that after receiving the inspection identifier, the content that needs to be monitored can be determined.

[0053] In one embodiment, the otoscope examination data includes video data during the otoscope examination, and may also include image data during the otoscope examination. The otoscope examination data may be determined based on data collected during the otoscope examination.

[0054] Specifically, for example, when performing an otoscope examination, a video can be captured through the otoscope to obtain video data of the otoscope examination, and multiple otoscope image data can also be obtained based on the doctor's images.

[0055] S202: Extracting an otoscope examination image based on the otoscope examination data.

[0056] Specifically, after obtaining the otoscope examination data, the video data in the otoscope examination data can be parsed to obtain the otoscope examination image, or the otoscope image data can be directly processed to obtain the otoscope examination image.

[0057] Specifically, the video data in the otoscope examination data can be parsed at 24 frames per second to obtain the otoscope examination image. When parsing the video data in the otoscope examination data, multiple frame images can also be deduplicated to obtain the otoscope examination image.

[0058] S203: Identify the otoscope examination image, and when the external auditory canal is identified, monitor the speed of the otoscope advancement in real time, and determine the first moment of identifying the external auditory canal and the area of the external auditory canal entrance.

[0059] Specifically, during the monitoring of the otoscope examination, it is necessary to monitor the parameters of the mirror insertion and the foreign objects or lesions present during the insertion process, while the foreign objects or lesions located in the auricle can be viewed intuitively, and the parameters of the otoscope insertion when it is at the auricle do not affect the examination results. Therefore, it is necessary to identify the external auditory canal to start monitoring the otoscope examination process, monitor the parameters of the mirror insertion and foreign objects or lesions in the external auditory canal, so as to monitor and prompt the doctor, improve the examination effect, and avoid missed detection.

[0060] In one embodiment, an initial in-ear / out-ear recognition model can be set, using VGG16, Resnet, or Inception. The labels can be set to "out-ear" and "in-ear." The dataset uses otoscope images to train the initial in-ear / out-ear recognition model, thereby obtaining a trained in-ear / out-ear recognition model. The trained in-ear / out-ear recognition model can then be used to recognize otoscope examination images. When the external auditory canal is identified, the speed of the mirror's advancement is monitored in real time, and the first moment of identification of the external auditory canal and the area of the external auditory canal entrance are determined.

[0061] Specifically, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of the ear. The ear includes the outer ear, middle ear and inner ear. The inner ear includes the semicircular canals, facial nerve, auditory nerve, cochlea, vestibule and Eustachian tube. The inner ear is connected to other organs. The auricle and mastoid located in the outer ear can be visually viewed when there are foreign objects or lesions. The otoscope examination process mainly involves entering the external auditory canal through the otoscope and examining the external auditory canal, tympanic membrane and various structures in the tympanic cavity. Therefore, the external auditory canal can be automatically identified to monitor the otoscope examination process.

[0062] Specifically, real-time monitoring of the speed of inserting the otoscope refers to real-time monitoring of the speed at which the doctor inserts the otoscope, and real-time monitoring can be achieved by processing images.

[0063] Specifically, the first moment refers to the moment when the external auditory canal is identified. The timing can be started from this moment and this moment can be used as the first moment. For example, taking Beijing time as an example, the moment when the external auditory canal is identified is 15:06:51, then the first moment can be 15:06:51; or the first moment can be directly set to 00:00:00, and subsequent moments can be timed based on the first moment.

[0064] Specifically, Figure 3 For example, the entrance area of the external auditory canal refers to the area from top to bottom where the external auditory canal and the auricle meet.

[0065] Specifically, the area of the external auditory canal entrance can be determined using an external auditory canal entrance segmentation model. A preset external auditory canal entrance segmentation model can be selected, including Unet++, Mask-rcnn, and Faster R-CNN. The label is outlined by the otoscopy doctor. The dataset uses otoscope images to train the initial external auditory canal entrance segmentation model to obtain a trained external auditory canal entrance segmentation model. Then, when the external auditory canal is identified, the trained external auditory canal entrance segmentation model can be used to segment and process the otoscope examination image to obtain the external auditory canal entrance area.

[0066] S204: Identify the otoscope examination image, and when the tympanic membrane is identified, identify the light cone and / or the malleus, and when the light cone and / or the malleus are identified, determine the second moment when the light cone and / or the malleus are identified and the effective area in the ear.

[0067] Specifically, after identifying the external auditory canal, the eardrum can be identified to monitor the parameters of the endoscope insertion. The doctor can be prompted by identifying the eardrum to avoid damaging the patient's eardrum during endoscope insertion, thereby improving the safety of the endoscope insertion process.

[0068] Specifically, you can set an initial tympanic membrane recognition model, using VGG16, Resnet, or Inception. Set the label to non-tympanic membrane or tympanic membrane. Use otoscope images as the dataset to train the initial tympanic membrane recognition model and obtain a trained tympanic membrane recognition model. The trained tympanic membrane recognition model can then be used to recognize otoscope examination images. When the tympanic membrane is recognized, subsequent actions can be taken or a prompt can be provided to the doctor.

[0069] In one embodiment, the steps of identifying the otoscope examination image, identifying the light cone and / or the malleus when the tympanic membrane is identified, and determining the second moment and the effective area in the ear when the light cone and / or the malleus are identified, include: identifying the otoscope examination image, giving a prompt when the tympanic membrane is identified, and using a tympanic membrane abnormality recognition model to identify the otoscope examination image to obtain tympanic membrane abnormality recognition data; giving a prompt when the tympanic membrane abnormality recognition data represents the tympanic membrane abnormality; identifying the light cone and / or the malleus, and determining the second moment and the effective area in the ear when the light cone and / or the malleus are identified, and using a tympanic cavity abnormality recognition model to identify the otoscope examination image to obtain tympanic cavity abnormality recognition data; and giving a prompt when the tympanic cavity abnormality recognition data represents the tympanic cavity abnormality. After the eardrum is identified, the doctor can be prompted, and abnormalities in the eardrum and tympanic cavity can be identified, providing prompts and auxiliary examinations to the doctor to avoid missed detection or damage to the patient's ears.

[0070] Specifically, when recognizing the otoscope examination image, a prompt can be given when the eardrum is recognized, so that the doctor can take appropriate measures according to the prompt, such as stopping the doctor from inserting the endoscope to avoid damaging the eardrum. The doctor can also observe the eardrum and tympanic cavity according to the prompt to avoid omissions or the inability to observe the eardrum and tympanic cavity due to being too far away from the eardrum.

[0071] Specifically, when using the tympanic membrane abnormality recognition model to identify the tympanic membrane, an initial tympanic membrane abnormality recognition model can be set. VGG16, Resnet, Inception can be selected. The labels are tympanic membrane abnormality and tympanic membrane normal, which can specifically include tympanic membrane rupture and no rupture. The data set uses tympanic membrane images to train the initial tympanic membrane abnormality recognition model to obtain the trained tympanic membrane abnormality recognition model, and the tympanic membrane abnormality recognition model is used to recognize the tympanic membrane image in the otoscope examination image. A prompt can be given when a tympanic membrane abnormality is identified. For example, if an abnormality is identified on the tympanic membrane, the doctor can be prompted to observe to determine whether an abnormality occurs, thereby improving the accuracy of the examination.

[0072] Specifically, such as Figure 4 As shown in the figure, a complete tympanic membrane and a perforated tympanic membrane are shown. The tympanic membrane image in the otoscope examination image can be identified by the tympanic membrane abnormality recognition model. When a perforated tympanic membrane is identified, a prompt can be given to indicate that it may be a perforated tympanic membrane, so that the doctor can observe further and avoid missed detection.

[0073] Specifically, after the tympanic membrane is identified, the light cone and / or the malleus can be identified to monitor the parameters of the endoscope insertion, and the status inside the ear can be monitored, so as to prompt the doctor based on the endoscope insertion parameters and the status inside the ear; and after the light cone and / or the malleus are identified, the doctor can stop inserting the endoscope and be prompted to observe the tympanic cavity to avoid missed detection.

[0074] Specifically, an initial light cone and / or malleus recognition model can be set. Resnet50 can be selected, with labels set to Other, Light Cone, and Malleus. The dataset uses tympanic membrane images to train the initial light cone and / or malleus recognition model, resulting in a trained light cone and / or malleus recognition model. The trained light cone and / or malleus recognition model can then be used to recognize tympanic membrane images. When the light cone and / or malleus are recognized, subsequent procedures can be performed or a prompt can be provided to the physician.

[0075] Specifically, such as Figure 5 As shown, the tympanic membrane includes the light cone, the tense part, the umbilicus, the anterior fold of the malleus, the posterior fold of the malleus, the flaccid part, and the hammer pattern. The light cone and the anterior fold of the malleus and the posterior fold of the malleus can be identified to determine whether the light cone and / or the malleus are identified, so as to perform subsequent operations or provide prompts to the doctor.

[0076] Specifically, after identifying the light cone and / or malleus, the second moment and the effective area inside the ear are determined. The entry parameters of the mirror can be determined based on the first moment and second moment of entering the external auditory canal, so that corresponding processing can be performed, and prompts can be given through the effective area inside the ear and the area of the foreign body, making the detection results more accurate.

[0077] Specifically, the second moment refers to the moment when the light cone and / or malleus is identified. Please refer to the description of the first moment. For example, if the first moment is based on Beijing time, then the second moment is also based on Beijing time, which makes it easier to determine the lens advancement parameters. Similarly, if the first moment is based on the set benchmark, then the second moment is determined based on the benchmark of the first moment, which makes it easier to determine the lens advancement parameters. Similarly, the third moment can be determined with reference to the determination method of the second moment, which will not be repeated in the following embodiments.

[0078] Specifically, the effective area inside the ear refers to the total area of the effective image inside the ear from the external auditory canal to the identification of the light cone and / or malleus, and after the end of the endoscope, the state inside the ear can be determined based on the effective area inside the ear and the area of the foreign body, and prompts can be given based on the state inside the ear.

[0079] Specifically, after identifying the light cone and / or hammer bone, the tympanic cavity image in the otoscope examination image can be identified through the tympanic cavity abnormality recognition model, the status inside the tympanic cavity can be monitored, and prompts can be given based on the status inside the tympanic cavity, so that the doctor can observe according to the prompts, avoid missed detection problems, and improve the examination effect.

[0080] Specifically, an initial tympanic cavity abnormality recognition model can be set. VGG16, Resnet, and Inception can be selected. The labels are effusion and no effusion. The data set uses tympanic cavity images to train the initial tympanic cavity abnormality recognition model to obtain the trained tympanic cavity abnormality recognition model. The tympanic cavity abnormality recognition model is used to recognize the tympanic cavity image in the otoscope examination image, and a prompt can be given when a tympanic cavity abnormality is recognized. For example, if effusion is recognized on the tympanic cavity, the doctor can be prompted to observe to determine whether an abnormality occurs, thereby improving the accuracy of the examination.

[0081] S205: Segment the abnormal region in the otoscope examination image using different types of segmentation models, and determine the third moment when the abnormal region appears, the area of the abnormal region, and abnormal image data.

[0082] Specifically, when processing otoscope examination images, different types of segmentation models can be used to segment the abnormal areas when abnormal areas are identified, so as to determine the location information of the abnormal areas according to the time when the abnormal areas appear, so as to facilitate the processing of the abnormal areas, and provide prompts based on the area of the abnormal areas and the abnormal image data, so as to assist doctors in processing, avoid missed detections, and improve the efficiency of the examination.

[0083] In one embodiment, the abnormal region includes an impurity region and a lesion to be determined region. The steps of using different types of segmentation models to segment the abnormal region in the otoscope examination image and determining the third moment of appearance of the abnormal region, the area of the abnormal region, and abnormal image data include: using an impurity segmentation model to segment the abnormal region in the otoscope examination image and determine the area of the impurity region; and using a lesion segmentation model to segment the abnormal region in the otoscope examination image and determine the third moment of appearance of the lesion to be determined region, the area of the lesion to be determined region, and lesion image data. By using the impurity segmentation model and the lesion segmentation model to segment the impurities and lesion to be determined region in the otoscope examination image, different prompts and processing can be performed according to the impurities and lesions, thereby improving processing efficiency and inspection accuracy and avoiding missed detection.

[0084] Specifically, an initial dirt segmentation model can be preset, with options such as Unet++, Mask-rcnn, or FasterR-CNN. The doctor then labels the dirt boundary, trains the initial dirt segmentation model, and uses it to segment the dirt region in the otoscope examination image to determine the area of the dirt region. Furthermore, a single image can be selected from the otoscope examination image to calculate the effective area of the otoscope.

[0085] Specifically, the unclean matter may include earwax, hair, and the like.

[0086] Specifically, an initial lesion segmentation model can be preset. The initial lesion segmentation model includes an initial lesion region segmentation model and an initial lesion recognition model. The initial lesion region segmentation model can be selected from Unet++, Mask-rcnn, or Faster R-CNN. The doctor outlines the lesion region boundary by labeling it. The initial lesion region segmentation model is trained to obtain a trained lesion region segmentation model. The initial lesion recognition model can be selected from Resnet50, with labels such as congestion, eczema, hemorrhage, inflammation, and tumor. The dataset is the image data segmented by the lesion region segmentation model. The initial lesion recognition model is trained to obtain a trained lesion recognition model, thereby obtaining a lesion segmentation model and providing prompts and processing based on different lesion types and areas.

[0087] In one embodiment, the lesion area to be determined includes a first lesion type area and a second lesion type area; the steps of using a lesion segmentation model to segment the abnormal area in the otoscope examination image and determining the third moment when the lesion area to be determined appears, the area of the lesion area to be determined, and lesion image data include: using a lesion segmentation model to segment the abnormal area in the otoscope examination image and determine the area of the lesion area to be determined; when the area of the lesion area to be determined changes, determine it as the first lesion type area; give a prompt; when the area of the lesion area to be determined does not change, determine it as the second lesion type area; determine the third moment when the lesion area to be determined appears, and lesion image data. By judging the change in the area of the lesion area to be determined, when the area of the lesion area to be determined changes, determine it as the first lesion type area, and when the area of the lesion area to be determined does not change, determine it as the second lesion type area, so that prompts can be given according to different lesion types to perform different processing.

[0088] Specifically, the first lesion type area may be a bleeding area. After segmenting the abnormal area using the lesion segmentation model, the lesion type of the abnormal area can be identified. For example, if it is likely bleeding, the change in area can be used to confirm the presence of bleeding. This can prompt the doctor to stop the bleeding promptly to avoid damaging the patient's ear or affecting subsequent examinations. If the area of the undetermined lesion area does not change, it may indicate another abnormality. The doctor can observe the abnormality and then take appropriate measures or inform the patient to avoid missed examinations.

[0089] S206: Determine the mirror advance parameters according to the first moment, the second moment, and the mirror advance speed.

[0090] Specifically, during the otoscope examination, the mirror advancement speed and the length of time the mirror is advanced will result in poor results in the examination process and may lead to missed detections. Therefore, the mirror advancement parameters can be determined based on the first moment, the second moment and the mirror advancement speed, so that the doctor can be prompted based on the mirror advancement parameters, so that the doctor can take action based on the prompts, such as re-examination or reducing the mirror advancement speed, thereby improving the accuracy of the examination results.

[0091] In one embodiment, the step of determining the otoscope entry parameters based on the first moment, the second moment, and the otoscope entry speed includes: determining the actual otoscope entry time based on the first moment and the second moment; obtaining the preset otoscope entry time and preset otoscope entry speed of the otoscope; and determining the otoscope entry parameters based on the actual otoscope entry time, the preset otoscope entry time, the otoscope entry speed, and the preset otoscope entry speed of the otoscope. By determining the actual otoscope entry time of each recorded moment and recording the otoscope entry speed, the otoscope entry parameters can be determined based on the actual otoscope entry time, the preset otoscope entry time, the otoscope entry speed, and the preset otoscope entry speed, so as to prompt the doctor based on the otoscope entry parameters.

[0092] Specifically, the preset lens entry time can be determined based on historical data. For example, if the lens entry time is less than 60 seconds, the inspection effect is poor, then the preset lens entry time can be set to 60 seconds. Similarly, the preset lens entry speed can be determined based on historical data.

[0093] S207: Determine the abnormality parameters in the ear according to the first moment, the third moment, the speed of the microscope advancement, the area of the entrance of the external auditory canal, the area of the abnormal region, and the effective area in the ear.

[0094] Specifically, when an abnormality occurs in the ear, the doctor can be prompted by determining the location of the abnormal area and the relative size of the abnormal area, so that the doctor can refer to the determined location and the relative size of the abnormal area during subsequent processing, and the doctor can be prompted to the area where the abnormality occurs to avoid missed detection.

[0095] In one embodiment, the step of determining the abnormal parameters in the ear based on the first moment, the third moment, the mirror advancement speed, the area of the external auditory canal entrance, the area of the abnormal region, and the effective area in the ear includes: determining the position information of the area where the lesion is to be determined based on the first moment, the third moment, and the mirror advancement speed; determining the relative value of the area where the lesion is to be determined based on the area of the external auditory canal entrance and the area of the area where the lesion is to be determined; obtaining a preset value; and determining the abnormal parameters in the ear based on the relative value and the preset value. By determining the position information of the area where the lesion is to be determined, the doctor can directly observe and process according to the position information during the operation, thereby improving the efficiency of the otoscope examination. At the same time, by determining the relative value of the area of the external auditory canal entrance and the area of the area where the lesion is to be determined, and determining the abnormal parameters in the ear based on the relative value and the preset value, the doctor can be assisted in determining the degree of the abnormality based on the abnormal parameters in the ear, thereby assisting the doctor in processing and informing the patient.

[0096] Specifically, the first moment is t0, the third moment is t i , the mirror speed is v t For example, we can use the time interval [t0, ti ]Mirror speed v t Integrate to obtain the distance d between the lesion area to be determined and the entrance of the external auditory canal. The formula is as follows: Determine the location information of the area where the lesion is to be determined.

[0097] Specifically, the ratio of the area of the lesion to be determined to the area of the external auditory canal entrance is used as a relative value. Then, according to the relative value and the preset value, for example, The area of the lesion to be determined is S Z , the area of the entrance to the external auditory canal is Aera, and the ratio of the area of the lesion area to be determined to the area of the entrance to the external auditory canal can be used as the relative value τ. According to the relative value τ and the preset value β, the abnormal parameters in the ear are determined.

[0098] Specifically, for example, if the identification result of the lesion area to be determined is a tumor, the size of the tumor can be judged based on the relative value, thereby determining the risk level of the tumor, such as whether direct surgery, drug treatment or other methods are needed to assist doctors in examination.

[0099] In one embodiment, the preset value can be determined based on historical data. For example, when the ratio of the two is greater than 0.1, the tumor risk level is high, and the preset value can be 0.1.

[0100] In one embodiment, the abnormal area is an area of unclean objects, and the step of determining the abnormal parameters in the ear based on the first moment, the third moment, the speed of the mirror advancement, the area of the entrance of the external auditory canal, the area of the abnormal area, and the effective area in the ear includes: determining the cleanliness of the ear canal based on the area of the unclean area and the effective area in the ear; obtaining a preset cleanliness; and determining the abnormal parameters in the ear based on the cleanliness of the ear canal and the preset cleanliness. When the abnormal area is an area of unclean objects, the cleanliness of the ear canal can be determined by the area of the unclean area and the effective area in the ear, and the abnormal parameters in the ear can be determined based on the cleanliness of the ear canal and the preset cleanliness. When the cleanliness of the ear canal does not meet the requirements, the doctor can re-examine the ear to avoid missed inspections or inaccurate inspections.

[0101] Specifically, the preset cleanliness level can be determined based on historical data. For example, when the cleanliness level is lower than a certain level, unclean objects may block the structure inside the ear, making it impossible to observe the lesion and resulting in missed detection. In this case, the cleanliness level can be set as the preset cleanliness level.

[0102] Specifically, the ear canal cleanliness is ψ, the effective area inside the ear is S, and the area of a single unclean area is A. i , the formula for ear canal cleanliness is as follows: By comparing the ear canal cleanliness ψ with the preset cleanliness δ, it is determined whether the ear canal cleanliness meets the requirements. If the ear canal cleanliness does not meet the requirements, a prompt will be given.

[0103] S208: Prompt based on abnormal image data, scope insertion parameters and abnormal parameters in the ear.

[0104] Specifically, after determining abnormal image data, endoscope parameters and abnormal parameters in the ear, prompts can be given based on the above parameters, so that the doctor can take corresponding measures or conduct further inspections to avoid missed inspections and improve inspection efficiency.

[0105] Specifically, if the actual scope insertion time in the scope insertion parameters is less than the preset scope insertion time, the doctor can be prompted that the scope insertion time is too short, allowing the doctor to re-insert the scope for observation or re-examine the abnormal area according to the prompt. If the scope insertion speed in the scope insertion parameters is greater than the preset scope insertion speed, it indicates that the scope insertion speed is too fast, which may result in missed examinations or damage to the ear. The doctor can then be prompted to reduce the scope insertion speed, thereby protecting the patient and avoiding missed examinations.

[0106] Specifically, for example, if the ear canal cleanliness among the abnormal parameters in the ear is less than the preset cleanliness, the doctor can be prompted that the ear cleanliness is unqualified and the ear environment needs to be cleaned before re-examination, so as to avoid the unclean objects in the ear affecting the examination results.

[0107] Specifically, for example, when determining that there may be a tumor in the ear, the location information of the tumor is determined, and the relative value of the abnormal area and the area of the entrance of the external auditory canal is determined. By comparing the relative value with the preset value, the doctor can be prompted according to the size of the abnormal area. For example, the doctor can be prompted to re-check, and the doctor can be assisted in informing the patient to take action. The location of the abnormal area and the abnormal image data can also be informed to the doctor at the same time, thereby improving the efficiency of the doctor's verification of the abnormal area.

[0108] An embodiment of the present application provides a monitoring method for otoscopy, which identifies various structures in the ear and records the parameters of each structure as well as the time and speed. Specifically, when the external auditory canal, light cone and / or malleus are identified in sequence, the first moment and the area of the external auditory canal entrance, the second moment and the effective area in the ear are recorded respectively. At the same time, different types of segmentation models are used to segment the abnormal area in the otoscopy image, and the third moment when the abnormal area appears, the area of the abnormal area and the abnormal image data are determined. The mirror advancement parameters can be determined based on the first moment, the second moment and the mirror advancement speed. The abnormal parameters in the ear can be determined based on the first moment, the third moment, the mirror advancement speed, the area of the external auditory canal entrance, the area of the abnormal area and the effective area in the ear. Prompts can be given based on the abnormal image data, the mirror advancement parameters and the abnormal parameters in the ear, thereby realizing monitoring and prompting of the otoscopy process and avoiding the problem of missed detection.

[0109] At the same time, the embodiment of the present application provides a monitoring device for otoscope examination, such as Figure 6 As shown, the monitoring device 300 for otoscopy includes:

[0110] The receiving module 301 is configured to receive an inspection instruction carrying an inspection identifier, and obtain otoscope inspection data when the otoscope is inspecting inside the ear based on the inspection identifier;

[0111] An extraction module 302 is configured to extract an otoscope examination image based on the otoscope examination data;

[0112] A first recognition module 303 is configured to recognize the otoscope examination image, and when the external auditory canal is recognized, monitor the speed of the otoscope advancement in real time, and determine a first moment and an area of the external auditory canal entrance;

[0113] a second recognition module 304 for recognizing the otoscope examination image, and when the tympanic membrane is recognized, recognizing the light cone and / or the malleus, and determining a second moment and an effective area in the ear when the light cone and / or the malleus are recognized;

[0114] a segmentation module 305 for segmenting the abnormal region in the otoscope examination image using different types of segmentation models, and determining a third moment of occurrence of the abnormal region, an area of the abnormal region, and abnormal image data;

[0115] A first determining module 306 is configured to determine a mirror advancing parameter according to the first moment, the second moment, and the mirror advancing speed;

[0116] A second determining module 307 is configured to determine an ear abnormality parameter based on the first moment, the third moment, the scope advancement speed, the area of the external auditory canal entrance, the area of the abnormal region, and the effective area of the ear;

[0117] The prompt module 308 is used to provide prompts based on the abnormal image data, the scope insertion parameters and the abnormal parameters in the ear.

[0118] In one embodiment, the second recognition module 304 is configured to recognize the otoscope examination image, provide a prompt when a tympanic membrane is recognized, and use a tympanic membrane abnormality recognition model to recognize the otoscope examination image to obtain tympanic membrane abnormality recognition data;

[0119] Prompting when the tympanic membrane abnormality identification data indicates that the tympanic membrane is abnormal;

[0120] Identifying the light cone and / or the malleus, and when the light cone and / or the malleus are identified, determining a second moment at which the light cone and / or the malleus are identified and an effective area within the ear, and identifying the otoscope examination image using a tympanic cavity abnormality recognition model to obtain tympanic cavity abnormality recognition data;

[0121] When the tympanic cavity abnormality identification data indicates tympanic cavity abnormality, a prompt is given.

[0122] In one embodiment, the segmentation module 305 is configured to segment the abnormal region in the otoscope examination image using a dirt segmentation model to determine the area of the dirt region;

[0123] The abnormal area in the otoscope examination image is segmented using a lesion segmentation model, and the third moment of appearance of the lesion to be determined area, the area of the lesion to be determined area, and the lesion image data are determined.

[0124] In one embodiment, the segmentation module 305 is configured to segment the abnormal region in the otoscopic examination image using a lesion segmentation model to determine the area of the lesion to be determined;

[0125] When the area of the lesion to be determined region changes, it is determined as a first lesion type region; a prompt is given; when the area of the lesion to be determined region does not change, it is determined as a second lesion type region;

[0126] The third moment when the lesion to be determined area appears and the lesion image data are determined.

[0127] In one embodiment, the second determining module 307 is configured to determine the location information of the lesion area to be determined based on the first moment, the third moment, and the scope advancement speed;

[0128] Determining a relative value of the area to be determined based on the area of the external auditory canal entrance and the area of the area to be determined;

[0129] Get the preset value;

[0130] The abnormal parameter in the ear is determined according to the relative value and the preset value.

[0131] In one embodiment, the second determining module 307 is configured to determine the cleanliness of the ear canal based on the area of the unclean matter region and the effective area of the ear;

[0132] Get preset cleanliness level;

[0133] The abnormal parameters in the ear are determined according to the ear canal cleanliness and the preset cleanliness.

[0134] In one embodiment, the first determining module 306 is configured to determine the actual insertion time of the otoscope based on the first moment and the second moment;

[0135] Obtain the preset otoscope insertion time and preset otoscope insertion speed;

[0136] The otoscope entry parameters are determined according to the actual otoscope entry time, the preset otoscope entry time, the otoscope entry speed and the preset otoscope entry speed.

[0137] The present application also provides a computer device that integrates any of the otoscope monitoring devices provided in the present application. The computer device includes:

[0138] one or more processors;

[0139] Memory; and

[0140] One or more applications, wherein the one or more applications are stored in the memory and configured to execute, by the processor, the steps of the method for monitoring otoscopy in any of the above-mentioned embodiments of the method for monitoring otoscopy.

[0141] like Figure 7 , which shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0142] The computer device may include one or more processing cores of a processor 401, one or more computer-readable storage media of a memory 402, a power supply 403, an input unit 404, and other components. Those skilled in the art will appreciate that the computer device structure shown in the figure does not limit the computer device, and may include more or fewer components than shown, or combine certain components, or arrange the components differently. Among them:

[0143] The processor 401 is the control center of the computer device. It uses various interfaces and lines to connect the various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 402 and calling data stored in the memory 402, it performs various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 401 may include one or more processing cores; the processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401 .

[0144] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0145] The computer device also includes a power supply 403 for supplying power to various components. Preferably, the power supply 403 can be logically connected to the processor 401 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 403 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0146] The computer device may further include an input unit 404, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0147] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the computer device will load the executable files corresponding to one or more application processes into the memory 402 according to the following instructions, and the processor 401 will run the application stored in the memory 402 to implement various functions as follows:

[0148] Receive an inspection instruction carrying an inspection identifier, and obtain otoscope inspection data when the otoscope is inspecting inside the ear based on the inspection identifier; extract an otoscope inspection image based on the otoscope inspection data; identify the otoscope inspection image, and when the external auditory canal is identified, monitor the speed of the otoscope advancement in real time, and determine the first moment when the external auditory canal is identified and the area of the entrance of the external auditory canal; identify the otoscope inspection image, and when the tympanic membrane is identified, identify the light cone and / or the malleus, and when the light cone and / or the malleus are identified, determine the second moment when the light cone and / or the malleus are identified and the effective area inside the ear; use different types of segmentation models to segment the abnormal area in the otoscope inspection image, and determine the third moment when the abnormal area appears, the area of the abnormal area and the abnormal image data; determine the otoscope advancement parameter according to the first moment, the second moment and the otoscope advancement speed; determine the abnormal parameters inside the ear according to the first moment, the third moment, the otoscope advancement speed, the area of the entrance of the external auditory canal, the area of the abnormal area and the effective area inside the ear; and provide prompts based on the abnormal image data, the otoscope advancement parameter and the abnormal parameters inside the ear.

[0149] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0150] To this end, embodiments of the present application provide a computer-readable storage medium, which may include a read-only memory (ROM), a random access memory (RAM), a disk, or an optical disk. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps of any of the otoscope examination monitoring methods provided in embodiments of the present application. For example, the computer program loaded by the processor may execute the following steps:

[0151] Receive an inspection instruction carrying an inspection identifier, and obtain otoscope inspection data when the otoscope is inspecting inside the ear based on the inspection identifier; extract an otoscope inspection image based on the otoscope inspection data; identify the otoscope inspection image, and when the external auditory canal is identified, monitor the speed of the otoscope advancement in real time, and determine the first moment when the external auditory canal is identified and the area of the entrance of the external auditory canal; identify the otoscope inspection image, and when the tympanic membrane is identified, identify the light cone and / or the malleus, and when the light cone and / or the malleus are identified, determine the second moment when the light cone and / or the malleus are identified and the effective area inside the ear; use different types of segmentation models to segment the abnormal area in the otoscope inspection image, and determine the third moment when the abnormal area appears, the area of the abnormal area and the abnormal image data; determine the otoscope advancement parameter according to the first moment, the second moment and the otoscope advancement speed; determine the abnormal parameters inside the ear according to the first moment, the third moment, the otoscope advancement speed, the area of the entrance of the external auditory canal, the area of the abnormal area and the effective area inside the ear; and provide prompts based on the abnormal image data, the otoscope advancement parameter and the abnormal parameters inside the ear.

[0152] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above and will not be repeated here.

[0153] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to implement as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments and will not be repeated here.

[0154] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0155] The above is a detailed introduction to the monitoring method and device for otoscope examination, computer equipment and readable storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for monitoring otoscopy, characterized in that: include: receiving an inspection instruction carrying an inspection identifier, and acquiring otoscope inspection data when the otoscope is inspecting inside the ear based on the inspection identifier; extracting an otoscopic examination image based on the otoscopic examination data; Recognize the otoscope examination image, and when the external auditory canal is identified, monitor the speed of the otoscope advancement in real time, and determine the first moment when the external auditory canal is identified and the area of the external auditory canal entrance; Identifying the otoscope image, and when the tympanic membrane is identified, identifying the light cone and / or the malleus, and when the light cone and / or the malleus are identified, determining a second moment at which the light cone and / or the malleus are identified and an effective area within the ear; Segmenting the abnormal region in the otoscope examination image using different types of segmentation models, and determining a third moment of occurrence of the abnormal region, an area of the abnormal region, and abnormal image data; determining a mirror advancement parameter according to the first moment, the second moment, and the mirror advancement speed; Determining an abnormality parameter in the ear according to the first moment, the third moment, the scope advancement speed, the area of the external auditory canal entrance, the area of the abnormal region, and the effective area in the ear; Prompt according to the abnormal image data, the scope insertion parameters and the abnormal parameters in the ear; The abnormal region includes an unclean matter region and a lesion to be determined region, and the steps of using different types of segmentation models to segment the abnormal region in the otoscope examination image and determine the third moment of appearance of the abnormal region, the area of the abnormal region, and abnormal image data include: using the unclean matter segmentation model to segment the abnormal region in the otoscope examination image and determine the area of the unclean matter region; using the lesion segmentation model to segment the abnormal region in the otoscope examination image and determine the third moment of appearance of the lesion to be determined region, the area of the lesion to be determined region, and lesion image data; When the abnormal area is the area to be determined lesion, the step of determining the abnormality parameter in the ear according to the first moment, the third moment, the scope advancement speed, the area of the entrance of the external auditory canal, the area of the abnormal area, and the effective area in the ear includes: determining the position information of the area to be determined lesion according to the first moment, the third moment, and the scope advancement speed; determining the relative value of the area to be determined lesion according to the area of the entrance of the external auditory canal and the area of the area to be determined lesion; obtaining a preset value; determining the abnormality parameter in the ear according to the relative value and the preset value; when the abnormal area is the unclean area, the step of determining the abnormality parameter in the ear according to the first moment, the third moment, the scope advancement speed, the area of the entrance of the external auditory canal, the area of the abnormal area, and the effective area in the ear includes: determining the cleanliness of the ear canal according to the area of the unclean area and the effective area in the ear; obtaining a preset cleanliness; and determining the abnormality parameter in the ear according to the ear canal cleanliness and the preset cleanliness; The step of determining the otoscope entry parameters based on the first moment, the second moment and the otoscope entry speed includes: determining the actual otoscope entry time based on the first moment and the second moment; obtaining the preset otoscope entry time and the preset otoscope entry speed of the otoscope; and determining the otoscope entry parameters based on the actual otoscope entry time, the preset otoscope entry time, the otoscope entry speed and the preset otoscope entry speed.

2. The method for monitoring otoscopy according to claim 1, wherein: The step of identifying the otoscope examination image, and when the tympanic membrane is identified, identifying the light cone and / or the malleus, and when the light cone and / or the malleus are identified, determining a second moment at which the light cone and / or the malleus are identified and an effective area in the ear comprises: Recognize the otoscope examination image, give a prompt when the tympanic membrane is recognized, and use the tympanic membrane abnormality recognition model to recognize the otoscope examination image to obtain tympanic membrane abnormality recognition data; Prompting when the tympanic membrane abnormality identification data indicates that the tympanic membrane is abnormal; Identifying the light cone and / or the malleus, and when the light cone and / or the malleus are identified, determining a second moment at which the light cone and / or the malleus are identified and an effective area within the ear, and identifying the otoscope examination image using a tympanic cavity abnormality recognition model to obtain tympanic cavity abnormality recognition data; When the tympanic cavity abnormality identification data indicates tympanic cavity abnormality, a prompt is given.

3. The method for monitoring otoscopy according to claim 1, wherein: The lesion area to be determined includes a first lesion type area and a second lesion type area; the steps of using a lesion segmentation model to segment the abnormal area in the otoscope examination image, and determining a third moment when the lesion area to be determined appears, an area of the lesion area to be determined, and lesion image data include: Segmenting the abnormal area in the otoscopic examination image using a lesion segmentation model to determine the area of the lesion to be determined; When the area of the lesion to be determined region changes, it is determined as a first lesion type region; a prompt is given; when the area of the lesion to be determined region does not change, it is determined as a second lesion type region; The third moment when the lesion to be determined area appears and the lesion image data are determined.

4. A monitoring device for otoscopy, characterized in that: include: a receiving module, configured to receive an inspection instruction carrying an inspection identifier, and obtain otoscope inspection data when the otoscope is inspecting inside the ear based on the inspection identifier; an extraction module, configured to extract an otoscope examination image based on the otoscope examination data; a first recognition module, configured to recognize the otoscope examination image, and when the external auditory canal is recognized, monitor the speed of the otoscope advancement in real time, and determine a first moment and an area of the external auditory canal entrance; a second recognition module for recognizing the otoscope examination image, and when the tympanic membrane is recognized, recognizing the light cone and / or the malleus, and determining a second moment and an effective area in the ear when the light cone and / or the malleus are recognized; a segmentation module for segmenting the abnormal region in the otoscope examination image using different types of segmentation models, and determining a third moment of appearance of the abnormal region, an area of the abnormal region, and abnormal image data; a first determining module, configured to determine a mirror advancing parameter according to the first moment, the second moment, and the mirror advancing speed; a second determining module, configured to determine an abnormality parameter in the ear according to the first moment, the third moment, the scope advancement speed, the area of the external auditory canal entrance, the area of the abnormal region, and the effective area in the ear; A prompt module, configured to provide prompts based on the abnormal image data, the scope insertion parameters, and the abnormal ear parameters; The abnormal region includes an unclean matter region and a lesion to be determined region, and the segmentation module is specifically configured to segment the abnormal region in the otoscope examination image using the unclean matter segmentation model to determine the area of the unclean matter region; segment the abnormal region in the otoscope examination image using the lesion segmentation model, and determine the third moment of appearance of the lesion to be determined region, the area of the lesion to be determined region, and lesion image data; When the abnormal area is the lesion area to be determined, the second determining module is specifically configured to determine the location information of the lesion area to be determined based on the first moment, the third moment, and the scope advancement speed; and determine the relative value of the lesion area to be determined based on the area of the external auditory canal entrance and the area of the lesion area to be determined; Obtaining a preset value; determining the abnormal parameter in the ear based on the relative value and the preset value; when the abnormal area is an unclean area, the second determining module is specifically configured to determine the cleanliness of the ear canal based on the area of the unclean area and the effective area in the ear; obtaining a preset cleanliness; determining the abnormal parameter in the ear based on the ear canal cleanliness and the preset cleanliness; The first determination module is specifically used to determine the actual entry time of the otoscope based on the first moment and the second moment; obtain the preset entry time and preset entry speed of the otoscope; and determine the entry parameters based on the actual entry time of the otoscope, the preset entry time of the otoscope, the entry speed and the preset entry speed.

5. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the method for monitoring otoscopy according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps of the method for monitoring otoscopy according to any one of claims 1 to 3.

Images