Automatic focusing method, device and equipment of endoscope system and medium

Abstract

The invention provides an automatic focusing method, device and equipment for an endoscope system and a medium, and the method comprises the steps: carrying out the light phase difference analysis based on a phase detection pixel, obtaining a reference defocusing amount, completing the focusing operation, and recording a first stepping value; a target lesion area and a target lesion type are determined from the real-time video stream, a target ratio is determined from a first mapping table based on the target lesion type, a second stepping value is obtained based on the target ratio and the first stepping value, and conversion ratios corresponding to multiple lesion types are recorded in the first mapping table; after the endoscope adjusts the zoom motor based on the second stepping value, focusing operation is executed according to the target focus area. According to the technical scheme of the embodiment of the invention, the second stepping value can be determined according to the focus type, so that the camera can be directly adjusted to the focusing position suitable for the focus area for focusing, the zooming range for focusing the focus area is effectively reduced, and the focusing efficiency when the focus is detected in the moving process is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent endoscope technology, and in particular to an automatic focusing method, apparatus, device and medium for an endoscope system. Background Technology

[0002] Currently, oral endoscopes are commonly used instruments for oral diagnosis, and the video stream captured by the endoscope can be sent to a display device for real-time viewing. Because patients' oral cavity shapes and doctors' usage habits differ, the endoscope needs to be refocused each time it is activated to ensure video clarity.

[0003] Various automatic focusing methods have been proposed in related technologies, such as focusing based on the conventional hill-climbing method. However, there are many types of lesions in the oral cavity, such as dental plaque, dental caries, or oral mucosal lesions. The focusing reference object when the endoscope is started is usually a tooth or oral tissue, while the depth of field and color of the lesion may be different from the focusing reference object. Related technologies can only re-execute the focusing algorithm for the lesion area after the lesion is detected in the video, which takes a long time and affects the diagnostic efficiency. Summary of the Invention

[0004] The present invention aims to at least solve one of the technical problems existing in the prior art. To this end, the present invention proposes an automatic focusing method, apparatus, device, and medium for an endoscope system, which can improve the focusing efficiency when the endoscope detects lesions.

[0005] In a first aspect, embodiments of the present invention provide an automatic focusing method for an endoscope system. The endoscope system includes a host computer and an endoscope. The camera of the endoscope is equipped with an image sensor and a zoom motor. The image sensor has multiple phase detection pixels preset. The automatic focusing method includes: The endoscope responds to the first focusing signal triggered by the user, performs light phase difference analysis based on the phase detection pixels to obtain a reference defocus amount, completes the focusing operation based on the reference defocus amount, and records the first step value of the zoom motor. The host computer determines the target lesion area and the corresponding target lesion type from the real-time video stream sent by the endoscope, determines the target ratio from the first mapping table based on the target lesion type, obtains the second step value based on the target ratio and the first step value, generates a second focusing signal based on the second step value and sends it to the endoscope, wherein the first mapping table records the conversion ratio values ​​corresponding to each of the multiple lesion types; The endoscope responds to the second focusing signal, adjusts the zoom motor based on the second step value, and then performs a focusing operation according to the target lesion area.

[0006] According to some embodiments of the present invention, focusing is performed based on the reference defocus amount, including: The reference defocus amount is determined as the first focus position, and the zoom motor is controlled to move to the first focus position. The camera acquires the first test image, determines the contrast peak of the image edge of the first test image, and adjusts the zoom motor based on a preset step value. Whenever the camera acquires a new test image, the target direction is determined based on the direction of change of the contrast peak value compared to the previous test image, and the zoom motor is adjusted based on the target direction and the preset step value.

[0007] According to some embodiments of the present invention, after performing a focusing operation based on the target lesion region, the method further includes: A second focusing position and a third step value are obtained, wherein the second focusing position is determined after the zoom motor is adjusted based on the second step value, and the third step value is the step value by which the zoom motor moves to perform focusing operation on the target lesion area this time; Determine a first ratio between the first focus position and the second focus position, and determine a second ratio between the first step value and the third step value; When the first ratio is less than the second ratio, the target ratio is replaced in the first mapping table based on the second ratio.

[0008] According to some embodiments of the present invention, adjusting the zoom motor based on the target direction and the preset step value includes: Determine the target distance to the target lesion area; The adjustment step value is determined from a preset second mapping table based on the target object distance and the target lesion type, wherein the second mapping table records the mapping relationship between object distance, lesion type and step value; The target step value is determined based on the adjusted step value and the preset step value; The zoom motor is adjusted based on the target direction and the target step value.

[0009] According to some embodiments of the present invention, after performing a focusing operation based on the target lesion region, the method further includes: Multiple real-time video frames acquired continuously are merged into a target video frame; The target video stream is composed of multiple target video frames and is sent to the host computer.

[0010] According to some embodiments of the present invention, before merging multiple consecutively acquired real-time video frames into a target video frame, the method further includes: The edges of the real-time video frames are cropped based on a preset ratio. The cropped real-time video frames are subjected to displacement compensation to align the image content of the multiple real-time video frames.

[0011] According to some embodiments of the present invention, the host computer determines the target lesion region and the corresponding target lesion type from the real-time video stream sent by the endoscope, including: The real-time video stream is input into a pre-trained target detection network to obtain the target detection result output by the target detection network. The target detection result records the target lesion region, the target lesion type, and the detection confidence level. The target lesion region is marked with a detection box in the real-time video stream, and the type of the target lesion and the detection confidence level are displayed in the detection box. When the detection confidence level is lower than a preset threshold, real-time zoom is performed based on the target lesion area.

[0012] In a second aspect, embodiments of the present invention provide an autofocusing device for an endoscope system, including at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform an autofocusing method for an endoscope system as described in the first aspect above.

[0013] Thirdly, embodiments of the present invention provide an electronic device including an autofocus device for an endoscope system as described in the second aspect above.

[0014] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions for performing an autofocusing method for an endoscope system as described in the first aspect above.

[0015] An automatic focusing method for an endoscope system according to an embodiment of the present invention has at least the following beneficial effects: The endoscope, in response to a first focusing signal triggered by a user, performs light phase difference analysis based on the phase detection pixels to obtain a reference defocus amount, completes a focusing operation based on the reference defocus amount, and records the first step value of the zoom motor; the host computer determines the target lesion region and the corresponding target lesion type from the real-time video stream sent by the endoscope, determines a target ratio from a first mapping table based on the target lesion type, obtains a second step value based on the target ratio and the first step value, generates a second focusing signal based on the second step value, and sends it to the endoscope, wherein the first mapping table records the conversion ratios corresponding to multiple lesion types; the endoscope, in response to the second focusing signal, adjusts the zoom motor based on the second step value and then performs a focusing operation according to the target lesion region. According to the technical solution of the present invention, when a lesion is detected, a second step value can be determined according to the lesion type, so that the camera can be directly adjusted to a focus position suitable for the lesion area for focusing, effectively reducing the zoom range for focusing on the lesion area and improving the focusing efficiency when a lesion is detected during movement. Attached Figure Description

[0016] Figure 1 This is a flowchart of an autofocusing method for an endoscope system provided in one embodiment of the present invention; Figure 2 This is a structural diagram of an autofocus device for an endoscope system provided in another embodiment of the present invention. Detailed Implementation

[0017] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0018] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0019] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0020] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0021] This invention provides an automatic focusing method, apparatus, device, and medium for an endoscope system. The automatic focusing method for an endoscope system includes: the endoscope responding to a first focusing signal triggered by a user, performing light phase difference analysis based on phase detection pixels to obtain a reference defocus amount, completing a focusing operation based on the reference defocus amount, and recording a first step value of the zoom motor; a host computer determining a target lesion region and corresponding target lesion type from a real-time video stream sent by the endoscope, determining a target ratio from a first mapping table based on the target lesion type, obtaining a second step value based on the target ratio and the first step value, generating a second focusing signal based on the second step value and sending it to the endoscope, wherein the first mapping table records multiple lesion types and their corresponding conversion ratios; the endoscope responding to the second focusing signal, adjusting the zoom motor based on the second step value, and then performing a focusing operation according to the target lesion region. According to the technical solution of the present invention, when a lesion is detected, a second step value can be determined according to the lesion type, so that the camera can be directly adjusted to a focus position suitable for the lesion area for focusing, effectively reducing the zoom range for focusing on the lesion area and improving the focusing efficiency when a lesion is detected during movement.

[0022] The technical solutions of the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0023] Reference Figure 1 , Figure 1 This is a flowchart of an autofocusing method for an endoscope system provided in an embodiment of the present invention. The autofocusing method for the endoscope system includes, but is not limited to, the following steps: S10, the endoscope responds to the first focusing signal triggered by the user, performs light phase difference analysis based on the phase detection pixels to obtain the reference defocus amount, completes the focusing operation based on the reference defocus amount, and records the first step value of the zoom motor.

[0024] It should be noted that the autofocus method of the endoscope system in this embodiment is applied to the endoscope system, which includes a host computer and an endoscope. The camera of the endoscope is equipped with an image sensor and a zoom motor. The image sensor has multiple phase detection pixels preset.

[0025] It should be noted that the image sensor in this embodiment can be a CMOS sensor, in which several dedicated phase detection pixels are embedded in a certain pattern among the conventional imaging pixels. These phase detection pixels are covered by a semi-mask and are able to detect the phase difference of light.

[0026] It should be noted that the first focusing signal is triggered by the user. For example, a button is set in the endoscope. After the user places the endoscope into the patient's mouth and adjusts its position, he or she clicks the button to trigger the first focusing signal, so that the endoscope can initially focus according to the oral tissue environment.

[0027] It should be noted that during the initial focusing, the endoscope obtains the reference defocus amount based on the phase difference analysis of the light rays from the phase detection pixels. For example, the phase detection pixels covered by the semi-mask will capture the signal waveforms of the left image A and the right image B respectively, and the endoscope's processor automatically calculates the phase difference between these two signal waveforms. The camera is precisely calibrated at the factory to obtain a calibration coefficient. In this embodiment, the product of the phase difference and the calibration coefficient is used to determine the reference defocus amount.

[0028] It should be noted that the reference defocus amount can be used to determine the number of motor steps of the zoom motor. After the zoom motor adjusts according to the reference defocus amount, the camera reaches a coarse focus position. Based on this, the focusing operation is performed, which can effectively reduce the zoom distance and improve the focusing effect. After focusing is completed, the first step value of the zoom motor is recorded. The first step value is used to characterize the motor parameters of the endoscope when it can clearly capture images of the inside of the oral cavity without detecting lesions. The first step value serves as the basis for the secondary focusing in subsequent steps, so that the zoom motor can quickly determine the defocus amount without performing a second light phase difference analysis.

[0029] S20, the host computer determines the target lesion area and the corresponding target lesion type from the real-time video stream sent by the endoscope, determines the target ratio from the first mapping table based on the target lesion type, obtains the second step value based on the target ratio and the first step value, generates the second focusing signal based on the second step value and sends it to the endoscope. The first mapping table records the conversion ratio corresponding to each of the multiple lesion types.

[0030] It should be noted that after the endoscope sends a real-time video stream to the host computer, the host computer performs target detection on the real-time video stream to determine the target lesion area and target lesion type. The host computer has sufficient computing power to deploy common target detection models (such as the YOLO model) for target detection. The specific models will not be elaborated here.

[0031] It should be noted that there are many types of lesions inside the oral cavity, and each type of lesion is located in a different position within the oral cavity. For example, dental plaque is usually on the tooth surface, while oral mucosal lesions are usually in the oral tissues. In this embodiment, a first mapping table is configured based on prior knowledge of the location of each type of lesion. The first mapping table records the step value conversion ratio for each type of lesion within the oral cavity. The conversion ratio characterizes the difference in focal length obtained by focusing on the oral surface and focusing on the lesion area. In other words, the conversion ratio can characterize the approximate location of the lesion area within the oral cavity. When the first focusing is completed in step S10, the target object is usually the oral cavity wall or teeth. When dental plaque is detected, the target ratio corresponding to the dental plaque is recorded as 10% in the first mapping table, which means that the second step value can be obtained by increasing the first step value by 10%. Similarly, when oral mucosal lesions are detected, the target ratio corresponding to the oral mucosal lesions can be found in the first mapping table as -5%, which means that the second step value can be obtained by decreasing the first step value by 5%. By using the positional difference between the initial focused object and the lesion represented by the target ratio, the second step value is quickly obtained as the defocus amount for the second focusing of the zoom motor, reducing the adjustment amount during the second focusing.

[0032] It is worth noting that in this embodiment, the second step value is determined by the host computer. The host computer can obtain the first step value of the endoscope in real time and has sufficient computing power for target detection. It can also complete table lookup and data calculation in a short time. After determining the second step value, the host computer generates a second focusing signal based on the second step value and sends the second focusing signal to the endoscope. Therefore, in this embodiment, the first focusing is triggered by the user to execute the endoscope, and the second focusing is triggered by the host computer to execute the endoscope.

[0033] S30, the endoscope responds to the second focusing signal, adjusts the zoom motor based on the second step value, and then performs focusing operation according to the target lesion area.

[0034] It should be noted that after the endoscope obtains the second focusing signal, it obtains the second step value from the second focusing signal and adjusts the zoom motor. After adjusting the zoom motor with the second step value, the position of the zoom motor is the focus position for the second focusing. The zoom motor can effectively reduce the amount of adjustment during focusing based on the second step value.

[0035] It should be noted that in this embodiment, the target lesion area is simultaneously carried in the second focusing signal, enabling the endoscope to focus on the target lesion area. The host computer determines the focusing area of ​​the endoscope, thereby improving focusing accuracy.

[0036] In another embodiment, in step S10, the focusing operation is completed based on the reference defocus amount, which specifically includes, but is not limited to, the following steps: S11, determine the reference defocus amount as the first focus position, and control the zoom motor to move to the first focus position; S12: Acquire the first test image through the camera, determine the contrast peak of the image edge of the first test image, and adjust the zoom motor based on the preset step value; S13: Whenever the camera acquires a new test image, it determines the target direction based on the direction of change in the peak contrast value compared to the previous test image, and adjusts the zoom motor based on the target direction and a preset step value.

[0037] It should be noted that during the first focusing process, this embodiment uses the defocus amount as the reference to determine the first focus position. After the zoom motor moves to the first focus position, the focusing operation is started, which can reduce the adjustment range during focusing.

[0038] It should be noted that after focusing begins, the camera acquires multiple test images, focusing by analyzing changes in the contrast peak values ​​at the image edges. After acquiring the first test image, the contrast peak value at the image edges is detected, and the zoom motor is adjusted once according to a preset step value. After adjustment, the second test image is acquired, and the contrast peak value at the image edges is detected and compared with the contrast peak value of the first test image. If it tends to be clearer, the adjustment direction is confirmed to be correct, and the zoom motor is adjusted again based on the preset step value. Focusing is completed through multiple repetitions, which will not be elaborated further here.

[0039] In another embodiment, after step S30 is performed, the following steps are included, but are not limited to: S41, obtain the second focusing position and the third step value, wherein the second focusing position is determined after the zoom motor is adjusted based on the second step value, and the third step value is the step value of the zoom motor moving to perform focusing operation on the target lesion area this time. S42, determine the first ratio of the first focus position and the second focus position, and determine the second ratio of the first step value and the third step value; S43, when the first ratio is less than the second ratio, replace the target ratio in the first mapping table based on the second ratio.

[0040] It should be noted that after the zoom motor is adjusted based on the second step value, the current position of the zoom motor is determined as the second focusing position. The processing of the second focusing can be referred to in steps S12 and S13, and will not be repeated here. The second focusing position is the basis for the second focusing. Based on the second focusing position, the third step value is moved to complete the focusing. The host computer obtains the third step value of the endoscope for subsequent operations.

[0041] It should be noted that in this embodiment, the second step value is determined using the ratio recorded in the first mapping table. This second step value is obtained by moving the second step value after focusing on the first focus position. Therefore, the first ratio between the first and second focus positions can characterize the difference between the two focusing starting positions. The third step value is the step value moved during the second focusing attempt, and the first step value is the step value moved during the first focusing attempt. Therefore, the second ratio can characterize the difference in the step values ​​moved during the two focusing attempts. When the first ratio is less than the second ratio, it can be determined that the proportion of the moving step value is greater than the difference in the starting position. That is, the magnitude of the focusing movement based on the second focus position is greater than the magnitude of the focusing movement based on the first focus position. The accuracy of the second focus position is insufficient. The second ratio replaces the target ratio, so that the next time secondary focusing is triggered, it can be adjusted to the new second focus position with a larger magnitude. Through multiple uses, the target ratio recorded in the first mapping table can continuously approach accuracy.

[0042] In another embodiment, in step S13, the zoom motor is adjusted based on the target direction and a preset step value, which specifically includes, but is not limited to, the following steps: S131, Determine the target distance to the target lesion area; S132, determine the adjustment step value from the preset second mapping table based on the target object distance and the target lesion type, wherein the second mapping table records the mapping relationship between object distance, lesion type and step value; S133, determine the target step value based on the adjusted step value and the preset step value; S134, adjusts the zoom motor based on the target direction and target step value.

[0043] It should be noted that after determining the target lesion area, the target object distance can be determined by simple image ranging. The location of lesions inside the oral cavity is uncertain, and the oral cavity is also irregularly shaped. Therefore, the deeper the lesion is located, the farther the adjustment distance is when focusing. Therefore, in this embodiment, the adjustment step value required for different lesion types is recorded in the second mapping table. For example, the zoom step value increases by 30 at a distance of 12nm for dental plaque, by 20 at a distance of 15nm, and by 10 at a distance of 18nm. The specific value can be set according to actual needs.

[0044] It should be noted that in this embodiment, the preset step value is adjusted according to the adjustment step value. For example, when the preset step value is 100, the adjustment step value is 30, and the target step value is 130. The zoom motor is adjusted based on the target step value, and then the next test image is obtained.

[0045] In another embodiment, after step S30 is performed, the following steps are included, but are not limited to: S51, merging multiple continuously acquired real-time video frames into a target video frame; S52, based on multiple target video frames, forms a target video stream, which is then sent to the host computer.

[0046] It should be noted that in this embodiment, after the endoscope captures a real-time video stream at high speed, it merges multiple consecutive real-time video frames into a single target video frame. The target video frame replaces the real-time video frames to obtain the target video stream. This allows the endoscope to achieve image stabilization through image merging during movement, thereby improving the display effect of the target video stream.

[0047] In another embodiment, before performing step S51, the following steps are included, but are not limited to: S501 performs edge cropping on real-time video frames based on a preset ratio; S502 performs displacement compensation on the cropped multi-frame real-time video to align the image content of the multi-frame real-time video.

[0048] It should be noted that, in order to further improve the image stabilization effect, this embodiment first performs a certain percentage cropping on the edges of the real-time video frame. For example, if the preset percentage is 10%, then 10% of the image outside the real-time video frame will be cropped and removed to reduce the area of ​​blurred edges.

[0049] It should be noted that each real-time video frame is captured during movement. After cropping, this embodiment performs displacement compensation on multiple real-time video frames. By incorporating a position sensor into the endoscope, the movement distance of multiple video frames can be predicted. For example, using the last real-time video frame as a reference, the pixels of the preceding real-time video frames are moved according to the movement distance, so that the same image content can be aligned. After displacement compensation, image merging can further reduce the probability of blurring and improve the image stabilization effect.

[0050] In another embodiment, in step S20, the host computer determines the target lesion area and the corresponding target lesion type from the real-time video stream sent by the endoscope, specifically including but not limited to the following steps: S21, input the real-time video stream into the pre-trained target detection network, and obtain the target detection results output by the target detection network. The target detection results record the target lesion area, target lesion type and detection confidence. S22, mark the target lesion area in the real-time video stream with a detection box, and display the target lesion type and detection confidence in the detection box; S23, when the detection confidence level is lower than the preset threshold, real-time zoom is performed based on the target lesion area.

[0051] It should be noted that the target detection network can be the common YOLO network. We will not go into detail about the specific network model here. Before use, it should be trained based on sample images of various lesions. After the real-time video stream is input into the target detection network, it can predict the target lesion area and target lesion type in the current picture in real time and quickly.

[0052] It should be noted that the detection results of target detection networks usually carry confidence scores. In this embodiment, the target lesion area, target lesion type, and detection confidence score are integrated into the target detection result. When the host computer displays the real-time video stream, the target lesion type and detection confidence score are displayed in the detection box that marks the target lesion area, thereby improving the visualization effect.

[0053] It is worth noting that the endoscope takes pictures while moving. After the target lesion area is detected, medical staff usually stop the endoscope to conduct a detailed examination. When the detection confidence is lower than the preset threshold, the focus may be inaccurate during the movement, resulting in low image clarity and affecting the detection effect of the target detection network. However, the target lesion area is definite. Therefore, this embodiment further performs real-time zoom based on the target lesion area and uses the detection confidence to trigger a third focus to improve the imaging effect of the endoscope.

[0054] like Figure 2 As shown, Figure 2 This is a structural diagram of an autofocus device for an endoscope system according to one embodiment of the present invention. The present invention also provides an autofocus device for an endoscope system, comprising: The processor 401 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 402 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 402 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 402 and is called and executed by the processor 401 to execute an autofocusing method for an endoscope system according to an embodiment of this application. Input / output interface 403 is used to implement information input and output; The communication interface 404 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 405 transmits information between various components of the device (e.g., processor 401, memory 402, input / output interface 403, and communication interface 404); The processor 401, memory 402, input / output interface 403 and communication interface 404 are connected to each other within the device via bus 405.

[0055] This application also provides an electronic device, including an autofocus device for an endoscope system as described above.

[0056] This application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program that, when executed by a processor, implements the above-described automatic focusing method for an endoscope system.

[0057] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate, and may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0058] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0059] The above provides a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. An autofocusing method of an endoscope system, characterized by, The endoscope system comprises a host computer and an endoscope, the camera head of the endoscope is provided with an image sensor and a zoom motor, the image sensor is pre-provided with a plurality of phase detection pixels, and the automatic focusing method comprises: The endoscope responds to a first focusing signal triggered by a user, performs light phase difference analysis based on the phase detection pixels to obtain a reference defocus amount, completes focusing operation based on the reference defocus amount, and records a first step value of the zoom motor; The host computer determines a target lesion area and a corresponding target lesion type from a real-time video stream sent by the endoscope, determines a target ratio from a first mapping table based on the target lesion type, obtains a second step value based on the target ratio and the first step value, generates a second focusing signal based on the second step value and sends it to the endoscope, wherein the first mapping table records the conversion ratio corresponding to each of the multiple lesion types; The endoscope responds to the second focusing signal, adjusts the zoom motor based on the second step value, and performs focusing operation according to the target lesion area.

2. The autofocusing method of an endoscope system according to claim 1, characterized by, The focusing operation based on the reference defocus amount comprises: determining the reference defocus amount as a first focusing position, and controlling the zoom motor to move to the first focusing position; acquiring a first test image through the camera head, determining a contrast peak value of an image edge of the first test image, and adjusting the zoom motor based on a preset step value; whenever the camera head acquires a new test image, determining a target direction based on the numerical change direction of the contrast peak value of the last test image, and adjusting the zoom motor based on the target direction and the preset step value.

3. The autofocusing method of an endoscope system according to claim 2, characterized by, After the focusing operation according to the target lesion area, the method further comprises: obtaining a second focusing position and a third step value, wherein the second focusing position is determined after the zoom motor is adjusted based on the second step value, and the third step value is the step value of the zoom motor moving for the focusing operation of the target lesion area this time; determining a first ratio of the first focusing position and the second focusing position, and determining a second ratio of the first step value and the third step value; when the first ratio is less than the second ratio, replacing the target ratio in the first mapping table based on the second ratio.

4. The autofocusing method of an endoscope system according to claim 2, characterized by, Adjusting the zoom motor based on the target direction and the preset step value comprises: determining a target object distance of the target lesion area; determining an adjustment step value from a preset second mapping table based on the target object distance and the target lesion type, wherein the second mapping table records the mapping relationship among object distance, lesion type and step value; determining a target step value based on the adjustment step value and the preset step value; adjusting the zoom motor based on the target direction and the target step value.

5. The autofocusing method of an endoscope system according to claim 1, characterized by, After the focusing operation according to the target lesion area, the method further comprises: merging a plurality of continuously acquired real-time video frames into a target video frame; based on a plurality of target video frames, a target video stream is formed, and the target video stream is sent to the host computer.

6. The autofocusing method of an endoscope system according to claim 5, characterized by, Before the plurality of real-time video frames continuously acquired are combined into a target video frame, the method further comprises: performing edge clipping processing on the real-time video frames based on a preset ratio; performing displacement compensation on the plurality of real-time video frames after clipping to align the image content of the plurality of real-time video frames.

7. The autofocusing method of an endoscope system according to claim 1, characterized by, The host computer determines a target lesion area and a corresponding target lesion type from the real-time video stream sent by the endoscope, comprising: inputting the real-time video stream into a pre-trained target detection network to obtain a target detection result output by the target detection network, wherein the target detection result records the target lesion area, the target lesion type and a detection confidence; annotating the target lesion area in the real-time video stream by a detection frame, displaying the target lesion type and the detection confidence in the detection frame; when the detection confidence is lower than a preset threshold, performing real-time zooming based on the target lesion area.

8. An autofocusing device of an endoscope system, characterized by comprising: The computer readable storage medium stores computer executable instructions for causing a computer to execute an auto-focusing method of an endoscope system according to any one of claims 1 to 7.

9. An electronic device, comprising: The auto-focusing device of the endoscope system according to claim 8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions for causing a computer to execute an auto-focusing method of an endoscope system according to any one of claims 1 to 7.

Images