An endoscope system, image processing method, and computer program product thereof

Abstract

The present disclosure relates to an endoscope system, an image processing method and a computer program product thereof, and relates to the technical field of medical imaging. The endoscope system of the present disclosure comprises a light source, an insertion part and an image processing part. The insertion part has at least a tip part configured to be inserted into an examination region. The tip part is provided with a first imaging module and a second imaging module. The light source is used to provide illumination for the examination region. The first imaging module is used to obtain a first image. The first imaging module has a first focusing range and a first imaging range. The second imaging module is used to obtain a second image. The second imaging module has a second focusing range and a second imaging range. The second focusing range and the first focusing range are at least partially different. The second imaging range and the first imaging range have an intersection. The image processing part is used to select at least one of the first image and the second image and output, or obtain a composite image based on the first image and the second image and output.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of medical imaging, in particular to an endoscope system, an image processing method and a computer program product thereof. BACKGROUND

[0002] Endoscope technology, as an imaging technology, is often used in the field of medical diagnosis. Endoscope technology enables a component with imaging function to enter the inside of a biological body (such as a human body, an animal body, etc.) through a natural orifice of the biological body or a small incision formed by surgery, and then obtain image information of a region to be examined inside the biological body, so as to provide a more intuitive and accurate diagnosis basis.

[0003] At present, endoscope technology has made significant development and is still progressing. How to obtain clearer images is one of the development directions of endoscope technology. Some embodiments of the present specification aim to provide an endoscope system and a corresponding image processing method to improve the clarity of endoscope images. SUMMARY

[0004] According to a first aspect, one or more embodiments of the present specification provide an endoscope system, comprising a light source, an insertion part, and an image processing part, the insertion part having at least a tip part configured to be inserted into a region to be examined, the tip part being provided with a first imaging module and a second imaging module: the light source is configured to provide illumination for the region to be examined; the first imaging module is configured to obtain a first image; the first imaging module has a first focusing range and a first imaging range; the second imaging module is configured to obtain a second image; the second imaging module has a second focusing range and a second imaging range; the second focusing range and the first focusing range are at least partially different, and the second imaging range and the first imaging range have an intersection; the image processing part is configured to select at least one of the first image and the second image and output, or obtain a composite image based on the first image and the second image and output.

[0005] According to a second aspect, one or more embodiments of the present specification provide an endoscope insertion part, having at least a tip part configured to be inserted into a region to be examined, the tip part being provided with a light source, a first imaging module and a second imaging module; the light source is configured to provide illumination for the region to be examined; the first imaging module is configured to obtain a first image; the first imaging module has a first focusing range and a first imaging range; the second imaging module is configured to obtain a second image; the second imaging module has a second focusing range and a second imaging range; the second focusing range and the first focusing range are at least partially different, and the second imaging range and the first imaging range have an intersection.

[0006] According to a third aspect, one or more embodiments of the present specification provide an image processing method for an endoscope system, comprising: obtaining a first image and a second image, the first image and the second image having different focal point regions, the first image corresponding imaging range and the second image corresponding imaging range having an intersection; selecting at least one of the first image and the second image and outputting, or obtaining a composite image based on the first image and the second image and outputting.

[0007] According to a fourth aspect, one or more embodiments of the present specification provide a computer readable storage medium storing computer code or instructions, when at least part of the computer code or instructions is executed by a processor, the image processing method provided by some embodiments of the present specification can be implemented.

[0008] According to a fifth aspect, one or more embodiments of the present specification provide a computer program product comprising computer code or instructions, when the computer code or instructions are executed by a processor, the image processing method provided by some embodiments of the present specification can be implemented. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 Structure diagram of an endoscope system shown in some embodiments of the present specification;

[0010] Figure 2 Structure diagram of an insertion part in an endoscope system shown in some embodiments of the present specification;

[0011] Figure 3 Working mode diagram of an endoscope system shown in some embodiments of the present specification;

[0012] Figure 4 Exemplary flowchart of an image processing method shown in some embodiments of the present specification;

[0013] Figure 5 Exemplary flowchart of obtaining image clarity shown in some embodiments of the present specification;

[0014] Figure 6 Exemplary flowchart of determining target image sub-region shown in some embodiments of the present specification;

[0015] Figure 7 Image registration principle diagram shown in some embodiments of the present specification;

[0016] Figure 8 Exemplary block diagram of an image processing system shown in some embodiments of the present specification.

[0017] Components in the figure are marked as follows: 100 endoscope system; 110 endoscope; 111 insertion portion; 111a tip portion; 112 operation portion; 120 processing device; 130 display device; 140 light source; 21 first imaging module; 211 first optical lens; 212 first image sensor; 22 second imaging module; 221 second optical lens; 222 second image sensor; 31 optical fiber. DETAILED DESCRIPTION

[0018] The application will be further described below in detail with specific embodiments and with reference to the accompanying drawings. Like elements in different embodiments are denoted by like reference numerals. In the following embodiments, many details are described in order to provide a better understanding of the present application. However, one skilled in the art can easily recognize that some features can be omitted in different cases, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, in order to avoid the core of the present application being overwhelmed by too much description, and it is not necessary to describe these related operations in detail for one skilled in the art, who can fully understand the related operations according to the description in the specification and general technical knowledge in the art.

[0019] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various embodiments. At the same time, the steps or actions in the method description can also be sequentially adjusted or adjusted in a manner that one skilled in the art can easily see. Therefore, the various sequences in the specification and drawings are only for the purpose of clearly describing a certain embodiment, and do not mean that the sequence is necessary, unless otherwise stated that a certain sequence must be followed.

[0020] The serial numbers of components in this paper, such as "first", "second", etc., are only used to distinguish the described objects, and do not have any sequence or technical meaning. The "connection" and "coupling" in the present application include direct and indirect connection (coupling) unless otherwise specified.

[0021] Endoscope technology, as an imaging technology, is often used in the field of medical diagnosis. Endoscope technology enables components with imaging functions to enter the inside of a biological body (such as a human body, an animal body, etc.) through natural orifices of the biological body or small incisions formed by surgery, and then obtains image information of the inspected area inside the biological body, to provide more intuitive and accurate diagnostic basis.

[0022] Figure 1 Structure diagram of the endoscope system shown in some embodiments. As shown in FIG. 1, the endoscope system 100 includes an endoscope 110, a processing device 120, a display device 130, and a light source 140. Figure 1As shown, the endoscope system 100 can include an endoscope 110 and a processing device 120. The endoscope 110 is configured to acquire image data of an examination region, and the processing device 120 is configured to process the image data to obtain a medical image required for diagnosis, and can also control the endoscope 110 or a part of components therein, so that the image data can better meet the requirements. In some embodiments, the endoscope system 100 can further include a display device 130, which can acquire the image output by the processing device 120 and display it to a user (e.g., a doctor). In other embodiments, the endoscope system 100 can not include the display device 130, and when the image needs to be viewed, the endoscope system 100 can be connected to the display device 130, and the processed image can be output to the display device 130 for display.

[0023] In some embodiments, the endoscope 110 can further include an insertion portion 111 and an operation portion 112. At least a part of the insertion portion 111 can be bent, and is configured to guide an imaging module into an examination region in a living body. In some embodiments, the insertion portion 111 can be a flexible tubular component having a certain length, one end of which is connected to the operation portion 112, and the other end of which can be inserted into the examination region in the living body. The operation portion 112 can be held and operated by a user to adjust the depth or direction of the insertion portion 111, so as to adjust the observation region and the viewing angle of the endoscope.

[0024] In some embodiments, a part of the insertion portion 111 that can be inserted into the examination region in the living body can be referred to as a front end portion or a tip end portion 111a (it should be understood that, Figure 1 The dashed line in the figure is a mark for indicating the position of the tip end portion, and should not be understood as an entity component in the endoscope system 100), and the imaging module can be arranged at the tip end portion 111a of the insertion portion to acquire image data of the examination region. In some alternative embodiments, the tip end portion 111a of the insertion portion can also be provided with a light source to provide illumination for the examination region, so that the imaging module can acquire an image with better recognition. Referring to Figure 2 In other alternative embodiments, the insertion portion 111 has an optical fiber 31 inside, which has an extension direction consistent with the insertion portion 111. One end surface of the optical fiber 31 is exposed from the end surface of the tip end portion 111a, and the light source 140 is arranged at the end of the optical fiber away from the tip end portion 111a. The optical fiber 31 can guide the light beam emitted by the light source 140 to the examination region to provide illumination.

[0025] In some embodiments, the imaging module can include an optical lens and an image sensor. The optical lens can be composed of one or more optical elements (e.g., lenses) for capturing light rays of the detected region to form an optical image. The image sensor is configured to receive the optical image and generate a computer-recognizable medical image. Exemplarily, the image sensor can be a CCD image sensor, a CMOS image sensor, or the like.

[0026] In some embodiments, the optical lens of the endoscope imaging module can be a fixed-focus lens. The fixed-focus lens has a fixed focal point, that is, the focal point of the fixed-focus lens cannot be adjusted. The fixed-focus lens has the advantage of small size compared with the zoom lens, which is suitable for the size limitation of the endoscope, but the disadvantage is that the focusing range of the fixed-focus lens is limited. In some embodiments, the focusing range can be understood as the spatial range that can be clearly imaged by the fixed-focus lens, which is related to the focal point of the fixed-focus lens. Specifically, the focusing range can be represented by the distance range from the main optical axis direction of the fixed-focus lens to the end surface of the fixed-focus lens. As an example, the focal point of a certain fixed-focus lens is located at a position 5 mm away from the outer end surface (or referred to as the front end surface or the outer surface, i.e., the end surface or surface close to the object side) in the main optical axis direction, and the fixed-focus lens can clearly image the objects or objects in the spatial range within a certain distance (e.g., 2 mm) from the focal point, which can be referred to as the focusing range. For intuitive understanding, it can also be said that the focusing range of a certain fixed-focus lens is the interval from 1 mm to 7 mm in the direction of the main optical axis of the lens to the outer end surface of the lens. In some embodiments, the focusing range of the imaging module is limited by the focal point or focusing range of the optical lens thereof, and therefore, the focal point or focusing range of the optical lens of the imaging module can also be determined as the focal point or focusing range of the imaging module.

[0027] It is not difficult to understand that the fixed-focus lens limits the spatial or distance range that the endoscope system can clearly image. In actual applications, in order to obtain clear images in a larger range of the detected region, the doctor needs to frequently operate to enable the tip of the insertion portion to reach more regions, which increases the operation complexity or the examination time. Therefore, some embodiments of the present specification provide an endoscope system, which is provided with two imaging modules inside the system. The two imaging modules have different focal point positions and similar or even the same imaging range, thereby expanding the focusing spatial range of the endoscope system and enabling the obtained images to have better clarity.

[0028] Figure 2 FIG. 1 is a structural schematic diagram of an insertion portion of an endoscope system according to some embodiments of the present specification, which is combined with FIG. 2 to illustrate the structure of the endoscope system according to some embodiments of the present specification. Figure 1 and Figure 2Some embodiments of the present disclosure provide an endoscope system, which can include a light source 140, an insertion portion 111, and an image processing portion. The insertion portion 111 has at least a tip portion 111a configured to be inserted into an examination region, and the tip portion 111a is provided with a first imaging module 21 and a second imaging module 22. The light source 140 is configured to provide illumination for the examination region. The first imaging module 21 is configured to acquire a first image, and the first imaging module 21 has a first focusing range and a first imaging range. The second imaging module 22 is configured to acquire a second image, and the second imaging module 22 has a second focusing range and a second imaging range. The second focusing range is at least partially different from the first focusing range, and the second imaging range has an intersection with the first imaging range. The image processing portion is configured to select at least one of the first image and the second image and output, or obtain a composite image based on the first image and the second image and output.

[0029] In some embodiments, the first imaging module 21 can further include a first optical lens 211 configured to acquire a first optical image, and a first image sensor 212 configured to receive the first optical image and convert it into the first image. The second imaging module 22 can further include a second optical lens 221 configured to acquire a second optical image, and a second image sensor 222 configured to receive the second optical image and convert it into the second image. More details about the optical lens and the image sensor can be found in other parts of the present disclosure, and will not be repeated here.

[0030] In some embodiments, the first optical lens and the second optical lens can have different focal points, so that the first focusing range and the second focusing range are at least partially different. As an example, the focal point of the first optical lens can be located in a region 7mm-15mm from the front end face of the lens. Specifically, the focal point of the first optical lens can be located in a section 7mm-15mm from the outer end face of the lens on the main optical axis, further, it can be located in a section 8mm-12mm, or a section 9mm-11mm, for example, the focal point of the first optical lens can be located at a position 7.5mm, 10mm, 11.5mm or 12mm from the outer end face of the lens on the main optical axis. In some embodiments, the focal point of the second optical lens can be located in a region 2mm-6mm from the front end face of the lens. Specifically, the focal point of the second optical lens can be located in a section 2mm-6mm from the outer end face of the lens on the main optical axis, further, it can be located in a section 3mm-5mm, for example, the focal point of the second optical lens can be located at a position 2.5mm, 3.5mm, 4mm or 5mm from the outer end face of the lens on the main optical axis.

[0031] The focusing range of the imaging module is affected by the focal point of the optical lens. Since the focal points of the first optical lens and the second optical lens are different, the first focusing range and the second focusing range are at least partially different. At least partially different can be understood as that the first focusing range and the second focusing range are different but have an intersection, or the first focusing range and the second focusing range have no intersection. As an example, the focal point of the first optical lens is 10 mm away from the outer end surface of the lens, and the first focusing range can include an interval of 5 mm to 50 mm along the main optical axis of the lens from the outer end surface of the lens. The focal point of the second optical lens is 5 mm away from the outer end surface of the lens, and the second focusing range can include an interval of 1 mm to 7 mm along the main optical axis of the lens from the outer end surface of the lens. The first imaging module and the second imaging module have different focusing ranges, which can expand the overall focusing range of the endoscope system. In the foregoing example, the endoscope system with the dual imaging modules can obtain a clear image of an object within a range of 1 mm to 50 mm from the outer end surface of the lens, which is larger than the focusing range of a single imaging module. With the dual imaging modules, the focal points of the two optical lenses can be flexibly selected as needed, so that the two focusing ranges are greatly different and adjacent to each other (for example, the first focusing range corresponds to an interval of 7 mm to 100 mm, and the second focusing range corresponds to an interval of 1 mm to 7 mm), and the overall focusing range of the endoscope system is greatly expanded (for example, the overall focusing range corresponds to an interval of 1 mm to 100 mm).

[0032] In some embodiments, the imaging range can be understood as the field of view (FOV) of the imaging module or the spatial region photographed thereby. The imaging range can be directly regarded as a space region surrounded by a circular truncated cone with the lens end surface of the imaging module as the upper base and the main optical axis of the lens as the central axis. Therefore, it can be considered that the imaging range is related to the position of the imaging module in space, the direction of the main optical axis of the lens, and the field of view angle (the included angle between the generatrix of the circular truncated cone and the central axis).

[0033] With reference to the foregoing Figure 2The first imaging module 21 and the second imaging module 22 can be arranged in parallel at the tip portion 111a of the insertion portion, and the optical axis of the first optical lens 211 and the optical axis of the second optical lens 221 are parallel. In this way, the imaging ranges of the two imaging modules have an intersection, and the same object (e.g., human tissue, organs, etc.) can be imaged. In some embodiments, the front end surface of the first optical lens 211 and the front end surface of the second optical lens 221 can be coplanar or flush. In some embodiments, the optical lens of the imaging module can be a glass lens or an injection molded lens. The injection molded lens is manufactured by injection molding technology, and the main materials include PC (polycarbonate), PMMA (polymethyl methacrylate), etc. The injection molded lens has the characteristics of high molding precision, light weight, good optical performance, and low cost. The injection molded lens has a smaller size than the glass lens, making it easier to arrange two optical lenses in parallel or side by side in the insertion portion of the endoscope, which has a large diameter limit. In some embodiments, the two imaging modules can be close to or in contact with each other, or the size of the optical lens in the imaging module can be further reduced, so that the distance between the optical axes of the two imaging modules is less than the radius of the tip portion. In this way, the imaging ranges of the two imaging modules can be further improved (e.g., the first image and the second image have an overlapping area of 80% or 90% or more), and the image synthesis of the first image and the second image is more suitable.

[0034] Some embodiments of the present disclosure provide an endoscope system with two imaging modules (i.e., using two optical lenses and two image sensors for imaging), so that each optical lens corresponds to one image sensor, reducing the assembly difficulty between the optical lens and the image sensor. Specifically, to obtain a clear image, the imaging surface of the optical lens and the photosensitive surface of the image sensor need to be coincident. One-to-one correspondence between the optical lens and the image sensor can significantly reduce the assembly difficulty of the imaging surface of the optical lens and the photosensitive surface of the image sensor, reduce costs, and facilitate mass production of products. On the other hand, one-to-one correspondence between the optical lens and the image sensor can also ensure that more light passing through the optical lens enters the corresponding image sensor, making the image brighter and facilitating observation.

[0035] The image processing unit can be located in a processing device (e.g., a computer, a server, etc.) connected to the endoscope, or can be integrated into the endoscope. Figure 1The image processing unit in the processing device 120 shown can be a control unit or a processor for performing image processing. In some embodiments, the image processing unit is only used for image selection, specifically, at least one of the first image and the second image can be selected and output. In some embodiments, the image processing unit can only be used for image synthesis, specifically, a synthesized image can be obtained based on the first image and the second image and output. In yet some embodiments, the image processing unit can be used for both image selection and image synthesis, at this time, the endoscope system can be considered to have two working modes, i.e., an image selection mode and an image synthesis mode, Figure 3 is a working mode schematic diagram of the endoscope system shown in some embodiments of the present specification, Figure 3 The image processing unit shown can work in the image selection mode or the image synthesis mode as needed.

[0036] In some embodiments, the image processing unit can work in the image selection mode based on a first instruction to select and output at least one of the first image and the second image, or work in the image synthesis mode to obtain and output a synthesized image based on the first image and the second image. The first instruction can be a mode selection instruction input by a user, as an example, a button, a knob, a selection switch or the like control can be provided on an operation unit (such as the operation unit 112), and the user can input the first instruction through the control to control the image processing unit to execute an image selection process (work in the image selection mode) or an image synthesis process (work in the image synthesis mode). In some alternative embodiments, the control can be provided on the processing device (such as the processing device 120), and the user can input the first instruction through the control on the processing device. In yet some alternative embodiments, the processing device can control a display device (such as the display device 130) to display a graphical user interface, and the graphical user interface can display a control, and the user can trigger the control on the graphical user interface to input the first instruction (such as by clicking the control with a mouse or by touching the control corresponding area on the touch screen of the display device 130). In some embodiments, the first instruction can be converted into an electrical signal input into the control unit or the processor of the processing device, when the first instruction is an electrical signal in a first state, it indicates that the working mode selected by the user is the image selection mode, and when the first instruction is an electrical signal in a second state, it indicates that the working mode selected by the user is the image synthesis mode.

[0037] In some embodiments, the image quality can be sharpness, which can refer to the degree of clarity of textures, boundaries, etc. on the image. Generally, when the contrast between light and dark regions in the image is large, and the texture details are rich, the image can be considered to be clearer. The sharpness indicator of the image can reflect the sharpness of the image. The higher the sharpness indicator, the better the sharpness of the image. The sharpness of the image can be related to the resolution, sharpness, or contrast of the image, etc. In order to facilitate measurement, in some embodiments, one or more combinations of the resolution, sharpness, and contrast of the image can be used as the sharpness indicator of the image. In some embodiments, the sharpness indicator can be normalized to a preset numerical range, such as the interval 1-100, or the interval 0-1, etc. In some embodiments, the difference between the sharpness indicators of the first image and the second image can be calculated, and the difference is compared with the first threshold. The first threshold can be pre-configured according to the user's requirement for image quality and the value space of the sharpness indicator. For example, the first threshold can be set to 15, 20, 30, 0.4, 0.5, etc. When the difference is greater than the first threshold, it can be considered that the sharpness difference between the first image and the second image is large, and the two images can be selectively output, thereby meeting the user's requirement for image sharpness. When the difference is less than or equal to the first threshold, it can be considered that the sharpness difference between the two images is small, and the two images can be synthesized to obtain a synthesized image with better sharpness. The calculation method of the image sharpness indicator and more descriptions of image synthesis can be found elsewhere in this specification, and will not be repeated here.

[0038] In some embodiments, the image quality can be sharpness, which can refer to the degree of clarity of textures, boundaries, etc. on the image. Generally, when the contrast between light and dark regions in the image is large, and the texture details are rich, the image can be considered to be clearer. The sharpness indicator of the image can reflect the sharpness of the image. The higher the sharpness indicator, the better the sharpness of the image. The sharpness of the image can be related to the resolution, sharpness, or contrast of the image, etc. In order to facilitate measurement, in some embodiments, one or more combinations of the resolution, sharpness, and contrast of the image can be used as the sharpness indicator of the image. In some embodiments, the sharpness indicator can be normalized to a preset numerical range, such as the interval 1-100, or the interval 0-1, etc. In some embodiments, the difference between the sharpness indicators of the first image and the second image can be calculated, and the difference is compared with the first threshold. The first threshold can be pre-configured according to the user's requirement for image quality and the value space of the sharpness indicator. For example, the first threshold can be set to 15, 20, 30, 0.4, 0.5, etc. When the difference is greater than the first threshold, it can be considered that the sharpness difference between the first image and the second image is large, and the two images can be selectively output, thereby meeting the user's requirement for image sharpness. When the difference is less than or equal to the first threshold, it can be considered that the sharpness difference between the two images is small, and the two images can be synthesized to obtain a synthesized image with better sharpness. The calculation method of the image sharpness indicator and more descriptions of image synthesis can be found elsewhere in this specification, and will not be repeated here.

[0039] In some embodiments, when operating in the image selection mode, the image processing unit can further output the first image, or output the second image, or output both the first image and the second image based on a second instruction. The second instruction can be an image selection instruction input by a user. As an example, a button, a knob, a selection switch, or the like can be provided on an operation unit (e.g., the operation unit 112), and the user can input the second instruction through the aforementioned control to select the first image and the second image. In some alternative embodiments, the aforementioned control can be provided on a processing device (e.g., the processing device 120), and the user can input the second instruction through the control on the processing device. In some further alternative embodiments, the processing device can control a display device (e.g., the display device 130) to display a graphical user interface, and the graphical user interface can display a control, and the user can input the second instruction by triggering the control on the graphical user interface. In some embodiments, the second instruction can be converted into an electrical signal and input to a control unit or a processor of the processing device. When the second instruction is an electrical signal in a third state, it indicates that the user selects the first image. When the second instruction is an electrical signal in a fourth state, it indicates that the user selects the second image. When the second instruction is an electrical signal in a fifth state, it indicates that the user selects to output both the first image and the second image.

[0040] In some embodiments, an image area can be determined on the aforementioned graphical user interface, and the first image, the second image, or both the first image and the second image can be displayed in the image area according to the user’s selection. When the user selects to output both the first image and the second image, as an example, the first image and the second image can be displayed side by side. The clear areas on the images output by the imaging modules with different focus ranges can be different, and displaying both the first image and the second image can help the user to diagnose the object under examination by combining the images with different clear areas of the same object under examination. It can be understood that the clear area on the image has a higher clarity than other areas.

[0041] In some embodiments, the image sensor of the imaging module can output images at a preset frequency (e.g., 1 frame / s, 5 frames / s, etc.). These images can be displayed in sequence in the image area in time sequence, or the first image or the second image can be a video-form image. In some embodiments, when the user selects the first image, or selects the second image, or simultaneously selects the first image and the second image, it can be understood that the user selects the first imaging module, or selects the second imaging module, or simultaneously selects both imaging modules, at this time, the imaging module corresponding to the image not selected by the user for output can stop working, specifically, the control part of the endoscope system can control the image sensor of the imaging module to stop image acquisition or output image. In this way, the energy consumption of the imaging module can be reduced, on the other hand, the image sensor will generate a certain amount of heat when working, so the heat accumulation at the tip of the endoscope insertion part can be reduced, and the influence on the subject being examined can be reduced.

[0042] In some embodiments, the image processing unit can output the image with higher image definition from the first image and the second image when the device is working in the image selection mode. Specifically, the image processing unit can obtain the definition indication value of the first image and the second image respectively, and output the image with higher definition indication value. More details about calculating the definition indication value can be found elsewhere in the specification, and will not be repeated here. In some embodiments, the first imaging module and the second imaging module can output the first image and the second image at the same frequency, and the image processing unit can compare the definition of the first image and the second image corresponding to each other in time sequence, and output the image with higher definition. In some other embodiments, the first imaging module and the second imaging module output images at different frequencies, and the image processing unit can select one or more first images and one or more second images respectively, and compare the definition of the one or more first images and the one or more second images. When the number of first images with higher definition is greater than the number of second images with higher definition, the first image of the first imaging module is selected for output, and the control unit can turn off the second imaging module. When the number of second images with higher definition is greater than the number of first images with higher definition, the second image of the second imaging module is selected for output, and the control unit can turn off the first imaging module. As an example, the image processing unit can select 5 first images and 5 second images from the images output by the two imaging modules in the same time period (such as the time period from 11:12 to 11:13), compare the definition of the first image and the second image with the same relative time sequence as a group (for example, compare the first image output first in the foregoing time period and the second image output first as a group, compare the second image with the second relative time sequence in the five first images and the second relative time sequence in the five second images as a group, and so on to obtain five groups of images), and determine whether the image with higher definition in each group of images is the first image or the second image. When the comparison results of each group show that the definition of the first image is higher in 3 groups, the first image of the first imaging module is selected for output, and the control unit can turn off the second imaging module.

[0043] In some embodiments, the control unit can be implemented by a processor in the processing device. In some embodiments, the control unit can obtain the second instruction or the selection result of the image processing unit, and control the imaging module corresponding to the image not selected for output to stop working. In some embodiments, the control unit and the image processing unit can be the same component or different components, for example, the control unit and the image processing unit can be implemented by the same processor, and for another example, the control unit and the image processing unit can be implemented by different computer codes executed by the same processor.

[0044] In some embodiments, when operating in the image synthesis mode, the image processing unit can determine one or more target image sub-regions from the first image and the second image respectively based on the sharpness; and stitch the one or more target image sub-regions to obtain a synthesized image. More details about image synthesis and determination of target image sub-regions can be found in the related description of Figure 4 and will not be repeated here.

[0045] In some embodiments, in order to reduce the system energy consumption, the frequency of the imaging module outputting images can be controlled. Specifically, the sampling frequency of the image sensor can be adjusted to reduce the system energy consumption as much as possible under the premise of meeting the image requirements, and to avoid heat accumulation at the front end of the imaging module. In some embodiments, the frequency of the imaging module outputting images can be referred to as the image output frequency, which can specifically be understood as the number of images (such as the number of image frames) output by the imaging module in a unit time (such as one minute or one second). In some embodiments, the image output frequency of the imaging module can be determined by the sampling frequency of the image sensor inside it, so the image output frequency of the imaging module can be adjusted by adjusting the sampling frequency of the image sensor. In some embodiments, the control unit can adjust the image output frequency of at least one of the first imaging module and the second imaging module based on the sharpness of the first image and the second image. Specifically, the control unit can obtain the sharpness indication values of the first image and the second image; and control the image output frequency of the imaging module corresponding to the image with the higher sharpness indication value to be higher than the image output frequency of the imaging module corresponding to the image with the lower sharpness indication value. As an example, the control unit can directly process the first image and the second image to obtain the sharpness indication values of the two images, or the control unit can obtain the sharpness indication values of the first image and the second image from the image processing unit. When the sharpness indication value of the first image is higher than that of the second image, the control unit can increase the image output frequency of the first imaging module (such as adjusting from 3 frames / s to 5 frames / s), or the control unit can decrease the image output frequency of the second imaging module (such as adjusting from 3 frames / s to 1 frame / s). In some embodiments, the control unit can obtain the sharpness indication values of the first image and the second image respectively at a certain period, and determine the image with the higher sharpness indication value. The aforementioned period can be 20s, 30s, 1 min, 2 min, etc. Further, the control unit can dynamically adjust the image output frequency of the first imaging module or the second imaging module according to the comparison result of the sharpness indication values. As a continuation of the previous example, when the sharpness indication value of the second image is higher than that of the first image at the next comparison of the sharpness indication values, the control unit can increase the image output frequency of the second imaging module (such as adjusting from 1 frame / s to 5 frames / s), or the control unit can decrease the image output frequency of the first imaging module (such as adjusting from 5 frames / s to 1 frame / s).

[0046] In some embodiments, the control unit can further adjust the image output frequency of the first imaging module or the second imaging module based on the difference between the sharpness indication values of the first image and the second image. Specifically, the control unit can obtain the sharpness indication values of the first image and the second image; when the difference between the sharpness indication values of the first image and the second image exceeds a second threshold value, control the image output frequency of the imaging module corresponding to the image with the higher sharpness indication value to be higher than the image output frequency of the imaging module corresponding to the image with the lower sharpness indication value; and when the difference between the sharpness indication values of the first image and the second image does not exceed the second threshold value, make the image output frequencies of the first imaging module and the second imaging module the same. In some embodiments, the difference between the sharpness indication values of the first image and the second image can be calculated to represent the difference between the two images, and the second threshold value can be determined based on the energy consumption or heat dissipation of the imaging module, the requirements of image quality, and the value range of the sharpness indication value. The second threshold value can be set to be the same as the first threshold value, or can be different.

[0047] To adapt to the endoscope system provided by some embodiments of the present specification, some other embodiments of the present specification further provide an image processing method. Figure 4 An exemplary flowchart of the image processing method shown in some embodiments of the present specification. Figure 4 The flowchart 400 shown can be executed by a processing device (such as the processing device 120), and further can be implemented by an image processing unit or the image processing system 800 in the processing device. As shown in Figure 4 The image processing method shown in some embodiments of the present specification can include the following steps.

[0048] Step 410: obtaining a first image and a second image. In some embodiments, step 410 can be implemented by the obtaining module 810.

[0049] In some embodiments, the image processing unit can receive a first image and a second image from a first imaging module and a second imaging module. In some embodiments, the imaging module can output images at a preset frequency, and accordingly, the image processing unit can obtain the first image and the second image output by the first imaging module and the second imaging module frame by frame.

[0050] Thereafter, the flowchart 400 can optionally execute to step 420 or step 430. For example, step 420 or step 430 can be executed based on a first instruction, and for another example, the sharpness indication values of the first image and the second image can be obtained; when the difference between the sharpness indication values of the first image and the second image exceeds a first threshold value, step 420 is executed; and when the difference between the sharpness indication values of the first image and the second image does not exceed the first threshold value, step 430 is executed. More information about the sharpness indication value and the first threshold value can be found in the foregoing, which will not be repeated here.

[0051] In some embodiments, the image indicators such as contrast, sharpness or resolution of the first image and the second image can be calculated respectively, and the distinctness of image value of the first image and the second image can be obtained based on a combination of one or more of them. Figure 5 An exemplary flowchart for obtaining the distinctness of image is shown for some other embodiments of the present disclosure. It should be understood that the image described in flow 5 can be the first image or the second image. In some embodiments, flow 500 can be implemented by the image processing unit or processing module 820. As shown, flow 500 can include the following steps. Figure 5

[0052] Step 510, divide the image into N image sub-regions.

[0053] In some embodiments, N can be an integer greater than 1, for example, N can take 9, 12, 16 or larger values. As an example, the image can be divided into 3x3 squares to obtain 9 image sub-regions, or divided into 3x5 squares to obtain 15 image sub-regions. In some embodiments, the first image can be divided into N1 image sub-regions, and the second image can be divided into N2 image sub-regions, N1 and N2 can be the same or different. In some embodiments, the size of the image sub-regions in the same image can be the same or different. The shape of the image sub-region can be rectangular, or triangular, pentagonal, hexagonal or other polygonal shape.

[0054] Step 520, calculate the distinctness of image value of each image sub-region in the image.

[0055] In some embodiments, the contrast, resolution, sharpness, etc. of each image sub-region in the first image can be calculated respectively, and the distinctness of image value can be obtained based on a combination of one or more of them. The contrast, resolution, sharpness, etc. of each image sub-region in the second image can be calculated respectively, and the distinctness of image value can be obtained based on a combination of one or more of them. As an example, the distinctness of image value can be obtained based on the sum of the contrast and sharpness of a certain image sub-region, or the contrast of a certain image sub-region can be taken as the distinctness of image value.

[0056] Step 530, determine the influence weight of each image sub-region in the image.

[0057] In some embodiments, a constant influence weight can be set for different image sub-regions of the image. For example, the influence weight of the image sub-region located in the center of the image can be set to a larger constant value, and the influence weight of the image sub-region located at the edge of the image can be set to a smaller constant value. In some other embodiments, the influence weight of different image sub-regions in the image can be determined based on the photographed object.

[0058] ​Specifically, the image processing unit can identify either the first image or the second image to determine the current subject. In some embodiments, the image processing unit can input the first image or the second image into a trained image recognition model to determine the current subject. Since the first image and the second image can image the same subject, the subject can be determined by identifying only one of the first and second images. The image recognition model can be a neural network machine learning model, specifically a convolutional neural network model, etc. The image recognition model can be trained using sample images and corresponding labels to enable it to recognize the subject in the image. The label is used to indicate the type or name of the object being photographed in its corresponding image; for example, the label can be the name of an organ or tissue of an organism, such as "stomach" or "intestines."

[0059] In some embodiments, the influence weights of each image sub-region in the corresponding image can be determined in advance based on the physiological structural characteristics of the subject and the diagnostic needs of the doctor, thereby establishing a correspondence between the subject and the influence weights. For example, when the subject is the stomach, because the stomach has a large space and relatively flat tissue, the doctor mainly observes the image center to make a diagnostic judgment. In this case, the image sub-regions closer to the image center can be set to have relatively higher weight values, while the image sub-regions closer to the image periphery can have relatively lower weight values. As another example, when the subject is the intestines, the intestines have a tubular structure, and the intestinal wall tissue is located on the periphery of the image. In this case, all image sub-regions can be set to have the same weight value, or the image sub-regions closer to the image center can have relatively lower weight values, while the image sub-regions closer to the image periphery can have relatively higher weight values. Taking the image divided into 3×3 image sub-regions (a nine-square grid) as an example, when the subject is the stomach, the influence weights of each image sub-region in the corresponding image are: When the subject being photographed is the intestines, the influence weights of each image sub-region in the corresponding image are: After determining the current subject, the image processing unit can determine the influence weight of each image sub-region in the first image and the influence weight of each image sub-region in the second image based on the correspondence between the subject and the influence weight.

[0060] Step 540: Determine the image sharpness based on the sharpness indicator values ​​of each image sub-region in the image and the influence weights.

[0061] Let's assume that the sharpness indicator values ​​of each sub-region in the first image are t. 11 t 12 、…、t 1N1 The influence weight of each image sub-region is s. 11 s 12 、…、s 1N1Therefore, the sharpness or sharpness indicator value of the first image can be calculated (s). 11 ×t 11 +s 12 ×t 12 +…+s 1N1 ×t 1N1 Assume that the sharpness indicator values ​​of each sub-region in the second image are t. 21 t 22 、…、t 2N2 The influence weight of each image sub-region is s. 21 s 22 、…、s 2N2 The sharpness or sharpness indicator value of the second image can be calculated (s). 21 ×t 21 +s 22 ×t 22 +…+s 2N2 ×t 2N2 ).

[0062] Step 420: Select at least one of the first image and the second image and output it. In some embodiments, step 420 may be implemented by processing module 820.

[0063] In some embodiments, a first image, a second image, or both images can be output based on a second instruction. More details about the second instruction can be found above and will not be repeated here. In still other embodiments, the image with higher resolution between the first and second images can be output. For information on methods for obtaining image resolution, please refer to [link to relevant documentation]. Figure 5 The relevant explanations will not be repeated here.

[0064] Step 430: Obtain and output a composite image based on the first image and the second image. In some embodiments, step 430 may be implemented by processing module 820.

[0065] In some embodiments, the imaging ranges of the first image and the second image overlap. For example, they may contain the same object being detected, but the clear areas they contain are different. In this case, the clarity of the image sub-regions corresponding to the same shooting area on the two images can be compared, and the image sub-region with higher clarity can be selected as the target image sub-region. By analogy, the target image sub-regions corresponding to different shooting areas on the first image or the second image can be obtained. Finally, these target image sub-regions can be stitched together to obtain a composite image with more clear areas.

[0066] Figure 6 This is an exemplary flowchart illustrating the determination of a target image sub-region according to some embodiments of this specification. In some embodiments, Figure 6The process 600 shown can be implemented by the processing module 820. For example... Figure 6 As shown, process 600 may include the following steps.

[0067] Step 610: Register the first image and the second image.

[0068] In some embodiments, the sampling times of the first image and the second image can be the same, or the interval time can not exceed a set time threshold. For example, the time threshold can be 0.1s, 0.5s, 1s, or 2s, etc., which can ensure that the subjects in the first image and the second image have as similar a state as possible. As mentioned above, in some embodiments, the optical lenses in the first imaging module and the second imaging module can be set to have a smaller size (e.g., a smaller diameter) or the distance between the principal optical axes of the first imaging module and the second imaging module can be made as small as possible, thereby ensuring that the two imaging modules have the most similar imaging range possible. In other words, the contents of the first image and the second image can have a high degree of overlap.

[0069] In some embodiments, to improve the quality of image synthesis, the first image and the second image can be registered first. Registration can be understood as aligning two or more images in space. In some embodiments, due to differences in the position and shooting angle of different imaging modules, the same subject in the first image and the second image may have different sizes or orientations. In order to align the first image and the second image in space, at least one of the images can be subjected to one or more of the following processes: scaling, rotation, and translation. Figure 7 This is a schematic diagram illustrating the image registration principle in some embodiments of this specification. Figure 7 Image 710 is the first image, and image 720 is the second image. It can be seen that the same subject (using a triangle as an example) has different orientations in the first and second images. To achieve image registration, feature points of the subject (such as feature points at the pylorus of the stomach, feature points at the cardia, etc.) can be identified in the first and second images respectively as positioning reference points. Figure 7The first image 710 contains positioning reference points A1, B1, and C1, while the second image 720 contains positioning reference points A2, B2, and C2. Further, coordinate systems can be established for the first image 710 and the second image 720, such as using the top-left corner of each image as the origin, with the horizontal direction to the right as the positive x-axis and the vertical direction downwards as the positive y-axis. The coordinate values ​​of positioning reference points A1, B1, and C1 in the first image coordinate system and the coordinate values ​​of positioning reference points A2, B2, and C2 in the second image coordinate system are obtained. Spatial transformations (scaling, translation, or rotation) can be performed on the pixels in the first image to ensure that the transformed coordinate values ​​of positioning reference points A1, B1, and C1 correspond to the coordinate values ​​of positioning reference points A2, B2, and C2. Thus, when the transformed first image 711 overlaps with the second image 720, the three positioning reference points can also coincide one by one, thereby achieving spatial alignment of the two images.

[0070] In other embodiments, registering the first and second images may further include reducing the following differences between them: brightness difference and color difference. In some embodiments, the brightness of pixel values ​​in the other image may be adjusted based on the image with lower brightness, thereby reducing the brightness difference between the two. For example, the hue or color saturation of the two images may be adjusted to reduce the color difference between them.

[0071] Step 620: Divide the registered first image and the second image into M first image sub-regions and M second image sub-regions, respectively.

[0072] M can be an integer greater than 1. In some embodiments, M first image sub-regions correspond one-to-one with M second image sub-regions. This correspondence can be understood as the first and second image sub-regions having the same or a repetition rate greater than 80% or a higher threshold of image content. Furthermore, the corresponding first and second image sub-regions may also have the same shape and size. In some embodiments, the registered first and second images can be divided in the same way, resulting in M ​​first image sub-regions that correspond one-to-one with M second image sub-regions.

[0073] Step 630: Calculate the sharpness indicator value for each first image sub-region and each second image sub-region.

[0074] The method for calculating the image sub-region sharpness indicator is the same as the method for calculating the image sharpness indicator. For the specific calculation method, please refer to the relevant explanation on calculating the image sharpness indicator in the previous text, which will not be repeated here.

[0075] Step 640: Select the one with the higher sharpness indicator value from the corresponding first image sub-region and second image sub-region as the target image sub-region, thereby obtaining M target image sub-regions.

[0076] In some embodiments, corresponding first and second image sub-regions can be grouped together, resulting in M ​​groups of image sub-regions. For each group of image sub-regions, the one with the higher sharpness indicator value can be selected as the target image sub-region, thus obtaining M target image sub-regions.

[0077] Finally, the M target image sub-regions are stitched together to obtain the composite image. Since the target image sub-regions with better clarity are selected in each group of image sub-regions, the resulting composite image can have more clear areas, meaning the overall clarity of the composite image will be better.

[0078] Figure 8 These are exemplary block diagrams of image processing systems shown in some embodiments of this specification. Figure 8 As shown, the image processing system 800 may include an acquisition module 810 and a processing module 820.

[0079] The acquisition module 810 is used to acquire a first image and a second image. The processing module 820 is used to select at least one of the first image and the second image and output it, or to obtain a composite image based on the first image and the second image and output it.

[0080] For more information on each module, please refer to [link / reference]. Figures 4 to 7 The relevant explanations will not be repeated here. It should be understood that... Figure 8 The systems and modules shown can be implemented in various ways. For example, in some embodiments, the systems and modules can be implemented by hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated hardware. Those skilled in the art will understand that the methods and systems described above can be implemented using computer-executable instructions and / or included in the control code of a processor, such as on a media such as a disk, CD, or DVD-ROM, or in the memory of a programmable device. The systems and modules of this specification can be implemented not only by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., but also by software, for example, executed by various types of processors, or by a combination of the aforementioned hardware circuits and software (e.g., firmware).

[0081] It should be noted that the above description of the system and its modules is for convenience only and should not be construed as limiting this specification to the embodiments described. It is understood that those skilled in the art, after understanding the principles of this system, may arbitrarily combine the various modules without departing from these principles to form subsystems connected to other modules. Alternatively, some modules may be split to obtain more modules or multiple units under a single module. Such modifications are all within the scope of this specification.

[0082] One or more embodiments of this specification also provide a computer program product, including computer instructions or computer code, which, when at least a portion of the computer instructions or computer code is executed by a processor, enables the implementation of the image processing method described above.

[0083] One or more embodiments of this specification also provide a computer-readable storage medium storing computer instructions or computer code that, when at least a portion of the computer instructions or computer code is executed by a processor, enables the implementation of the image processing method described above.

[0084] In some embodiments, the aforementioned processor may be a combination of one or more of the following processors: central processing unit (CPU), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), graphics processing unit (GPU), physical processing unit (PPU), digital signal processor (DSP), field-programmable gate array (FPGA), programmable logic device (PLD), programmable logic controller (PLC), reduced instruction set computer (RISC), and microprocessor.

[0085] In some embodiments, the aforementioned storage medium may include one or more combinations of the following: mass storage, removable storage, volatile read-write memory, and read-only memory (ROM). Exemplary mass storage may include disks, optical disks, solid-state drives, etc. Exemplary removable storage may include flash drives, floppy disks, optical disks, memory cards, compressed hard disks, magnetic tapes, etc. Exemplary volatile read-write memory may include random access memory (RAM). Exemplary RAM may include dynamic random access memory (DRAM), dual data rate synchronous dynamic random access memory (DDRSDRAM), static random access memory (SRAM), silicon controlled retrieval memory (T-RAM), and zero-capacitance memory (Z-RAM), etc. Exemplary read-only memory may include masked read-only memory (MROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compressed hard disk read-only memory (CD-ROM), and digital multifunction hard disk read-only memory, etc.

[0086] The beneficial effects that the embodiments of this specification may bring include, but are not limited to: (1) using dual imaging modules to acquire endoscopic images can effectively expand the focusing range or depth of field of the endoscopic system; (2) the two imaging modules have their own optical lenses, and the focal position of the optical lenses can be flexibly selected, so that the focusing range of the two imaging modules is significantly different and adjacent to each other, thus greatly expanding the overall focusing range of the endoscopic system; (3) using injection-molded lenses can make the diameter of the insertion part of the endoscopic system smaller even if dual imaging modules are used; (4) the endoscopic system can work in image selection mode and image synthesis mode to meet the different imaging needs of users; (5) automatically displaying a first image, a second image or a synthesized image with better clarity, improving the overall imaging clarity of the endoscopic system; (6) adjusting the image output frequency of the first imaging module and the second imaging module based on the clarity, reducing system energy consumption while ensuring imaging quality, and effectively avoiding heat accumulation at the tip of the endoscope, thus greatly reducing the impact on the object being examined, etc. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects may be any one or a combination of the above, or any other possible beneficial effects.

[0087] The basic concepts have been described above. It is clear that the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, various modifications, improvements, and corrections may be made to this specification by those skilled in the art. Such modifications, improvements, and corrections are taught in this specification and therefore remain within the concept and scope of the exemplary embodiments described herein.

Claims

1. An endoscope system, characterized in that, It includes a light source, an insertion part, and an image processing part. The insertion part has at least a tip configured to penetrate into the area to be inspected. The tip is provided with a first imaging module and a second imaging module. The first imaging module includes a first optical lens and a first image sensor. The first optical lens is used to acquire a first optical image, and the first image sensor is used to receive the first optical image and convert it into the first image. The second imaging module includes a second optical lens and a second image sensor. The second optical lens is used to acquire a second optical image, and the second image sensor is used to receive the second optical image and convert it into the second image. The principal optical axis of the first optical lens is parallel to the principal optical axis of the second optical lens. The light source is used to provide illumination to the area being inspected; The first imaging module is used to acquire the first image; The first imaging module has a first focusing range and a first imaging range; The second imaging module is used to acquire the second image; The second imaging module has a second focus range and a second imaging range; the second focus range and the first focus range are at least partially different, and the second imaging range and the first imaging range have an intersection; The image processing unit is used to select at least one of the first image and the second image and output it, or to obtain and output a composite image based on the first image and the second image, wherein: sharpness indicator values ​​of the first image and the second image are obtained; one or more target image sub-regions are determined from the first image and the second image based on the sharpness indicator values; and the one or more target image sub-regions are stitched together to obtain the composite image; The endoscope system has an image selection mode and an image synthesis mode; wherein: The image processing unit is configured to: operate in the image selection mode based on a first instruction to select at least one of the first image and the second image and output it; or operate in the image synthesis mode to obtain a synthesized image based on the first image and the second image and output it; or... The image processing unit is configured to: when the difference in sharpness indicator values ​​between the first image and the second image exceeds a first threshold, operate in the image selection mode to select at least one of the first image and the second image and output it; When the difference in sharpness indicator values ​​between the first image and the second image does not exceed the first threshold, the image synthesis mode is used to obtain and output a synthesized image based on the first image and the second image.

2. The endoscope system according to claim 1, characterized in that, When operating in the image selection mode, the image processing unit is further configured to output the first image, or output the second image, or output both the first image and the second image simultaneously based on the second instruction.

3. The endoscope system according to claim 1, characterized in that, When operating in the image selection mode, the image processing unit is further used to output the image with higher clarity between the first image and the second image.

4. The endoscope system according to claim 3, characterized in that, In order to output the image with higher image clarity between the first image and the second image, the image processing unit is further configured to: Divide the first image into N1 image sub-regions, where N1 is an integer greater than 1; calculate the sharpness indicator value for each image sub-region in the first image; Determine the influence weight of each image sub-region in the first image; The sharpness of the first image is determined based on the sharpness indicator value of each image sub-region in the first image and the influence weight; Divide the second image into N2 image sub-regions, where N2 is an integer greater than 1; calculate the sharpness indicator value for each image sub-region in the second image; Determine the influence weight of each image sub-region in the second image; The sharpness of the second image is determined based on the sharpness indicator value of each image sub-region in the second image and the influence weight; Compare the sharpness of the first image and the second image, and output the image with higher sharpness between the first image and the second image; Among them, N1 and N2 may be equal or unequal.

5. The endoscope system according to claim 4, characterized in that, The image processing unit is also used to determine the influence weight of each image sub-region in the first image or the second image; To determine the influence weights of each image sub-region in the first image or the second image, the image processing unit is further configured to: Identify the first image or the second image to determine the current subject being photographed; Based on the correspondence between the photographed object and the influence weight, the influence weight of each image sub-region in the first image or the second image is determined.

6. The endoscope system according to claim 1, characterized in that, In order to determine one or more target image sub-regions from the first image and the second image respectively based on the sharpness indicator value, the image processing unit is further configured to: Register the first image with the second image; The registered first image and the second image are divided into M first image sub-regions and M second image sub-regions, respectively, where M is an integer greater than 1, and the M first image sub-regions correspond one-to-one with the M second image sub-regions; Calculate the sharpness indicator value for each first image sub-region and each second image sub-region respectively; Select the one with the higher sharpness indicator value from the corresponding first image sub-region and second image sub-region as the target image sub-region, and then obtain M target image sub-regions.

7. The endoscope system according to claim 6, characterized in that, In order to register the first image with the second image, the image processing unit is further configured to: Perform one or more of the following operations on at least one of the first image and the second image: scaling, translation, rotation, so that two or more positioning reference points in the first image and two or more positioning reference points in the second image are aligned one by one; Alternatively, reduce the following differences between the first image and the second image: brightness difference and color difference.

8. The endoscope system according to claim 1, characterized in that, It also includes the control unit; The control unit is used to adjust the image output frequency of at least one of the first imaging module and the second imaging module based on the sharpness of the first image and the second image.

9. The endoscope system according to claim 8, characterized in that, The control unit is further used for: Obtain the sharpness indicator values ​​of the first image and the second image; The image output frequency of the imaging module corresponding to the image with a higher sharpness indicator value in the first image and the second image is controlled to be higher than the image output frequency of the imaging module corresponding to the image with a lower sharpness indicator value.

10. The endoscope system according to claim 8, characterized in that, The control unit is further used for: Obtain the sharpness indicator values ​​of the first image and the second image; When the difference in sharpness indicator values ​​between the first image and the second image exceeds a second threshold, the image output frequency of the imaging module corresponding to the image with the higher sharpness indicator value is controlled to be higher than the image output frequency of the imaging module corresponding to the image with the lower sharpness indicator value. When the difference in sharpness indicator values ​​between the first image and the second image does not exceed a second threshold, the image output frequencies of the first imaging module and the second imaging module are made to be the same.

11. The endoscope system according to claim 1, characterized in that, The first optical lens and the second optical lens are fixed-focus lenses; The focal point of the first optical lens is located in the area of ​​7mm to 15mm on its front surface, and the focal point of the second optical lens is located in the area of ​​2mm to 6mm on its front surface.

12. The endoscope system according to claim 1, characterized in that, The distance between the principal optical axis of the first optical lens and the principal optical axis of the second optical lens is less than the radius of the tip end.

13. An endoscope insertion part, characterized in that, It has at least a tip configured to penetrate into an area to be inspected, the tip being provided with a first imaging module and a second imaging module; the first imaging module includes a first optical lens and a first image sensor, the first optical lens being used to acquire a first optical image, and the first image sensor being used to receive the first optical image and convert it into the first image; the second imaging module includes a second optical lens and a second image sensor, the second optical lens being used to acquire a second optical image, and the second image sensor being used to receive the second optical image and convert it into the second image; the principal optical axis of the first optical lens is parallel to the principal optical axis of the second optical lens; The first imaging module is used to acquire the first image; The first imaging module has a first focusing range and a first imaging range; The second imaging module is used to acquire the second image; The second imaging module has a second focus range and a second imaging range; the second focus range and the first focus range are at least partially different, and the second imaging range and the first imaging range have an intersection.

14. An image processing method for an endoscope system, characterized in that, include: Acquire a first image and a second image, the first image and the second image having different focal regions, and the imaging range corresponding to the first image and the imaging range corresponding to the second image having an intersection; The first image and the second image are output by a first imaging module and a second imaging module. The first imaging module includes a first optical lens and a first image sensor. The first optical lens is used to acquire a first optical image, and the first image sensor is used to receive the first optical image and convert it into the first image. The second imaging module includes a second optical lens and a second image sensor. The second optical lens is used to acquire a second optical image, and the second image sensor is used to receive the second optical image and convert it into the second image. The principal optical axis of the first optical lens is parallel to the principal optical axis of the second optical lens. Select at least one of the first image and the second image and output it, or obtain a composite image based on the first image and the second image and output it, wherein: obtain the sharpness indicator value of the first image and the second image; determine one or more target image sub-regions from the first image and the second image respectively based on the sharpness indicator value; and stitch the one or more target image sub-regions together to obtain the composite image; The image processing method includes an image selection mode and an image synthesis mode, wherein: Based on the first instruction, it operates in the image selection mode to select and output at least one of the first image and the second image, or operates in the image synthesis mode to obtain and output a synthesized image based on the first image and the second image; or... When the difference in sharpness indicator values ​​between the first image and the second image exceeds a first threshold, at least one of the first image and the second image is selected and output; when the difference in sharpness indicator values ​​between the first image and the second image does not exceed the first threshold, a composite image is obtained based on the first image and the second image and output.

15. The image processing method according to claim 14, characterized in that, The step of selecting and outputting at least one of the first image and the second image includes: Output the image with higher image clarity between the first image and the second image.

16. The image processing method according to claim 15, characterized in that, The step of outputting the image with higher image clarity between the first image and the second image includes: The first image is divided into N1 image sub-regions, where N1 is an integer greater than 1; the sharpness indicator value of each image sub-region in the first image is calculated; the influence weight of each image sub-region in the first image is determined; the sharpness of the first image is determined based on the sharpness indicator value and influence weight of each image sub-region in the first image. The second image is divided into N2 image sub-regions, where N2 is an integer greater than 1; the sharpness indicator value of each image sub-region in the second image is calculated; the influence weight of each image sub-region in the second image is determined; the sharpness of the second image is determined based on the sharpness indicator value and influence weight of each image sub-region in the second image. Compare the sharpness of the first image and the second image, and output the image with higher sharpness between the first image and the second image; Among them, N1 and N2 may be equal or unequal.

17. The image processing method according to claim 16, characterized in that, Also includes: Determine the influence weight of each image sub-region in the first image or the second image; The determination of the influence weights of each image sub-region in the first image or the second image further includes: Identify the first image or the second image to determine the current subject being photographed; Based on the correspondence between the photographed object and the influence weight, the influence weight of each image sub-region in the first image or the second image is determined.

18. The image processing method according to claim 14, characterized in that, The step of determining one or more target image sub-regions from the first image and the second image based on the sharpness indicator value includes: Register the first image with the second image; The registered first image and the second image are divided into M first image sub-regions and M second image sub-regions, respectively, where M is an integer greater than 1, and the M first image sub-regions correspond one-to-one with the M second image sub-regions; Calculate the sharpness indicator value for each first image sub-region and each second image sub-region respectively; Select the one with the higher sharpness indicator value from the corresponding first image sub-region and second image sub-region as the target image sub-region, and then obtain M target image sub-regions.

19. The image processing method according to claim 18, characterized in that, The registration of the first image and the second image includes: Perform one or more of the following operations on at least one of the first image and the second image: scaling, translation, rotation, so that two or more positioning reference points in the first image and two or more positioning reference points in the second image are aligned one by one; Alternatively, reduce the following differences between the first image and the second image: brightness difference and color difference.

20. A computer-readable storage medium storing computer code or instructions that, when at least a portion of the computer code or instructions is executed by a processor, enables the implementation of the method as claimed in any one of claims 14 to 19.

Images