Endoscope system, image processing method and computer program product thereof

Abstract

The invention relates to an endoscope system, an image processing method and a computer program product thereof, and relates to the technical field of medical imaging. The endoscope system comprises a light source, an insertion part and an image processing part, the insertion part is at least provided with a tip part configured to stretch into an examined area, and the tip part is provided with a first imaging module and a second imaging module; the light source is used for providing illumination for an inspected area; the first imaging module is used for acquiring a first image; the first imaging module has a first focusing range and a first imaging range; the second imaging module is used for acquiring a second image; the second imaging module has a second focusing range and a second imaging range; the second focusing range and the first focusing range are at least partially different, and the second imaging range and the first imaging range have an intersection; the image processing section selects and outputs at least one of the first image and the second image, or obtains and outputs a composite image based on the first image and the second image.

Description

Technical Field

[0001] The present disclosure relates to the field of medical imaging technology, and in particular to an endoscope system, an image processing method and a computer program product thereof. Background Art

[0002] Endoscopic technology, as an imaging technology, is often used in the field of medical diagnosis. Endoscopic technology enables components with imaging functions to enter the interior of organisms (such as human bodies, animals, etc.) through natural channels or small incisions formed by surgery, and then obtain image information of the inspected area inside the organism to provide a more intuitive and accurate basis for diagnosis.

[0003] At present, endoscope technology has made significant progress and is still making progress. How to obtain clearer images is one of the development directions of endoscope technology. Some embodiments of this specification are intended to provide an endoscope system and a corresponding image processing method to improve the clarity of endoscopic images. Summary of the invention

[0004] According to a first aspect, one or more embodiments of the present specification provide an endoscope system, comprising a light source, an insertion portion and an image processing portion, wherein the insertion portion has at least a tip portion configured to probe into an inspected area, and the tip portion is provided with a first imaging module and a second imaging module: the light source is used to provide illumination for the inspected area; the first imaging module is used to acquire a first image; the first imaging module has a first focusing range and a first imaging range; the second imaging module is used to acquire a second image; the second imaging module has a second focusing range and a second imaging range; the second focusing range and the first focusing range are at least partially different, and the second imaging range and the first imaging range have an intersection; the image processing portion is used to select at least one of the first image and the second image and output it, or to obtain a composite image based on the first image and the second image and output it.

[0005] According to the second aspect, one or more embodiments of the present specification provide an endoscope insertion portion, which has at least a tip portion configured to probe into an inspected area, the tip portion being provided with a light source, a first imaging module and a second imaging module; the light source is used to provide illumination for the inspected area; the first imaging module is used to acquire a first image; the first imaging module has a first focusing range and a first imaging range; the second imaging module is used to acquire a second image; the second imaging module has a second focusing range and a second imaging range; the second focusing range and the first focusing range are at least partially different, and the second imaging range and the first imaging range have an intersection.

[0006] According to the third aspect, one or more embodiments of the present specification provide an image processing method for an endoscope system, comprising: acquiring a first image and a second image, the first image and the second image having different focal areas, and the imaging range corresponding to the first image and the imaging range corresponding to the second image having an intersection; selecting at least one of the first image and the second image and outputting it, or obtaining a composite image based on the first image and the second image and outputting it.

[0007] According to a fourth aspect, one or more embodiments of the present specification provide a computer-readable storage medium storing computer codes or instructions, which, when at least part of the computer codes or instructions are executed by a processor, can implement the image processing methods provided in some embodiments of the present specification.

[0008] According to a fifth aspect, one or more embodiments of the present specification provide a computer program product, including computer codes or instructions, which, when executed by a processor, can implement the image processing methods provided in some embodiments of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a schematic diagram of the structure of an endoscope system shown in some embodiments;

[0010] Figure 2 A schematic diagram of the structure of an insertion portion in an endoscope system shown in some embodiments of this specification;

[0011] Figure 3 A schematic diagram of the working mode of the endoscope system shown in some embodiments of this specification;

[0012] Figure 4 is an exemplary flow chart of an image processing method shown in some embodiments of this specification;

[0013] Figure 5 An exemplary flow chart of obtaining image clarity shown in some embodiments of this specification;

[0014] Figure 6 An exemplary flow chart for determining a target image sub-region according to some embodiments of this specification;

[0015] Figure 7 This is a schematic diagram of the image registration principle shown in some embodiments of this specification;

[0016] Figure 8 This is an exemplary block diagram of an image processing system shown in some embodiments of this specification.

[0017] Component markings in the figure: 100 endoscope system; 110 endoscope; 111 insertion portion; 111a distal end portion; 112 operating portion; 120 processing device; 130 display device; 140 light source; 21 first imaging module; 211 first optical lens; 212 first image sensor; 22 second imaging module; 221 second optical lens; 222 second image sensor; 31 optical fiber. DETAILED DESCRIPTION

[0018] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0019] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0020] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0021] Endoscopic technology, as an imaging technology, is often used in the field of medical diagnosis. Endoscopic technology enables components with imaging functions to enter the interior of organisms (such as human bodies, animals, etc.) through natural channels or small incisions formed by surgery, and then obtain image information of the inspected area inside the organism to provide a more intuitive and accurate basis for diagnosis.

[0022] Figure 1 FIG. 1 is a schematic diagram of the structure of an endoscope system shown in some embodiments. Figure 1As shown, the endoscope system 100 may include an endoscope 110 and a processing device 120. The endoscope 110 is used to obtain image data of the inspected area, and the processing device 120 is used to process the image data to obtain the medical image required for diagnosis. The processing device 120 can also control the endoscope 110 or some components thereof so that the image data can better meet the needs. In some embodiments, the endoscope system 100 may also include a display device 130, which can obtain the image output by the processing device 120 and display it to the user (such as a doctor). In other embodiments, the endoscope system 100 may not include the display device 130. When it is necessary to view the image, the endoscope system 100 can be connected to an external display device 130 to output the processed image to the display device 130 for display.

[0023] In some embodiments, the endoscope 110 may further include an insertion portion 111 and an operating portion 112, and at least a portion of the insertion portion 111 may be bent to introduce the imaging module into the inspected area in the body. In some embodiments, the insertion portion 111 may be a flexible tubular component of a certain length, one end of which is connected to the operating portion 112, and the other end of which can be inserted into the inspected area in the body. The operating portion 112 can be held and operated by the user to adjust the depth or direction of the insertion portion 111, thereby adjusting the observation area and viewing angle of the endoscope.

[0024] In some embodiments, a section of the insertion portion 111 that can be inserted into the area to be examined in the living body can be referred to as the front end portion or the tip end portion 111a (it should be understood that Figure 1 The dotted circle in the figure is a mark used to indicate the position of the tip portion and should not be understood as a physical component in the endoscope system 100). The imaging module can be arranged at the tip portion 111a of the insertion portion to obtain image data of the inspected area. In some optional embodiments, the tip portion 111a of the insertion portion can also be provided with a light source to provide illumination for the inspected area, thereby enabling the imaging module to obtain an image with better recognition. Figure 2 In other alternative embodiments, the insertion portion 111 has an optical fiber 31 extending in the same direction as the insertion portion 111, one end face of the optical fiber 31 is exposed from the end face of the distal end portion 111a, and the light source 140 is arranged at the end of the optical fiber away from the distal end portion 111a. The optical fiber 31 can guide the light beam emitted by the light source 140 to the inspected area to provide illumination for it.

[0025] In some embodiments, the imaging module may include an optical lens and an image sensor. The optical lens may be composed of one or more optical elements (such as lenses), which are used to capture light from the detected area to form an optical image. The image sensor is used to receive the optical image and then generate a computer-recognizable medical image. Exemplarily, the image sensor may specifically be a CCD image sensor, a CMOS image sensor, etc.

[0026] In some embodiments, the optical lens of the endoscope imaging module may be a fixed-focus lens. The fixed-focus lens has a fixed focus, that is, the focus of the fixed-focus lens cannot be adjusted. The fixed-focus lens has the characteristic of being small in size relative to the zoom lens, and is suitable for the restrictive requirements of the endoscope on the size, but the disadvantage is that the focus range of the fixed-focus lens is limited. In some embodiments, the focus range can be understood as the spatial range in which the fixed-focus lens can clearly image, which is related to the focus of the fixed-focus lens. Specifically, the focus range can be characterized by the distance range from the main optical axis direction of the fixed-focus lens to the end face of the fixed-focus lens. As an example, the focus of a fixed-focus lens is located at a distance of 5 mm from the outer end face of the lens (or the front end face or outer surface, that is, the end face or surface on the side close to the object) in the direction of the main optical axis. The fixed-focus lens can clearly image objects or objects in a spatial range within a certain distance (such as 2 mm) from its focus, and this spatial range can be called the focus range. For intuitive understanding, it can also be said that the focus range of a fixed-focus lens is an interval of 1 mm to 7 mm from the main optical axis direction of the lens to the outer end face of the lens. In some embodiments, the focus range of the imaging module is limited by the focus or focus range of its optical lens. Therefore, the focus or focus range of the optical lens of the imaging module can also be determined as the focus or focus range of the imaging module.

[0027] It is not difficult to understand that the fixed-focus lens limits the space or distance range of clear imaging of the endoscope system. In practical applications, in order to obtain clear images in a larger range of the inspected area, doctors need to operate frequently so that the tip of the insertion part can reach more areas, which increases the complexity of the operation or the length of the inspection time. To this end, some embodiments of this specification provide an endoscope system, in which a dual imaging module is arranged, and the two imaging modules have different focal positions and similar or even the same imaging range, thereby expanding the focus space range of the endoscope system, so that the image obtained has better clarity.

[0028] Figure 2 is a schematic diagram of the structure of the insertion portion of the endoscope system shown in some embodiments of this specification, combined with Figure 1 and Figure 2The endoscope system provided in some embodiments of the present specification may include a light source 140, an insertion portion 111, and an image processing portion, wherein the insertion portion has at least a tip portion 111a configured to probe into the inspected area, and the tip portion 111a is provided with a first imaging module 21 and a second imaging module 22. The light source 140 is used to provide illumination for the inspected area. The first imaging module 21 is used to acquire a first image, and the first imaging module 21 has a first focusing range and a first imaging range. The second imaging module 22 is used to acquire a second image, and the second imaging module 22 has a second focusing range and a second imaging range, the second focusing range and the first focusing range are at least partially different, and the second imaging range and the first imaging range have an intersection. The image processing portion is used to select and output at least one of the first image and the second image, or to obtain and output a composite image based on the first image and the second image.

[0029] In some embodiments, the first imaging module 21 may further include a first optical lens 211 and a first image sensor 212, wherein the first optical lens 211 is used to obtain a first optical image, and the first image sensor 212 is used to receive the first optical image and convert it into a first image. The second imaging module 22 may further include a second optical lens 221 and a second image sensor 222, wherein the second optical lens 221 is used to obtain a second optical image, and the second image sensor 222 is used to receive the second optical image and convert it into a second image. For more descriptions of the optical lens and the image sensor, please refer to other parts of the specification, which will not be repeated here.

[0030] In some embodiments, the first optical lens and the second optical lens may have different focal points, so that the first focusing range and the second focusing range are at least partially different. As an example, the focus of the first optical lens may be located in an area of ​​7mm to 15mm on its front end face. Specifically, it can be understood that the focus of the first optical lens is located in a section of 7mm to 15mm from the main optical axis to the outer end face of the lens, and further, it can be in a section of 8mm to 12mm, or in a section of 9mm to 11mm. For example, the focus of the first optical lens may be located at a position of 7.5mm, 10mm, 11.5mm or 12mm from the main optical axis to the outer end face of the lens. In some embodiments, the focus of the second optical lens may be located in an area of ​​2mm to 6mm on its front end face. Specifically, it can be understood that the focus of the second optical lens is located in a section from 2mm to 6mm on the main optical axis to the outer end surface of the lens. Further, it can be located in a section from 3mm to 5mm. For example, the focus of the second optical lens can be located at a position from 2.5mm, 3.5mm, 4mm or 5mm on the main optical axis to the outer end surface of the lens.

[0031] The focusing range of the imaging module is affected by the focus of the optical lens. Since the focus positions of the first optical lens and the second optical lens are different, the first focusing range is at least partially different from the second focusing range. Among them, at least partially different can be understood as the first focusing range and the second focusing range are different but have an intersection, or the first focusing range and the second focusing range have no intersection. As an example, the distance from the focus of the first optical lens to the outer end face of the lens is 10mm, and the first focusing range may include an interval of 5mm to 50mm from the outer end face of the lens along the main optical axis of the lens. The distance from the focus of the second optical lens to the outer end face of the lens is 5mm, and the second focusing range may include an interval of 1mm to 7mm from the outer end face of the lens along the main optical axis of the lens. The first imaging module and the second imaging module have different focusing ranges, which can expand the overall focusing range of the endoscope system. Continuing with the previous example, the endoscope system with dual imaging modules can obtain a clear image of an object within a range of 1 to 50mm from the outer end face of the lens, which is larger than the focusing range of a single imaging module. By adopting a dual imaging module, the focal positions of the two optical lenses can be flexibly selected as needed, so that the two focusing ranges are quite different and adjacent to each other (such as the first focusing range corresponds to the interval of 7mm to 100mm, and the second focusing range corresponds to the interval of 1mm to 7mm), thereby greatly expanding the overall focusing range of the endoscope system (such as the overall focusing range corresponds to the interval of 1mm to 100mm).

[0032] In some embodiments, the imaging range can be understood as the field of view (FOV) of the imaging module or the spatial area captured by it. The imaging range can be intuitively regarded as the spatial area surrounded by a truncated cone with the end face of the imaging module as the upper base and the main optical axis of the lens as the central axis. Therefore, it can be considered that the imaging range is related to the position of the imaging module in space, the direction of the main optical axis of the lens, and the field of view angle (the angle between the truncated cone generatrix and the central axis).

[0033] Continue to refer Figure 2, the first imaging module 21 and the second imaging module 22 can be placed in parallel at the tip 111a of the insertion part, so that the main optical axis of the first optical lens 211 is parallel to the main optical axis of the second optical lens 221. Such an arrangement can make the imaging ranges of the two imaging modules have an intersection, for example, the same object to be detected (such as human tissue, organs, etc.) can be imaged. In some embodiments, the front end surface of the first optical lens 211 can be further made coplanar or flush with the front end surface of the second optical lens 221. In some embodiments, the optical lens of the imaging module can be a glass lens or an injection molded lens. Among them, the injection molded lens is a lens manufactured by injection molding technology, and the main materials include PC (polycarbonate), PMMA (polymethyl methacrylate), etc. The injection molded lens has the characteristics of high molding precision, light weight, good optical performance and low cost. The injection molded lens can have a smaller volume size than the glass lens, making it easy to place two optical lenses in parallel or side by side in the insertion part of the endoscope with a large diameter size restriction. In some embodiments, the two imaging modules can be brought close to or close to each other, or the size of the optical lens in the imaging module can be further reduced, so that the distance between the main optical axes of the two imaging modules is smaller than the radius of the tip. Such a setting can further increase the degree of proximity of the imaging ranges of the two imaging modules (such as the first image and the second image have an overlapping area of ​​80% or more than 90%), which is more suitable for image synthesis of the first image and the second image.

[0034] The endoscope system provided in some embodiments of this specification adopts two imaging modules (i.e., using two optical lenses and two image sensors for imaging), so that each optical lens corresponds to an image sensor, reducing the difficulty of assembling the optical lens and the image sensor. Specifically, in order to obtain a clear image, it is necessary to make the imaging surface of the optical lens and the photosensitive surface of the image sensor coincide. The one-to-one correspondence between the optical lens and the image sensor will help to significantly reduce the difficulty of assembling and adjusting the imaging surface of the optical lens and the photosensitive surface of the image sensor, reduce costs, and facilitate mass production of products. On the other hand, the one-to-one correspondence between the optical lens and the image sensor can also ensure that more light penetrating the optical lens can enter the corresponding image sensor, making the image brighter and easier to observe.

[0035] The image processing unit may be located in a processing device (such as Figure 1In the processing device 120 shown in the figure, specifically, the image processing unit can be a control unit or a processor for performing image processing. In some embodiments, the image processing unit is only used for image selection, and specifically, at least one can be selected from the first image and the second image and output. In some embodiments, the image processing unit can be only used for image synthesis, and specifically, a synthesized image can be obtained based on the first image and the second image and output. In some other embodiments, the image processing unit can be used for both image selection and image synthesis. In this case, it can be considered that the endoscope system has two working modes: image selection mode and image synthesis mode. Figure 3 is a schematic diagram of the working mode of the endoscope system shown in some embodiments of this specification, Figure 3 The image processing unit shown can operate in an image selection mode or an image synthesis mode as required.

[0036] In some embodiments, the image processing unit may work in an image selection mode based on a first instruction to select at least one of the first image and the second image and output it, or work in an image synthesis mode to obtain a synthesized image based on the first image and the second image and output it. Wherein, the first instruction may be a mode selection instruction input by a user. As an example, a button, a knob, a selection switch or other controls may be set on the operating unit (such as the operating unit 112), and the user may input the first instruction through the aforementioned controls to control the image processing unit to execute the image selection process (working in the image selection mode) or to execute the image synthesis process (working in the image synthesis mode). In other alternative embodiments, the aforementioned controls may be set on a processing device (such as the processing device 120), and the user may input the first instruction through the controls on the processing device. In yet other alternative embodiments, the processing device may control a display device (such as the display device 130) to display a graphical user interface, and controls may be displayed on the graphical user interface, and the user may trigger the controls on the graphical user interface to input the first instruction (such as by clicking the controls with a mouse or by touching the area corresponding to the controls on the touch screen of the display device 130). In some embodiments, the first instruction can be converted into an electrical signal and input into the control unit or processor of the processing device. When the first instruction is an electrical signal in a first state, it indicates that the working mode selected by the user is the image selection mode. When the first instruction is an electrical signal in a second state, it indicates that the working mode selected by the user is the image synthesis mode.

[0037] In other embodiments, the image processing unit may automatically select to operate in the image selection mode or the image synthesis mode according to the image quality of the first image and the second image. Specifically, the image processing unit may obtain the clarity indication value of the first image and the second image; when the difference in the clarity indication value of the first image and the second image exceeds a first threshold, the image processing unit may operate in the image selection mode to select at least one of the first image and the second image and output it; when the difference in the clarity indication value of the first image and the second image does not exceed the first threshold, the image processing unit may operate in the image synthesis mode to obtain a synthesized image based on the first image and the second image and output it.

[0038] In some embodiments, the image quality may be clarity, and clarity may refer to the clarity of textures, boundaries, etc. on the image. Generally speaking, when the contrast between light and dark areas in the image is large and the texture details are rich, the image can be considered to be relatively clear. The clarity indicator value of the image can reflect the clarity of the image. The higher the clarity indicator value, the better the clarity of the image. The clarity of the image may be related to indicators such as the resolution, sharpness, or contrast of the image. For the convenience of measurement, in some embodiments, a combination of one or more of the indicators such as the resolution, sharpness, and contrast of the image may be used as the clarity indicator value of the image. In some embodiments, the clarity indicator value may be normalized to a preset numerical range, such as an interval of 1 to 100, or an interval of 0 to 1, etc. In some embodiments, the difference between the clarity indicator values ​​of the first image and the second image may be calculated, and the difference may be compared with a first threshold. The first threshold may be pre-configured according to the user's requirements for image quality and the value space of the clarity indicator value. For example, the first threshold may be set to 15, 20, 30, 0.4, 0.5, etc. When the difference is greater than the first threshold, it can be considered that the clarity difference between the first image and the second image is large, and the two images can be selectively output to meet the user's image clarity requirements. When the difference is less than or equal to the first threshold, it can be considered that the clarity difference between the two images is small, and the two images can be synthesized, and the clear sub-areas of the two images can be taken to obtain a synthesized image with better clarity. More explanations on the calculation method of the image clarity index and image synthesis can be found elsewhere in this manual and will not be repeated here.

[0039] In some embodiments, when working in the image selection mode, the image processing unit may further output the first image, or output the second image, or output the first image and the second image simultaneously based on the second instruction. The second instruction may be an image selection instruction input by the user. As an example, a button, a knob, a selection switch or other controls may be set on the operating unit (such as the operating unit 112), and the user may input the second instruction through the aforementioned controls to select the first image and the second image. In other alternative embodiments, the aforementioned controls may be set on the processing device (such as the processing device 120), and the user may input the second instruction through the controls on the processing device. In some other alternative embodiments, the processing device may control the display device (such as the display device 130) to display a graphical user interface, and the graphical user interface may display controls, and the user may trigger the controls on the graphical user interface to input the second instruction. In some embodiments, the second instruction may be converted into an electrical signal and input into the control unit or processor of the processing device. When the second instruction is an electrical signal in the third state, it indicates the user to select the first image, when the second instruction is an electrical signal in the fourth state, it indicates the user to select the second image, and when the second instruction is an electrical signal in the fifth state, it indicates the user to select to output the first image and the second image simultaneously.

[0040] In some embodiments, the image area can be determined on the aforementioned graphical user interface, and only the first image, only the second image, or the first image and the second image can be displayed simultaneously in the image area according to the user's selection. When the first image and the second image are selected to be output simultaneously, as an example, the first image and the second image can be displayed side by side left and right. The clear areas on the images output by imaging modules with different focus ranges may be different. Displaying the first image and the second image simultaneously helps the user to diagnose the inspected object by combining images with different clear areas of the same inspected area. It can be understood that the clear area on the image has a higher clarity than other areas.

[0041] In some embodiments, the image sensor of the imaging module can output images at a preset frequency (such as 1 frame / s, 5 frames / s, etc.). These images can be displayed in the image area in sequence, or the first image or the second image can be an image in video form. In some embodiments, when the user selects the first image, or the second image, or the first image and the second image at the same time, it can be understood that the user has selected the first imaging module, or the second imaging module, or both imaging modules at the same time. At this time, the imaging module corresponding to the image not selected by the user to output can stop working. Specifically, the control unit of the endoscope system can control the image sensor of the imaging module to stop image acquisition or output images. In this way, the working energy consumption of the imaging module can be reduced. On the other hand, the image sensor will generate a certain amount of heat when working. Such a setting can also reduce the heat accumulation at the tip of the endoscope insertion part and reduce the impact on the object being inspected.

[0042] In other embodiments, when working in the image selection mode, the image processing unit can automatically output the one with higher image clarity between the first image and the second image. Specifically, the image processing unit can obtain the clarity indication values ​​of the first image and the second image respectively, and select the image with higher clarity indication value between the two to output. More information about calculating the image clarity indication value can be found elsewhere in the specification, and will not be repeated here. In some embodiments, the first imaging module and the second imaging module can output the first image and the second image at the same frequency, and the image processing unit can compare the clarity of the first image and the second image corresponding to each other in the time sequence, and output the image with higher clarity. In other embodiments, the first imaging module and the second imaging module have different frequencies of outputting images. At this time, the image processing unit can select one or more first images and one or more second images respectively, and compare the clarity of one or more first images and one or more second images. When the number of first images with higher clarity is greater than the number of second images with higher clarity, the first image of the first imaging module is selected for output, and the control unit can turn off the second imaging module. When the number of second images with higher definition is greater than the number of first images with higher definition, the second image output of the second imaging module is selected, and the control unit can turn off the first imaging module. As an example, the image processing unit can select 5 first images and 5 second images from the images output by the two imaging modules in the same time period (such as the time period from 11:12 to 11:13), respectively, and compare the definition of the first images and the second images with the same relative timing as a group (for example, the first image output first and the second image output first in the aforementioned time period are compared as a group, and the first image ranked second in the relative timing among the five first images and the second image ranked second in the relative timing among the five second images are compared as a group, and five groups of images are obtained by analogy), and determine whether the image with higher definition in each group of images is the first image or the second image. When there are 3 groups of comparison results in each group of comparison results that the first image has higher definition, the first image output of the first imaging module is selected, and the control unit can turn off the second imaging module.

[0043] In some embodiments, the aforementioned control unit can be implemented by a processor in a processing device. In some embodiments, the control unit can obtain a second instruction or obtain a selection result of the image processing unit, and control the imaging module corresponding to the image that is not selected for output to stop working. In some embodiments, the control unit and the image processing unit can be the same component or different components. For example, the control unit and the image processing unit can be implemented by the same processor, and for another example, the control unit and the image processing unit can be implemented by different computer codes executed by the same processor.

[0044] In some embodiments, when working in the image synthesis mode, the image processing unit can determine one or more target image sub-regions from the first image and the second image based on the clarity; and splice the one or more target image sub-regions to obtain a synthesized image. More information about image synthesis and determining target image sub-regions can be found in Figure 4 can be found in the relevant instructions, which will not be repeated here.

[0045] In some embodiments, in order to reduce system energy consumption, the frequency of the image output by the imaging module can be controlled. Specifically, the sampling frequency of the image sensor can be adjusted to reduce the system energy consumption as much as possible under the premise of meeting the image requirements, and avoid heat accumulation at the front end of the imaging module. In some embodiments, the frequency of the image output by the imaging module can be called the image output frequency, which can be specifically understood as the number of images (such as the number of image frames) output by the imaging module within a unit time (such as one minute, one second). In some embodiments, the image output frequency of the imaging module can be determined by the sampling frequency of the image sensor inside it, so the image output frequency of the imaging module can be adjusted by adjusting the sampling frequency of the image sensor. In some embodiments, the control unit can adjust the image output frequency of at least one of the first imaging module and the second imaging module based on the clarity of the first image and the second image. Specifically, the control unit can obtain the clarity indication value of the first image and the second image; control the image output frequency of the imaging module corresponding to the image with a higher clarity indication value in the first image and the second image to be higher than the image output frequency of the imaging module corresponding to the image with a lower clarity indication value. As an example, the control unit can directly process the first image and the second image to obtain the clarity indication values ​​of the two, or the control unit can obtain the clarity indication values ​​of the first image and the second image from the image processing unit. When the clarity indicator value of the first image is higher than that of the second image, the control unit can increase the image output frequency of the first imaging module (such as adjusting the original 3 frames / s to 5 frames / s), or the control unit can reduce the image output frequency of the second imaging module (such as adjusting the original 3 frames / s to 1 frame / s). In some embodiments, the control unit can obtain the clarity indicator values ​​of the first image and the second image respectively according to a certain period, and determine the image with a higher clarity index value. The aforementioned period can be 20s, 30s, 1min, 2min, etc. Further, the control unit can dynamically adjust the image output frequency of the first imaging module or the second imaging module according to the comparison result of the clarity indicator value. Continuing with the previous example, when the clarity indicator value of the second image is higher than that of the first image in the next clarity indicator value comparison, the control unit can increase the image output frequency of the second imaging module (such as adjusting the original 1 frame / s to 5 frames / s), or the control unit can reduce the image output frequency of the first imaging module (such as adjusting the original 5 frames / s to 1 frame / s).

[0046] In some embodiments, the control unit can also adjust the image output frequency of the first imaging module or the second imaging module based on the difference in clarity indication values ​​between the first image and the second image. Specifically, the control unit can obtain the clarity indication values ​​of the first image and the second image; when the difference in clarity indication values ​​between the first image and the second image exceeds a second threshold, the image output frequency of the imaging module corresponding to the image with a higher clarity indication value in the first image and the second image is controlled to be higher than the image output frequency of the imaging module corresponding to the image with a lower clarity indication value; when the difference in clarity indication values ​​between the first image and the second image does not exceed the second threshold, the image output frequencies of the first imaging module and the second imaging module are made the same. In some embodiments, the difference between the clarity indication values ​​of the first image and the second image can be calculated to characterize the difference between the two. The second threshold can be determined based on the energy consumption or heat dissipation of the imaging module, the image quality requirements, and the value range of the clarity indication value. The second threshold can be set to be the same as or different from the first threshold.

[0047] In order to adapt to the endoscope system with dual imaging modules provided in some embodiments of this specification, some other embodiments of this specification also provide an image processing method. Figure 4 This is an exemplary flow chart of the image processing method shown in some embodiments of this specification. Figure 4 The process 400 shown may be executed by a processing device (such as the processing device 120), and further, may be implemented by an image processing unit or an image processing system 800 in the processing device. Figure 4 As shown, the image processing method shown in some embodiments of this specification may include the following steps.

[0048] Step 410 , acquiring a first image and a second image. In some embodiments, step 410 may be implemented by an acquisition module 810 .

[0049] In some embodiments, the image processing unit can receive the first image and the second image from the first imaging module and the second imaging module. In some embodiments, the imaging module can output images at a preset frequency, and accordingly, the image processing unit can acquire the first image and the second image output by the first imaging module and the second imaging module frame by frame.

[0050] Thereafter, the process 400 may optionally proceed to step 420 or step 430. For example, step 420 or step 430 may be executed based on the first instruction. For another example, the clarity indication value of the first image and the second image may be obtained; when the difference between the clarity indication values ​​of the first image and the second image exceeds a first threshold, step 420 is executed; when the difference between the clarity indication values ​​of the first image and the second image does not exceed the first threshold, step 430 is executed. More information about the clarity indication value and the first threshold can be found in the previous text and will not be repeated here.

[0051] In some embodiments, image indicators such as contrast, sharpness or resolution of the first image and the second image may be calculated respectively, and clarity indication values ​​of the first image and the second image may be obtained respectively based on a combination of one or more of them. Figure 5 500 is an exemplary flow chart of obtaining image clarity shown in other embodiments of this specification. It should be understood that the image described in process 5 can be the first image or the second image. In some embodiments, process 500 can be implemented by the image processing unit or processing module 820. Figure 5 As shown, process 500 may include the following steps.

[0052] Step 510: divide the image into N image sub-regions.

[0053] In some embodiments, N may be an integer greater than 1, for example, N may be 9, 12, 16 or a greater value. As an example, the image may be divided into 3×3 squares to obtain 9 image sub-regions, or the image may be divided into 3×5 squares to obtain 15 image sub-regions. In some embodiments, the first image may be divided into N1 image sub-regions, and the second image may be divided into N2 image sub-regions, where N1 and N2 may be the same or different. In some embodiments, the sizes of the image sub-regions in the same image may be the same or different. The shape of the image sub-region may be a rectangle, or other polygons such as a triangle, a pentagon, or a hexagon.

[0054] Step 520: Calculate the clarity indicator value of each image sub-region in the image.

[0055] In some embodiments, the contrast, resolution, sharpness, etc. of each image sub-region in the first image may be calculated respectively, and the clarity indication value may be obtained based on a combination of one or more thereof. The contrast, resolution, sharpness, etc. of each image sub-region in the second image may be calculated respectively, and the clarity indication value may be obtained based on a combination of one or more thereof. As an example, the clarity indication value may be obtained based on the sum of the contrast and sharpness of a certain image sub-region, or the contrast of a certain image sub-region may be used as the clarity indication value.

[0056] Step 530: determine the influence weight of each image sub-region in the image.

[0057] In some embodiments, constant influence weights may be set for different image sub-regions of an image. For example, the influence weight of an image sub-region located at the center of the image may be set to a larger constant value, and the influence weight of an image sub-region located at the edge of the image may be set to a smaller constant value. In other embodiments, the influence weights of different image sub-regions in an image may be determined based on the photographed object.

[0058] Specifically, the image processing unit can identify the first image or the second image to determine the current photographed object. In some embodiments, the image processing unit can input the first image or the second image into a trained image recognition model to determine the current photographed object. Since the first image and the second image can image the same photographed object, the photographed object can be determined by only identifying one of the first image and the second image. The image recognition model can be a neural network machine learning model, specifically a convolutional neural network model, etc. The image recognition model can be trained using sample images and corresponding labels so that it can recognize the photographed object in the image. The label is used to indicate the type or name of the photographed object in the corresponding image. For example, the label can be the name of an organ or tissue of a living organism such as "stomach" or "intestine".

[0059] In some embodiments, the influence weights of each image sub-region in the corresponding image can be determined in advance based on the physiological structure characteristics of the photographed object and the needs of the doctor's diagnosis, and then the corresponding relationship between the photographed object and the influence weight can be established. For example, when the photographed object is the stomach, due to the large space and flat tissue of the stomach, the doctor mainly observes the image in the center of the image to make a diagnosis and treatment judgment. At this time, the weight value of the image sub-region close to the center of the image can be set to be relatively high, and the weight value of the image sub-region close to the periphery of the image can be set to be relatively low. For another example, when the photographed object is the intestine, the intestine is a tubular structure, and the intestinal wall tissue is located at the periphery of the image. At this time, the weight values ​​of all image sub-regions can be set to be the same, or the weight value of the image sub-region close to the center of the image can be relatively low, and the weight value of the image sub-region close to the periphery of the image can be relatively high. Taking the division of the image into 3×3 image sub-regions (nine-square grid) as an example, when the photographed object is the stomach, the influence weights of each image sub-region of the corresponding image are When the photographed object is the intestine, the influence weight of each image sub-region of the corresponding image is After determining the current photographed object, the image processing unit may determine the influence weight of each image sub-region in the first image and the influence weight of each image sub-region in the second image based on the correspondence between the photographed object and the influence weight.

[0060] Step 540: Determine the clarity of the image based on the clarity indication value and the influence weight of each image sub-region in the image.

[0061] Let’s assume that the clarity indicator values ​​of each image sub-region in the first image are t 11 ,t 12 ,…,t 1N1 , the influence weight of each image sub-region is s 11 、s 12 ,…,s 1N1, and then the clarity or clarity indicator value of the first image can be calculated as (s 11 ×t 11 +s 12 ×t 12 +…+s 1N1 ×t 1N1 ), assuming that the clarity indicator values ​​of each image sub-region in the second image are t 21 ,t 22 ,…,t 2N2 , the influence weight of each image sub-region is s 21 、s 22 ,…,s 2N2 , the clarity or clarity indicator value of the second image can be calculated as (s 21 ×t 21 +s 22 ×t 22 +…+s 2N2 ×t 2N2 ).

[0062] Step 420 , selecting and outputting at least one of the first image and the second image. In some embodiments, step 420 may be implemented by the processing module 820 .

[0063] In some embodiments, the first image or the second image may be output based on the second instruction, or the first image and the second image may be output simultaneously. More information about the second instruction can be found in the previous text and will not be repeated here. In some other embodiments, the one with higher image definition of the first image and the second image may be output. For information about how to obtain image definition, please refer to Figure 5 The relevant description will not be repeated here.

[0064] Step 430 , obtaining a composite image based on the first image and the second image and outputting the composite image. In some embodiments, step 430 may be implemented by the processing module 820 .

[0065] In some embodiments, the imaging ranges of the first image and the second image have an intersection. For example, the two may contain the same detected object, but the clear areas contained in the two are different. At this time, the clarity of the image sub-areas corresponding to the same shooting area on the two images can be compared, and the image sub-areas with higher clarity can be screened out as the target image sub-areas. Similarly, the target image sub-areas corresponding to different shooting areas on the first image or the second image can be obtained. Finally, these target image sub-areas can be spliced ​​to obtain a composite image with more clear areas.

[0066] Figure 6 This is an exemplary flow chart for determining a target image sub-region as shown in some embodiments of this specification. In some embodiments, Figure 6The process 600 shown can be implemented by the processing module 820. Figure 6 As shown, process 600 may include the following steps.

[0067] Step 610: register the first image and the second image.

[0068] In some embodiments, the sampling moments of the first image and the second image may be the same, or the interval time may not exceed a set time threshold. For example, the time threshold may be 0.1s, 0.5s, 1s or 2s, etc., so as to ensure that the photographed objects in the first image and the second image have the same state as much as possible. As mentioned above, in some embodiments, the optical lenses in the first imaging module and the second imaging module may be set to have a smaller size (such as a smaller diameter) or the distance between the main optical axes of the first imaging module and the second imaging module may be as small as possible, so as to ensure that the two imaging modules have the same imaging range as much as possible. In other words, the contents of the first image and the second image may have a high degree of overlap.

[0069] In some embodiments, in order to improve the quality of image synthesis, the first image and the second image may be registered first. Registration can be understood as aligning two or more images in space. In some embodiments, due to the differences in the positions and shooting angles of different imaging modules, the same photographed object in the first image and the second image may have different sizes or orientations. In order to align the first image and the second image in space, at least one of the images may be scaled, rotated, and translated. Figure 7 is a schematic diagram of the image registration principle shown in some embodiments of this specification, Figure 7 In the figure, image 710 is the first image, and image 720 is the second image. It can be seen that the same photographed object (taking a triangle as an example) has different orientations in the first image and the second image. In order to achieve image registration, feature points of the photographed object (such as feature points at the pylorus of the stomach, feature points at the cardia, etc.) can be identified in the first image and the second image respectively as positioning reference points, such as Figure 7The positioning reference points A1, B1 and C1 in the first image 710, and the positioning reference points A2, B2 and C2 in the second image 720. Further, coordinate systems can be established for the first image 710 and the second image 720, respectively, such as taking the upper left corner vertex of each image as the origin, horizontally to the right as the positive direction of the x-axis, and vertically downward as the positive direction of the y-axis, to establish the first image coordinate system and the second image coordinate system. The coordinate values ​​of the positioning reference points A1, B1 and C1 in the first image coordinate system are obtained, and the coordinate values ​​of the positioning reference points A2, B2 and C2 in the second image coordinate system are obtained. The pixel points in the first image can be spatially transformed (scaled, translated or rotated) so that the transformed coordinate values ​​of the positioning reference points A1, B1 and C1 are the same as the coordinate values ​​of the positioning reference points A2, B2 and C2. In this way, when the transformed first image 711 overlaps with the second image 720, the three positioning reference points can also coincide one by one, that is, the spatial alignment of the two images is achieved.

[0070] In other embodiments, registering the first image and the second image may further include reducing the following differences between the two images: brightness difference, color difference. In some embodiments, the brightness of the pixel value of the other image may be adjusted based on the image with lower brightness, so that the brightness difference between the two images is reduced. For another example, the hue or color saturation of the two images may be adjusted to reduce the color difference between the two images.

[0071] Step 620: Divide the registered first image and second image into M first image sub-regions and M second image sub-regions respectively.

[0072] M may be an integer greater than 1. In some embodiments, the M first image sub-regions correspond one-to-one to the M second image sub-regions. The aforementioned correspondence may be understood as the first image sub-region and the second image sub-region having the same image content or having a repetition degree greater than 80% or a higher threshold value, and further, the first image sub-region and the second image sub-region corresponding to each other may also have the same shape and size. In some embodiments, the first image and the second image after registration may be divided in the same manner, so that the M first image sub-regions obtained can correspond one-to-one to the M second image sub-regions.

[0073] Step 630: Calculate the clarity indication value of each first image sub-region and each second image sub-region respectively.

[0074] The method for calculating the image sub-region clarity indication value may be the same as the method for calculating the image clarity indication value. The specific calculation method may refer to the related description on calculating the image clarity indication value in the previous text, which will not be repeated here.

[0075] Step 640: Select one with a higher definition indicator value from the first image sub-region and the second image sub-region corresponding to each other as the target image sub-region, thereby obtaining M target image sub-regions.

[0076] In some embodiments, the first image sub-region and the second image sub-region corresponding to each other can be taken as a group, so as to obtain M groups of image sub-regions. For each group of image sub-regions, a sub-region with a higher definition indication value can be selected as a target image sub-region, thereby obtaining M target image sub-regions.

[0077] Finally, the M target image sub-regions are spliced ​​to obtain a composite image. Since the target image sub-regions with better clarity are selected in each group of image sub-regions, the obtained composite image can have more clear regions, that is, the overall clarity of the composite image will be better.

[0078] Figure 8 is an exemplary block diagram of an image processing system shown in some embodiments of this specification. Figure 8 As shown, the image processing system 800 may include an acquisition module 810 and a processing module 820 .

[0079] The acquisition module 810 is used to acquire a first image and a second image. The processing module 820 is used to select and output at least one of the first image and the second image, or to obtain and output a composite image based on the first image and the second image.

[0080] For more information about each module, see Figure 4 to Figure 7 The relevant description of will not be repeated here. It should be understood that Figure 8 The system and its modules shown can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented by hardware, software, or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or a dedicated design hardware. Those skilled in the art will understand that the above methods and systems can be implemented using computer executable instructions and / or control codes contained in a processor, such as a disk, CD or DVD-ROM, etc., and such codes are provided in the memory of a programmable device. The system and its modules of this specification can not only be implemented by hardware circuits such as ultra-large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., but can also be implemented by software such as executed by various types of processors, and can also be implemented by a combination of the above hardware circuits and software (e.g., firmware).

[0081] It should be noted that the above description of the system and its modules is only for convenience of description and does not limit this specification to the scope of the embodiments. It is understandable that for those skilled in the art, after understanding the principle of the system, it is possible to arbitrarily combine the modules without deviating from this principle to form a subsystem connected to other modules. Or some modules may be split to obtain more modules or multiple units under the module. Such variations are all within the scope of the disclosure of this specification.

[0082] One or more embodiments of the present specification further provide a computer program product, including computer instructions or computer codes, which can implement the above-mentioned image processing method when at least part of the computer instructions or computer codes are executed by a processor.

[0083] One or more embodiments of the present specification further provide a computer-readable storage medium storing computer instructions or computer codes, which can implement the above-mentioned image processing method when at least part of the computer instructions or computer codes are executed by a processor.

[0084] In some embodiments, the aforementioned processor may be a combination of one or more of the following processors: a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a programmable logic controller (PLC), a reduced instruction set computer (RISC), and a microprocessor.

[0085] In some embodiments, the aforementioned storage medium may include one or more of the following combinations: mass storage, removable storage, volatile read-write memory, read-only memory (ROM). Exemplary mass storage may include magnetic disks, optical disks, solid-state hard disks, etc. Exemplary removable storage may include flash disks, floppy disks, optical disks, memory cards, compressed hard disks, magnetic tapes, etc. Exemplary volatile read-write memory may include random access memory (RAM). Exemplary random access memory may include dynamic random access memory (DRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), static random access memory (SRAM), thyristor random access memory (T-RAM) and zero capacitance memory (Z-RAM), etc. Exemplary read-only memory may include masked read-only memory (MROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) and digital multifunction hard disk read-only memory, etc.

[0086] The beneficial effects that may be brought about by the embodiments of this specification include but are not limited to: (1) the use of dual imaging modules to obtain endoscopic images can effectively expand the focus range or depth of field of the endoscopic system; (2) the two imaging modules have their own optical lenses, and the focal position of the optical lenses can be flexibly selected, so that the focus ranges of the two imaging modules are greatly different and adjacent to each other, thereby greatly expanding the overall focus range of the endoscopic system; (3) the use of injection-molded optical lenses can make the diameter of the insertion part of the endoscopic system smaller even if dual imaging modules are used; (4) the endoscopic system can operate in image selection mode and image synthesis mode to meet the different imaging needs of users; (5) the first image, second image or synthesized image with better clarity is automatically displayed, thereby improving the overall imaging clarity of the endoscopic system; (6) the image output frequency of the first imaging module and the second imaging module is adjusted based on the clarity, while ensuring the imaging quality, reducing the system energy consumption, and effectively avoiding the heat accumulation at the tip of the endoscope, greatly reducing the impact on the inspected object, etc. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other beneficial effects that may be obtained.

[0087] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are taught in this specification, so such modifications, improvements and corrections still belong to the concept and scope of the exemplary embodiments of this specification.

Claims

1. An endoscope system, characterized in that: The device comprises a light source, an insertion part and an image processing part, wherein the insertion part at least has a tip end configured to be inserted into the inspected area, and the tip end is provided with a first imaging module and a second imaging module; The light source is used to provide illumination for the inspected area; The first imaging module is used to acquire a first image; The first imaging module has a first focusing range and a first imaging range; The second imaging module is used to acquire a second image; The second imaging module has a second focusing range and a second imaging range; the second focusing range and the first focusing range are at least partially different, and the second imaging range and the first imaging range have an intersection; The image processing unit is used to select and output at least one of the first image and the second image, or to obtain and output a composite image based on the first image and the second image.

2. The endoscope system according to claim 1, characterized in that: It has an image selection mode and an image synthesis mode; the image processing unit is further used to work in the image selection mode based on a first instruction to select at least one of the first image and the second image and output it, or work in the image synthesis mode to obtain a synthesized image based on the first image and the second image and output it.

3. The endoscope system according to claim 1, characterized in that: It has an image selection mode and an image synthesis mode; the image processing unit is further used for: Acquire clarity indication values ​​of the first image and the second image; When the difference between the clarity indication values ​​of the first image and the second image exceeds a first threshold, operating in the image selection mode to select and output at least one of the first image and the second image; When the difference between the clarity indication values ​​of the first image and the second image does not exceed the first threshold, the image synthesis mode is operated to obtain a synthesized image based on the first image and the second image and output it.

4. The endoscope system according to claim 2 or 3, characterized in that: When operating in the image selection mode, the image processing unit is further configured to output the first image, or output the second image, or output the first image and the second image simultaneously based on a second instruction.

5. The endoscope system according to claim 2 or 3, characterized in that: When operating in the image selection mode, the image processing unit is further configured to output the one with higher image definition between the first image and the second image.

6. The endoscope system according to claim 5, characterized in that: In order to output the one with higher image definition between the first image and the second image, the image processing unit is further used to: Dividing the first image into N1 image sub-regions, where N1 is an integer greater than 1; calculating a clarity indication value of each image sub-region in the first image; determining an influence weight of each image sub-region in the first image; Determining the clarity of the first image based on the clarity indication value and the influence weight of each image sub-region in the first image; Dividing the second image into N2 image sub-regions, where N2 is an integer greater than 1; calculating a clarity indication value of each image sub-region in the second image; determining an influence weight of each image sub-region in the second image; determining the definition of the second image based on the definition indication value and the influence weight of each image sub-region in the second image; comparing the clarity of the first image and the second image, and outputting the one with higher image clarity between the first image and the second image; Among them, N1 and N2 are equal or unequal.

7. The endoscope system according to claim 6, characterized in that: The image processing unit is further used to determine the influence weight of each image sub-region in the first image or the second image; In order to determine the influence weight of each image sub-region in the first image or the second image, the image processing unit is further used to: Identify the first image or the second image to determine a current photographed object; Based on the correspondence between the photographed object and the influence weight, the influence weight of each image sub-region in the first image or the second image is determined.

8. The endoscope system according to claim 2 or 3, characterized in that: When working in the image synthesis mode, the image processing unit is further used for: Determine one or more target image sub-regions from the first image and the second image respectively based on the clarity; The one or more target image sub-regions are spliced ​​together to obtain the composite image.

9. The endoscope system according to claim 8, characterized in that: In order to determine one or more target image sub-regions from the first image and the second image respectively based on the clarity, the image processing unit is further used to: registering the first image with the second image; Respectively dividing the registered first image and the second image into M first image sub-regions and M second image sub-regions, where M is an integer greater than 1, and the M first image sub-regions correspond one-to-one to the M second image sub-regions; respectively calculating the clarity indication value of each first image sub-region and each second image sub-region; From the first image sub-region and the second image sub-region corresponding to each other, the one with a higher definition indication value is selected as the target image sub-region, thereby obtaining M target image sub-regions.

10. The endoscope system according to claim 9, characterized in that: In order to register the first image with the second image, the image processing unit is further used for: Perform one or more of the following operations on at least one of the first image and the second image: scaling, translating, and rotating, so that two or more positioning reference points in the first image and two or more positioning reference points in the second image are aligned one by one; Alternatively, the following differences between the first image and the second image are reduced: brightness difference, color difference.

11. The endoscope system according to claim 1, characterized in that: It also includes a control unit; The control unit is used to adjust the image output frequency of at least one of the first imaging module and the second imaging module based on the clarity of the first image and the second image.

12. The endoscope system according to claim 11, characterized in that: The control unit is further used for: Acquire clarity indication values ​​of the first image and the second image; The image output frequency of the imaging module corresponding to the image with a higher clarity indication value among the first image and the second image is controlled to be higher than the image output frequency of the imaging module corresponding to the image with a lower clarity indication value.

13. The endoscope system according to claim 11, characterized in that: The control unit is further used for: Acquire clarity indication values ​​of the first image and the second image; When the difference between the clarity indication values ​​of the first image and the second image exceeds a second threshold, controlling the image output frequency of the imaging module corresponding to the image with a higher clarity indication value among the first image and the second image to be higher than the image output frequency of the imaging module corresponding to the image with a lower clarity indication value; When the difference in clarity indication values ​​between the first image and the second image does not exceed a second threshold, the image output frequencies of the first imaging module and the second imaging module are made the same.

14. The endoscope system according to claim 1, characterized in that: The first imaging module includes a first optical lens and a first image sensor, the first optical lens is used to obtain a first optical image, and the first image sensor is used to receive the first optical image and convert it into the first image; The second imaging module includes a second optical lens and a second image sensor, the second optical lens is used to obtain a second optical image, and the second image sensor is used to receive the second optical image and convert it into the second image; The main optical axis of the first optical lens is parallel to the main optical axis of the second optical lens.

15. The endoscope system according to claim 14, characterized in that: The first optical lens and the second optical lens are fixed-focus lenses; The focus of the first optical lens is located in an area of ​​7 mm to 15 mm on its front end surface, and the focus of the second optical lens is located in an area of ​​2 mm to 6 mm on its front end surface.

16. The endoscope system according to claim 14, characterized in that: A distance between a principal optical axis of the first optical lens and a principal optical axis of the second optical lens is smaller than a radius of the tip portion.

17. An endoscope insertion portion, characterized in that: At least having a tip portion configured to probe into the inspected area, the tip portion being provided with a first imaging module and a second imaging module; The first imaging module is used to acquire a first image; The first imaging module has a first focusing range and a first imaging range; The second imaging module is used to acquire a second image; The second imaging module has a second focusing range and a second imaging range; the second focusing range is at least partially different from the first focusing range, and the second imaging range and the first imaging range have an intersection.

18. An image processing method for an endoscope system, characterized in that: include: Acquire a first image and a second image, wherein the first image and the second image have different focus areas, and an imaging range corresponding to the first image and an imaging range corresponding to the second image have an intersection; At least one of the first image and the second image is selected and outputted, or a composite image is obtained based on the first image and the second image and outputted.

19. The image processing method according to claim 18, characterized in that: The selecting and outputting at least one of the first image and the second image, or obtaining and outputting a composite image based on the first image and the second image, comprises: Acquire clarity indication values ​​of the first image and the second image; When the difference between the clarity indication values ​​of the first image and the second image exceeds a first threshold, selecting at least one of the first image and the second image and outputting the image; When the difference between the clarity indication values ​​of the first image and the second image does not exceed the first threshold, a composite image is obtained based on the first image and the second image and outputted.

20. The image processing method according to claim 18, characterized in that: The selecting and outputting at least one of the first image and the second image comprises: The one of the first image and the second image with higher image definition is output.

21. The image processing method according to claim 20, characterized in that: The outputting the one with higher image definition between the first image and the second image comprises: Dividing the first image into N1 image sub-regions, where N1 is an integer greater than 1; calculating a definition indication value of each image sub-region in the first image; determining an influence weight of each image sub-region in the first image; and determining the definition of the first image based on the definition indication value and the influence weight of each image sub-region in the first image; Divide the second image into N2 image sub-regions, where N2 is an integer greater than 1; calculate the clarity indication value of each image sub-region in the second image; determine the influence weight of each image sub-region in the second image; and determine the clarity of the second image based on the clarity indication value and the influence weight of each image sub-region in the second image; comparing the clarity of the first image and the second image, and outputting the one with higher image clarity between the first image and the second image; Among them, N1 and N2 are equal or unequal.

22. The image processing method according to claim 21, characterized in that: Also includes: determining an influence weight of each image sub-region in the first image or the second image; The determining of the influence weight of each image sub-region in the first image or the second image further comprises: Identify the first image or the second image to determine a current photographed object; Based on the correspondence between the photographed object and the influence weight, the influence weight of each image sub-region in the first image or the second image is determined.

23. The image processing method according to claim 18, characterized in that: The step of obtaining and outputting a composite image based on the first image and the second image includes: Determine one or more target image sub-regions from the first image and the second image respectively based on the clarity; The one or more target image sub-regions are spliced ​​together to obtain the composite image.

24. The image processing method according to claim 23, characterized in that: The determining one or more target image sub-regions from the first image and the second image respectively based on the clarity comprises: registering the first image with the second image; Respectively dividing the registered first image and the second image into M first image sub-regions and M second image sub-regions, where M is an integer greater than 1, and the M first image sub-regions correspond one-to-one to the M second image sub-regions; respectively calculating the clarity indication value of each first image sub-region and each second image sub-region; From the first image sub-region and the second image sub-region corresponding to each other, the one with a higher definition indication value is selected as the target image sub-region, thereby obtaining M target image sub-regions.

25. The endoscope system according to claim 24, characterized in that The registering the first image with the second image comprises: Perform one or more of the following operations on at least one of the first image and the second image: scaling, translating, and rotating, so that two or more positioning reference points in the first image and two or more positioning reference points in the second image are aligned one by one; Alternatively, the following differences between the first image and the second image are reduced: brightness difference, color difference.

26. A computer-readable storage medium storing computer codes or instructions, which can implement the method according to any one of claims 18 to 25 when at least part of the computer codes or instructions are executed by a processor.

Images