Endoscope system, image processing method, and computer program product therefor

Abstract

The present disclosure relates to the technical field of medical imaging, and relates to an endoscope system, an image processing method, and a computer program product therefor. The endoscope system to which the present disclosure relates comprises a light source, an insertion portion, and an image processing portion. The insertion portion is at least provided with a distal end portion configured to reach into a region to be inspected, and the distal end portion is provided with a first imaging module and a second imaging module. The light source is used for providing illumination to the region to be inspected. The first imaging module is used for acquiring a first image, and the first imaging module has a first focusing range and a first imaging range; the second imaging module is used for acquiring a second image, and the second imaging module has a second focusing range and a second imaging range; and the second focusing range is at least partially different from the first focusing range, and the second imaging range and the first imaging range have an intersection. The image processing portion is used for selecting and outputting at least one of the first image and the second image, or obtaining a composite image on the basis of the first image and the second image and outputting the composite image.

Description

An endoscope system, an image processing method, and a computer program product thereof. Technical Field

[0001] This disclosure relates to the field of medical imaging technology, specifically to an endoscope system, an image processing method, and a computer program product thereof. Background Technology

[0002] Endoscopic technology, as an imaging technique, is frequently used in the field of medical diagnosis. Endoscopic technology allows imaging components to enter the interior of a living organism (such as a human or animal) through natural openings or small surgical incisions, thereby acquiring images of the examined area and providing more intuitive and accurate diagnostic information.

[0003] Currently, endoscopic technology has made significant progress and continues to advance. Obtaining clearer images is one of the development directions of endoscopic technology. Some embodiments in this specification aim to provide an endoscopic system and a corresponding image processing method to improve the clarity of endoscopic images. Summary of the Invention

[0004] According to a first aspect, one or more embodiments of this specification provide an endoscope system including a light source, an insertion section, and an image processing section. The insertion section has at least a tip configured to be inserted into an area to be examined. The tip is provided with a first imaging module and a second imaging module. The light source is used to provide illumination to the area to be examined. The first imaging module is used to acquire a first image. The first imaging module has a first focus range and a first imaging range. The second imaging module is used to acquire a second image. The second imaging module has a second focus range and a second imaging range. The second focus range and the first focus range are at least partially different, and the second imaging range and the first imaging range intersect. The image processing section is used to select at least one of the first image and the second image and output it, or to obtain a composite image based on the first image and the second image and output it.

[0005] According to a second aspect, one or more embodiments of this specification provide an endoscope insertion portion having at least a tip configured to be inserted into an area to be examined, the tip being provided with a light source, a first imaging module, and a second imaging module; the light source is used to provide illumination to the area to be examined; the first imaging module is used to acquire a first image; the first imaging module has a first focus range and a first imaging range; the second imaging module is used to acquire a second image; the second imaging module has a second focus range and a second imaging range; the second focus range and the first focus range are at least partially different, and the second imaging range and the first imaging range intersect.

[0006] According to a third aspect, one or more embodiments of this specification provide an image processing method for an endoscope system, comprising: acquiring a first image and a second image, the first image and the second image having different focal regions, and the imaging range corresponding to the first image and the imaging range corresponding to the second image having an intersection; selecting at least one of the first image and the second image and outputting it, or obtaining a synthesized image based on the first image and the second image and outputting it.

[0007] According to a fourth aspect, one or more embodiments of this specification provide a computer-readable storage medium storing computer code or instructions that, when at least a portion of the computer code or instructions is executed by a processor, enables the implementation of the image processing methods provided in some embodiments of this specification.

[0008] According to a fifth aspect, one or more embodiments of this specification provide a computer program product, including computer code or instructions, which, when executed by a processor, can implement the image processing method provided in some embodiments of this specification. Attached Figure Description

[0009] Figure 1 is a schematic diagram of the structure of an endoscope system shown in some embodiments;

[0010] Figure 2 is a schematic diagram of the insertion section in an endoscope system shown in some embodiments of this specification;

[0011] Figure 3 is a schematic diagram of the working mode of the endoscope system shown in some embodiments of this specification;

[0012] Figure 4 is an exemplary flowchart of an image processing method shown in some embodiments of this specification;

[0013] Figure 5 is an exemplary flowchart illustrating the acquisition of image sharpness in some embodiments of this specification;

[0014] Figure 6 is an exemplary flowchart of determining a target image sub-region as shown in some embodiments of this specification;

[0015] Figure 7 is a schematic diagram of the image registration principle shown in some embodiments of this specification;

[0016] Figure 8 is an exemplary block diagram of an image processing system shown in some embodiments of this specification.

[0017] Components labeled in the figure: 100 Endoscopic system; 110 Endoscope; 111 Insertion section; 111a Tip section; 112 Operation section; 120 Processing device; 130 Display device; 140 Light source; 21 First imaging module; 211 First optical lens; 212 First image sensor; 22 Second imaging module; 221 Second optical lens; 222 Second image sensor; 31 Optical fiber. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0019] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0020] The serial numbers assigned to components in this document, such as "first" and "second," are used only to distinguish the described objects and have no sequential or technical meaning. The terms "connection" and "linkage" used in this application, unless otherwise specified, include both direct and indirect connections (linkages).

[0021] Endoscopic technology, as an imaging technique, is frequently used in the field of medical diagnosis. Endoscopic technology allows imaging components to enter the interior of a living organism (such as a human or animal) through natural openings or small surgical incisions, thereby acquiring images of the examined area and providing more intuitive and accurate diagnostic information.

[0022] Figure 1 is a schematic diagram of the structure of an endoscope system according to some embodiments. As shown in Figure 1, the endoscope system 100 may include an endoscope 110 and a processing device 120. The endoscope 110 is used to acquire image data of the area to be examined, and the processing device 120 is used to process the image data to obtain medical images required for diagnosis. The processing device 120 can also control the endoscope 110 or some of its components to better meet the needs of the image data. In some embodiments, the endoscope system 100 may also include a display device 130, which can acquire the image output by the processing device 120 and display it to a user (e.g., a doctor). In other embodiments, the endoscope system 100 may not include a display device 130. When it is necessary to view the image, the endoscope system 100 can be connected to an external display device 130 to output the processed image for display.

[0023] In some embodiments, the endoscope 110 may further include an insertion portion 111 and an operating portion 112. At least a portion of the insertion portion 111 may be bent for inserting the imaging module into the area to be examined within the biological body. In some embodiments, the insertion portion 111 may be a flexible tubular assembly of a certain length, one end of which is connected to the operating portion 112, and the other end of which can be inserted into the area to be examined within the biological body. The operating portion 112 may be held and operated by a user to adjust the depth or direction of the insertion portion 111, thereby adjusting the observation area and viewing angle of the endoscope.

[0024] In some embodiments, a section of the insertion part 111 that can penetrate into the area to be examined within a biological body can be referred to as the front end or tip 111a (it should be understood that the dashed circle in FIG1 is a marker used to indicate the position of the tip and should not be construed as a physical component in the endoscope system 100). An imaging module can be disposed at the tip 111a of the insertion part to acquire image data of the area to be examined. In some alternative embodiments, the tip 111a of the insertion part can also be provided with a light source to provide illumination to the area to be examined, thereby enabling the imaging module to acquire an image with better clarity. Referring to FIG2, in some alternative embodiments, the insertion part 111 has an optical fiber 31 extending in the same direction as the insertion part 111. One end face of the optical fiber 31 protrudes from the end face of the tip 111a. A light source 140 is disposed at the end of the optical fiber away from the tip 111a. The optical fiber 31 can guide the light beam emitted by the light source 140 to the area to be examined, providing illumination.

[0025] In some embodiments, the imaging module may include an optical lens and an image sensor. The optical lens may consist of one or more optical elements (such as lenses) used to capture light from the area being detected to form an optical image. The image sensor is used to receive the optical image and generate a computer-recognizable medical image. Specifically, the image sensor may be a CCD image sensor, a CMOS image sensor, etc.

[0026] In some embodiments, the optical lens of the endoscope imaging module can be a fixed-focus lens. A fixed-focus lens has a fixed focal point, meaning its focal point cannot be adjusted. Fixed-focus lenses are smaller than zoom lenses, making them suitable for endoscopes with size constraints, but their limitation is a limited focusing range. In some embodiments, the focusing range can be understood as the spatial range within which a fixed-focus lens can clearly image the image, and it is related to the lens's focal point. Specifically, the focusing range can be characterized by the distance from the principal optical axis of the fixed-focus lens to its end face. As an example, if the focal point of a fixed-focus lens is located 5mm from the lens's outer end face (or front face or outer surface, i.e., the end face or surface closer to the object) along the principal optical axis, this fixed-focus lens can clearly image objects or features within a certain spatial range (e.g., 2mm) from its focal point; this spatial range can be called the focusing range. For intuitive understanding, the focusing range of a fixed-focus lens can also be described as the interval between 1mm and 7mm from the lens's outer end face along the principal optical axis. In some embodiments, the focusing range of the imaging module is limited by the focus or focusing range of its optical lens. Therefore, the focus or focusing range of the optical lens of the imaging module can also be defined as the focus or focusing range of the imaging module.

[0027] It is easy to understand that a fixed-focus lens limits the spatial or distance range at which an endoscopic system can clearly image. In practical applications, to obtain a clear image over a larger area of ​​the examined region, the doctor needs to frequently manipulate the lens to allow the tip of the insertion device to reach more areas, increasing operational complexity or examination time. Therefore, some embodiments of this specification provide an endoscopic system with dual imaging modules. These two modules have different focal positions and similar or even identical imaging ranges, thereby expanding the focusing space of the endoscopic system and resulting in images with better clarity.

[0028] Figure 2 is a schematic diagram of the insertion part of an endoscope system according to some embodiments of this specification. Referring to Figures 1 and 2, the endoscope system provided in some embodiments of this specification may include a light source 140, an insertion part 111, and an image processing unit. The insertion part has at least a tip 111a configured to penetrate into an area to be examined. The tip 111a is provided with a first imaging module 21 and a second imaging module 22. The light source 140 is used to provide illumination to the area to be examined. The first imaging module 21 is used to acquire a first image and has a first focus range and a first imaging range. The second imaging module 22 is used to acquire a second image and has a second focus range and a second imaging range. The second focus range and the first focus range are at least partially different, and the second imaging range and the first imaging range intersect. The image processing unit is used to select at least one of the first image and the second image and output it, or to obtain a composite image based on the first image and the second image and output it.

[0029] In some embodiments, the first imaging module 21 may further include a first optical lens 211 and a first image sensor 212. The first optical lens 211 is used to acquire a first optical image, and the first image sensor 212 is used to receive the first optical image and convert it into a first image. The second imaging module 22 may further include a second optical lens 221 and a second image sensor 222. The second optical lens 221 is used to acquire a second optical image, and the second image sensor 222 is used to receive the second optical image and convert it into a second image. Further descriptions of the optical lenses and image sensors can be found in other parts of the specification and will not be repeated here.

[0030] In some embodiments, the first optical lens and the second optical lens may have different focal points, thereby making the first focus range and the second focus range at least partially different. As an example, the focal point of the first optical lens may be located within a region of 7mm to 15mm from its front end face. Specifically, this can be understood as the focal point of the first optical lens being located within a segment on the principal optical axis from the outer end face of the lens of 7mm to 15mm, further, within a segment of 8mm to 12mm, or within a segment of 9mm to 11mm. For example, the focal point of the first optical lens may be located at a position on the principal optical axis at a distance of 7.5mm, 10mm, 11.5mm, or 12mm from the outer end face of the lens. In some embodiments, the focal point of the second optical lens may be located within a region of 2mm to 6mm from its front end face. Specifically, the focal point of the second optical lens can be understood as being located within a range of 2mm to 6mm from the principal optical axis to the outer end face of the lens, or further, within a range of 3mm to 5mm. For example, the focal point of the second optical lens can be located at a distance of 2.5mm, 3.5mm, 4mm, or 5mm from the principal optical axis to the outer end face of the lens.

[0031] The focusing range of the imaging module is affected by the focal point of the optical lens. Since the focal points of the first and second optical lenses are different, the first focusing range and the second focusing range are at least partially different. "At least partially different" can be understood as the first and second focusing ranges being different but overlapping, or the first and second focusing ranges not overlapping. For example, if the focal point of the first optical lens is 10mm from its outer end face, the first focusing range may include a range of 5mm to 50mm from the outer end face along the lens's principal optical axis. If the focal point of the second optical lens is 5mm from its outer end face, the second focusing range may include a range of 1mm to 7mm from the outer end face along the lens's principal optical axis. The first and second imaging modules having different focusing ranges allows the overall focusing range of the endoscope system to be expanded. Continuing the previous example, an endoscope system with dual imaging modules can obtain clear images of objects within a range of 1 to 50mm from the outer end face of the lens, which is a larger focusing range compared to a single imaging module. By employing a dual imaging module, the focal positions of the two optical lenses can be flexibly selected as needed, resulting in two focal ranges that are significantly different yet adjacent to each other (e.g., the first focal range corresponds to the interval of 7mm to 100mm, and the second focal range corresponds to the interval of 1mm to 7mm), thus greatly expanding the overall focal range of the endoscope system (e.g., the overall focal range corresponds to the interval of 1mm to 100mm).

[0032] In some embodiments, the imaging range can be understood as the field of view (FOV) of the imaging module or the spatial area it captures. The imaging range can be intuitively viewed as the spatial area enclosed by a frustum with the lens end face of the imaging module as the top base and the principal optical axis of the lens as the central axis. Therefore, the imaging range can be considered to be related to the position of the imaging module in space, the direction of the principal optical axis of the lens, and the field of view angle (the angle between the generatrix of the frustum and the central axis).

[0033] Referring again to Figure 2, the first imaging module 21 and the second imaging module 22 can be placed parallel to each other at the tip end 111a of the insertion portion, thereby making the principal optical axis of the first optical lens 211 parallel to the principal optical axis of the second optical lens 221. This arrangement allows the imaging ranges of the two imaging modules to overlap, for example, enabling imaging of the same object being examined (such as human tissue, organs, etc.). In some embodiments, the front end face of the first optical lens 211 can be further made coplanar or flush with the front end face of the second optical lens 221. In some embodiments, the optical lenses of the imaging modules can be made of glass or injection-molded lenses. Injection-molded lenses are lenses manufactured using injection molding technology, and the main materials include PC (polycarbonate), PMMA (polymethyl methacrylate), etc. Injection-molded lenses have the characteristics of high molding precision, light weight, good optical performance, and low cost. Injection-molded lenses can have a smaller volume size than glass lenses, making it easier to place two optical lenses in parallel or side-by-side in the endoscope insertion portion where the diameter is relatively limited. In some embodiments, the two imaging modules can be brought close to or adjacent to each other, or the size of the optical lens in the imaging module can be further reduced so that the distance between the main optical axes of the two imaging modules is less than the radius of the tip. Such a setting can further improve the closeness of the imaging range of the two imaging modules (e.g., the first image and the second image have an overlap area of ​​80% or 90% or more), making it more suitable for image synthesis of the first image and the second image.

[0034] The endoscope system provided in some embodiments of this specification employs two imaging modules (i.e., using two optical lenses and two image sensors for imaging), so that each optical lens corresponds to one image sensor, reducing the assembly difficulty between the optical lenses and image sensors. Specifically, to obtain a clear image, the imaging surface of the optical lens and the photosensitive surface of the image sensor need to be aligned. This one-to-one correspondence between optical lenses and image sensors significantly reduces the assembly and adjustment difficulty of the imaging surfaces of the optical lenses and the photosensitive surfaces of the image sensors, lowers costs, and facilitates mass production. Furthermore, this one-to-one correspondence between optical lenses and image sensors ensures that more light passing through the optical lenses enters the corresponding image sensors, resulting in brighter images and better observation.

[0035] The image processing unit may be located in a processing device (processing device 120 as shown in FIG1). Specifically, the image processing unit may be a control unit or a processor for performing image processing. In some embodiments, the image processing unit is used only for image selection; specifically, it may select at least one from a first image and a second image and output it. In some embodiments, the image processing unit may be used only for image synthesis; specifically, it may obtain and output a synthesized image based on the first image and the second image. In still other embodiments, the image processing unit may be used for both image selection and image synthesis. In this case, the endoscope system can be considered to have two working modes: an image selection mode and an image synthesis mode. FIG3 is a schematic diagram of the working modes of an endoscope system shown in some embodiments of this specification. The image processing unit shown in FIG3 can operate in either the image selection mode or the image synthesis mode as needed.

[0036] In some embodiments, the image processing unit may operate in an image selection mode based on a first instruction to select and output at least one of a first image and a second image, or operate in an image compositing mode to obtain and output a composite image based on the first image and the second image. The first instruction may be a mode selection instruction input by a user. For example, controls such as buttons, knobs, and selection switches may be provided on the operation unit (e.g., operation unit 112), allowing the user to input the first instruction through these controls to control the image processing unit to execute an image selection process (operating in image selection mode) or an image compositing process (operating in image compositing mode). In other alternative embodiments, the aforementioned controls may be provided on a processing device (e.g., processing device 120), allowing the user to input the first instruction through the controls on the processing device. In still other alternative embodiments, the processing device may control a display device (e.g., display device 130) to display a graphical user interface (GUI). The GUI may display controls, and the user may trigger the controls on the GUI to input the first instruction (e.g., by clicking the controls with a mouse or by touching the area corresponding to the controls on the touchscreen of display device 130). In some embodiments, the first instruction can be converted into an electrical signal and input to the control unit or processor of the processing device. When the first instruction is an electrical signal in a first state, it indicates that the user selects an image selection mode. When the first instruction is an electrical signal in a second state, it indicates that the user selects an image synthesis mode.

[0037] In other embodiments, the image processing unit can automatically select between an image selection mode and an image synthesis mode based on the image quality of the first image and the second image. Specifically, the image processing unit can acquire the sharpness indicator values ​​of the first image and the second image; when the difference between the sharpness indicator values ​​of the first image and the second image exceeds a first threshold, it operates in the image selection mode to select at least one of the first image and the second image and output it; when the difference between the sharpness indicator values ​​of the first image and the second image does not exceed the first threshold, it operates in the image synthesis mode to obtain a synthesized image based on the first image and the second image and output it.

[0038] In some embodiments, image quality can be sharpness, which refers to the clarity of textures, boundaries, etc., in an image. Generally, when the contrast between bright and dark areas in an image is large and the texture details are rich, the image can be considered relatively sharp. The image sharpness indicator value reflects the image sharpness; a higher sharpness indicator value means better image sharpness. Image sharpness can be related to indicators such as image resolution, sharpness, or contrast. For ease of measurement, in some embodiments, a combination of one or more of the image resolution, sharpness, and contrast indicators can be used as the image sharpness indicator value. In some embodiments, the sharpness indicator value can be normalized to a preset numerical range, such as the range 1 to 100, or the range 0 to 1, etc. In some embodiments, the difference between the sharpness indicator values ​​of the first image and the second image can be calculated, and the difference can be compared with a first threshold. The first threshold can be pre-configured according to the user's requirements for image quality and the value space of the sharpness indicator value. For example, the first threshold can be set to 15, 20, 30, 0.4, 0.5, etc. When the difference is greater than the first threshold, the first and second images are considered to have a significant difference in sharpness. In this case, selective output of the two images can be performed to meet the user's image sharpness requirements. When the difference is less than or equal to the first threshold, the two images are considered to have a small difference in sharpness. In this case, the two images can be combined, and the sharper sub-region is selected to obtain a composite image with better sharpness. Further explanation of the calculation method for image sharpness indicators and image synthesis can be found elsewhere in this manual and will not be repeated here.

[0039] In some embodiments, when operating in image selection mode, the image processing unit may further output a first image, a second image, or both images simultaneously based on a second instruction. The second instruction may be a user-inputted image selection instruction. For example, controls such as buttons, knobs, or selection switches may be provided on the operation unit (e.g., operation unit 112), allowing the user to input the second instruction to select between the first and second images. In other alternative embodiments, the aforementioned controls may be provided on the processing device (e.g., processing device 120), allowing the user to input the second instruction via the controls on the processing device. In still other alternative embodiments, the processing device may control a display device (e.g., display device 130) to display a graphical user interface (GUI), which may display controls that the user can trigger to input the second instruction. In some embodiments, the second instruction may be converted into an electrical signal and input to the control unit or processor of the processing device. When the second instruction is a third-state electrical signal, it instructs the user to select the first image; when the second instruction is a fourth-state electrical signal, it instructs the user to select the second image; and when the second instruction is a fifth-state electrical signal, it instructs the user to simultaneously output both the first and second images.

[0040] In some embodiments, an image region can be defined on the aforementioned graphical user interface, and depending on the user's selection, only the first image, only the second image, or both the first and second images can be displayed in the image region. For example, when selecting to output both the first and second images simultaneously, they can be displayed side-by-side. Imaging modules with different focus ranges may output images with different areas of sharpness. Displaying the first and second images simultaneously helps the user diagnose the object under examination by combining images of the same area with different areas of sharpness. It is understood that areas of sharpness in an image have higher clarity than other areas.

[0041] In some embodiments, the image sensor of the imaging module can output images at a preset frequency (e.g., 1 frame / s, 5 frames / s, etc.). These images can be displayed sequentially in the image area, or the first or second image can be a video image. In some embodiments, when the user selects the first image, or selects the second image, or selects both the first and second images simultaneously, it can be understood that the user has selected the first imaging module, or selected the second imaging module, or selected both imaging modules simultaneously. At this time, the imaging module corresponding to the image not selected by the user can stop working. Specifically, the control unit of the endoscope system can control the image sensor of that imaging module to stop image acquisition or image output. This reduces the energy consumption of the imaging module. On the other hand, the image sensor generates a certain amount of heat when working; this setting can also reduce heat accumulation at the tip of the endoscope insertion part, reducing the impact on the object being examined.

[0042] In some embodiments, when operating in image selection mode, the image processing unit can automatically output the image with higher sharpness between the first image and the second image. Specifically, the image processing unit can acquire the sharpness indicator values ​​of the first image and the second image respectively, and select the image with the higher sharpness indicator value for output. More details on calculating the image sharpness indicator value can be found elsewhere in the specification and will not be repeated here. In some embodiments, the first imaging module and the second imaging module can output the first image and the second image at the same frequency. The image processing unit can compare the sharpness of the first image and the second image that are corresponding to each other in time and output the image with higher sharpness. In some embodiments, the first imaging module and the second imaging module output images at different frequencies. In this case, the image processing unit can select one or more first images and one or more second images respectively, and compare the sharpness of the one or more first images and one or more second images. When the number of first images with higher sharpness is greater than the number of second images with higher sharpness, the first image of the first imaging module is selected for output, and the control unit can turn off the second imaging module. When the number of higher-resolution second images exceeds the number of higher-resolution first images, the second image from the second imaging module is selected for output, and the control unit can simultaneously shut down the first imaging module. For example, the image processing unit can select 5 first images and 5 second images from the images output by the two imaging modules within the same time period (e.g., from 11:12 to 11:13). First and second images with the same relative time sequence are grouped together for resolution comparison (e.g., the first and second images output earlier in the aforementioned time period are grouped together for resolution comparison, the first image that ranks second in relative time sequence among the five first images is grouped together with the second image that ranks second in relative time sequence among the five second images is grouped together for resolution comparison, and so on, resulting in five groups). The unit then determines whether the image with higher resolution in each group is the first or second image. If, in three out of the comparison results, the first image from the first imaging module is selected for output, and the control unit can simultaneously shut down the second imaging module.

[0043] In some embodiments, the aforementioned control unit may be implemented by a processor in a processing device. In some embodiments, the control unit may acquire a second instruction or acquire a selection result from the image processing unit, and control the imaging module corresponding to the image that was not selected to stop working. In some embodiments, the control unit and the image processing unit may be the same component or different components. For example, the control unit and the image processing unit may be implemented by the same processor, or, for example, the control unit and the image processing unit may be implemented by different computer code executed by the same processor.

[0044] In some embodiments, when operating in image compositing mode, the image processing unit can determine one or more target image sub-regions from the first image and the second image respectively based on sharpness; and stitch the one or more target image sub-regions together to obtain a composite image. More details regarding image compositing and the determination of target image sub-regions can be found in the related description of Figure 4, and will not be repeated here.

[0045] In some embodiments, to reduce system energy consumption, the frequency of image output by the imaging module can be controlled. Specifically, the sampling frequency of the image sensor can be adjusted to minimize system energy consumption while meeting image requirements and avoid heat accumulation at the front end of the imaging module. In some embodiments, the frequency of image output by the imaging module can be referred to as the image output frequency, which can be understood as the number of images (e.g., image frames) output by the imaging module per unit time (e.g., one minute, one second). In some embodiments, the image output frequency of the imaging module can be determined by the sampling frequency of its internal image sensor; therefore, the image output frequency of the imaging module can be adjusted by adjusting the sampling frequency of the image sensor. In some embodiments, the control unit can adjust the image output frequency of at least one of the first imaging module and the second imaging module based on the sharpness of the first image and the second image. Specifically, the control unit can obtain the sharpness indication values ​​of the first image and the second image; and control the image output frequency of the imaging module corresponding to the image with the higher sharpness indication value to be higher than the image output frequency of the imaging module corresponding to the image with the lower sharpness indication value. As an example, the control unit can directly process the first image and the second image to obtain their sharpness indication values, or the control unit can obtain the sharpness indication values ​​of the first image and the second image from the image processing unit. When the sharpness indicator value of the first image is higher than that of the second image, the control unit can increase the image output frequency of the first imaging module (e.g., adjust from 3 frames / s to 5 frames / s), or the control unit can decrease the image output frequency of the second imaging module (e.g., adjust from 3 frames / s to 1 frame / s). In some embodiments, the control unit can acquire the sharpness indicator values ​​of the first and second images at a certain period and determine the image with the higher sharpness indicator value. The aforementioned period can be 20s, 30s, 1min, 2min, etc. Furthermore, the control unit can dynamically adjust the image output frequency of the first or second imaging module based on the comparison result of the sharpness indicator values. Continuing the previous example, when the sharpness indicator value of the second image is higher than that of the first image in the next sharpness indicator value comparison, the control unit can increase the image output frequency of the second imaging module (e.g., adjust from 1 frame / s to 5 frames / s), or the control unit can decrease the image output frequency of the first imaging module (e.g., adjust from 5 frames / s to 1 frame / s).

[0046] In some embodiments, the control unit may also adjust the image output frequency of the first imaging module or the second imaging module based on the difference in sharpness indicator values ​​between the first image and the second image. Specifically, the control unit may acquire the sharpness indicator values ​​of the first image and the second image; when the difference in sharpness indicator values ​​between the first image and the second image exceeds a second threshold, the control unit controls the image output frequency of the imaging module corresponding to the image with the higher sharpness indicator value to be higher than the image output frequency of the imaging module corresponding to the image with the lower sharpness indicator value; when the difference in sharpness indicator values ​​between the first image and the second image does not exceed the second threshold, the image output frequencies of the first imaging module and the second imaging module are made the same. In some embodiments, the difference in sharpness indicator values ​​between the first image and the second image can be calculated to characterize their difference. The second threshold can be determined based on the energy consumption or heat dissipation of the imaging module, the image quality requirements, and the range of sharpness indicator values. The second threshold can be set to be the same as or different from the first threshold.

[0047] To adapt to the dual-imaging module endoscope system provided in some embodiments of this specification, other embodiments of this specification also provide an image processing method. Figure 4 is an exemplary flowchart of the image processing method shown in some embodiments of this specification. The process 400 shown in Figure 4 can be executed by a processing device (such as processing device 120), and further, can be implemented by an image processing unit or image processing system 800 in the processing device. As shown in Figure 4, the image processing method shown in some embodiments of this specification may include the following steps.

[0048] Step 410: Acquire the first image and the second image. In some embodiments, step 410 may be implemented by the acquisition module 810.

[0049] In some embodiments, the image processing unit may receive a first image and a second image from a first imaging module and a second imaging module. In some embodiments, the imaging module may output images at a preset frequency, and correspondingly, the image processing unit may acquire the first image and the second image output by the first imaging module and the second imaging module frame by frame.

[0050] Subsequently, process 400 can optionally proceed to step 420 or step 430. For example, step 420 or step 430 can be executed based on the first instruction. Alternatively, the sharpness indicator values ​​of the first image and the second image can be obtained; when the difference between the sharpness indicator values ​​of the first image and the second image exceeds a first threshold, step 420 is executed; when the difference between the sharpness indicator values ​​of the first image and the second image does not exceed the first threshold, step 430 is executed. More information about the sharpness indicator value and the first threshold can be found above and will not be repeated here.

[0051] In some embodiments, image metrics such as contrast, sharpness, or resolution of the first and second images can be calculated respectively, and sharpness indicator values ​​of the first and second images can be obtained based on one or more combinations thereof. Figure 5 is an exemplary flowchart of obtaining image sharpness according to other embodiments of this specification. It should be understood that the image described in process 5 can be either the first image or the second image. In some embodiments, process 500 can be implemented by an image processing unit or processing module 820. As shown in Figure 5, process 500 may include the following steps.

[0052] Step 510: Divide the image into N image sub-regions.

[0053] In some embodiments, N can be an integer greater than 1, such as 9, 12, 16, or larger. For example, an image can be divided into 3×3 squares to obtain 9 image sub-regions, or into 3×5 squares to obtain 15 image sub-regions. In some embodiments, a first image can be divided into N1 image sub-regions, and a second image can be divided into N2 image sub-regions; N1 and N2 can be the same or different. In some embodiments, the size of image sub-regions within the same image can be the same or different. The shape of the image sub-regions can be rectangular, or it can be a triangle, pentagon, hexagon, or other polygon.

[0054] Step 520: Calculate the sharpness indicator value for each image sub-region in the image.

[0055] In some embodiments, the contrast, resolution, sharpness, etc., of each image sub-region in the first image can be calculated separately, and a sharpness indicator value can be obtained based on a combination of one or more of them. Similarly, the contrast, resolution, sharpness, etc., of each image sub-region in the second image can be calculated separately, and a sharpness indicator value can be obtained based on a combination of one or more of them. For example, the sharpness indicator value can be obtained based on the sum of the contrast and sharpness of a certain image sub-region, or the contrast of a certain image sub-region can be used as the sharpness indicator value.

[0056] Step 530: Determine the influence weight of each image sub-region in the image.

[0057] In some embodiments, constant influence weights can be set for different image sub-regions. For example, the influence weight of the image sub-region located at the center of the image can be set to a larger constant value, while the influence weight of the image sub-region located at the edge of the image can be set to a smaller constant value. In other embodiments, the influence weights of different image sub-regions in the image can be determined based on the subject being photographed.

[0058] Specifically, the image processing unit can identify either the first image or the second image to determine the current subject. In some embodiments, the image processing unit can input the first image or the second image into a trained image recognition model to determine the current subject. Since the first image and the second image can image the same subject, the subject can be determined by identifying only one of the first and second images. The image recognition model can be a neural network machine learning model, specifically a convolutional neural network model, etc. The image recognition model can be trained using sample images and corresponding labels to enable it to recognize the subject in the image. The label is used to indicate the type or name of the object being photographed in its corresponding image; for example, the label can be the name of an organ or tissue of an organism, such as "stomach" or "intestines."

[0059] In some embodiments, the influence weights of each image sub-region in the corresponding image can be determined in advance based on the physiological structural characteristics of the subject and the diagnostic needs of the doctor, thereby establishing a correspondence between the subject and the influence weights. For example, when the subject is the stomach, because the stomach has a large space and relatively flat tissue, the doctor mainly observes the image center to make a diagnostic judgment. In this case, the image sub-regions closer to the image center can be set to have relatively higher weight values, while the image sub-regions closer to the image periphery can have relatively lower weight values. As another example, when the subject is the intestines, the intestines have a tubular structure, and the intestinal wall tissue is located on the periphery of the image. In this case, all image sub-regions can be set to have the same weight value, or the image sub-regions closer to the image center can have relatively lower weight values, while the image sub-regions closer to the image periphery can have relatively higher weight values. Taking the image divided into 3×3 image sub-regions (a nine-square grid) as an example, when the subject is the stomach, the influence weights of each image sub-region in the corresponding image are: When the subject being photographed is the intestines, the influence weights of each image sub-region in the corresponding image are: After determining the current subject, the image processing unit can determine the influence weight of each image sub-region in the first image and the influence weight of each image sub-region in the second image based on the correspondence between the subject and the influence weight.

[0060] Step 540: Determine the image sharpness based on the sharpness indicator values ​​of each image sub-region in the image and the influence weights.

[0061] Let's assume that the sharpness indicator values ​​of each sub-region in the first image are t. 11 t 12 、…、t 1N1 The influence weight of each image sub-region is s. 11 s 12 、…、s 1N1Therefore, the sharpness or sharpness indicator value of the first image can be calculated (s). 11 ×t 11 +s 12 ×t 12 +…+s 1N1 ×t 1N1 Assume that the sharpness indicator values ​​of each sub-region in the second image are t. 21 t 22 、…、t 2N2 The influence weight of each image sub-region is s. 21 s 22 、…、s 2N2 The sharpness or sharpness indicator value of the second image can be calculated (s). 21 ×t 21 +s 22 ×t 22 +…+s 2N2 ×t 2N2 ).

[0062] Step 420: Select at least one of the first image and the second image and output it. In some embodiments, step 420 may be implemented by processing module 820.

[0063] In some embodiments, a first image, a second image, or both images can be output based on a second instruction. More details about the second instruction can be found above and will not be repeated here. In still other embodiments, the image with higher clarity between the first and second images can be output. The method for obtaining image clarity can be found in the relevant description in Figure 5 and will not be repeated here.

[0064] Step 430: Obtain and output a composite image based on the first image and the second image. In some embodiments, step 430 may be implemented by processing module 820.

[0065] In some embodiments, the imaging ranges of the first image and the second image overlap. For example, they may contain the same object being detected, but the clear areas they contain are different. In this case, the clarity of the image sub-regions corresponding to the same shooting area on the two images can be compared, and the image sub-region with higher clarity can be selected as the target image sub-region. By analogy, the target image sub-regions corresponding to different shooting areas on the first image or the second image can be obtained. Finally, these target image sub-regions can be stitched together to obtain a composite image with more clear areas.

[0066] Figure 6 is an exemplary flowchart illustrating the determination of a target image sub-region according to some embodiments of this specification. In some embodiments, the process 600 shown in Figure 6 can be implemented by the processing module 820. As shown in Figure 6, process 600 may include the following steps.

[0067] Step 610: Register the first image and the second image.

[0068] In some embodiments, the sampling times of the first image and the second image can be the same, or the interval time can not exceed a set time threshold. For example, the time threshold can be 0.1s, 0.5s, 1s, or 2s, etc., which can ensure that the subjects in the first image and the second image have as similar a state as possible. As mentioned above, in some embodiments, the optical lenses in the first imaging module and the second imaging module can be set to have a smaller size (e.g., a smaller diameter) or the distance between the principal optical axes of the first imaging module and the second imaging module can be made as small as possible, thereby ensuring that the two imaging modules have the most similar imaging range possible. In other words, the contents of the first image and the second image can have a high degree of overlap.

[0069] In some embodiments, to improve the quality of image synthesis, the first image and the second image can be registered first. Registration can be understood as aligning two or more images in space. In some embodiments, due to differences in the position and shooting angle of different imaging modules, the same subject in the first image and the second image may have different sizes or orientations. In order to align the first image and the second image in space, at least one of the images can be subjected to one or more of scaling, rotation, and translation. Figure 7 is a schematic diagram of the image registration principle shown in some embodiments of this specification. In Figure 7, image 710 is the first image and image 720 is the second image. It can be seen that the same subject (taking a triangle as an example) has different orientations in the first image and the second image. In order to achieve image registration, feature points of the subject (such as feature points at the pylorus of the stomach, feature points at the cardia, etc.) can be identified in the first image and the second image respectively as positioning reference points, as shown in the positioning reference points A1, B1, and C1 in the first image 710 and A2, B2, and C2 in the second image 720 in Figure 7. Furthermore, coordinate systems can be established for the first image 710 and the second image 720 respectively. For example, the top left corner of each image can be used as the origin, with the positive x-axis pointing horizontally to the right and the positive y-axis pointing vertically downwards, to establish the first image coordinate system and the second image coordinate system. The coordinate values ​​of positioning reference points A1, B1, and C1 in the first image coordinate system, and the coordinate values ​​of positioning reference points A2, B2, and C2 in the second image coordinate system, can be obtained. Spatial transformations (scaling, translation, or rotation) can be performed on the pixels in the first image so that the transformed coordinate values ​​of positioning reference points A1, B1, and C1 correspond to the coordinate values ​​of positioning reference points A2, B2, and C2. Thus, when the transformed first image 710 overlaps with the second image 720, the three positioning reference points can also coincide, achieving spatial alignment of the two images.

[0070] In other embodiments, registering the first and second images may further include reducing the following differences between them: brightness difference and color difference. In some embodiments, the brightness of pixel values ​​in the other image may be adjusted based on the image with lower brightness, thereby reducing the brightness difference between the two. For example, the hue or color saturation of the two images may be adjusted to reduce the color difference between them.

[0071] Step 620: Divide the registered first image and the second image into M first image sub-regions and M second image sub-regions, respectively.

[0072] M can be an integer greater than 1. In some embodiments, M first image sub-regions correspond one-to-one with M second image sub-regions. This correspondence can be understood as the first and second image sub-regions having the same or a repetition rate greater than 80% or a higher threshold of image content. Furthermore, the corresponding first and second image sub-regions may also have the same shape and size. In some embodiments, the registered first and second images can be divided in the same way, resulting in M ​​first image sub-regions that correspond one-to-one with M second image sub-regions.

[0073] Step 630: Calculate the sharpness indicator value for each first image sub-region and each second image sub-region.

[0074] The method for calculating the image sub-region sharpness indicator is the same as the method for calculating the image sharpness indicator. For the specific calculation method, please refer to the relevant explanation on calculating the image sharpness indicator in the previous text, which will not be repeated here.

[0075] Step 640: Select the one with the higher sharpness indicator value from the corresponding first image sub-region and second image sub-region as the target image sub-region, thereby obtaining M target image sub-regions.

[0076] In some embodiments, corresponding first and second image sub-regions can be grouped together, resulting in M ​​groups of image sub-regions. For each group of image sub-regions, the one with the higher sharpness indicator value can be selected as the target image sub-region, thus obtaining M target image sub-regions.

[0077] Finally, the M target image sub-regions are stitched together to obtain the composite image. Since the target image sub-regions with better clarity are selected in each group of image sub-regions, the resulting composite image can have more clear areas, meaning the overall clarity of the composite image will be better.

[0078] Figure 8 is an exemplary block diagram of an image processing system shown in some embodiments of this specification. As shown in Figure 8, the image processing system 800 may include an acquisition module 810 and a processing module 820.

[0079] The acquisition module 810 is used to acquire a first image and a second image. The processing module 820 is used to select at least one of the first image and the second image and output it, or to obtain a composite image based on the first image and the second image and output it.

[0080] For more details on each module, please refer to the relevant descriptions in Figures 4 to 7, which will not be repeated here. It should be understood that the system and its modules shown in Figure 8 can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented using hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated hardware design. Those skilled in the art will understand that the above methods and systems can be implemented using computer-executable instructions and / or included in the control code of a processor, such as on a media such as a disk, CD, or DVD-ROM, or in the memory of a programmable device. The systems and modules of this specification can be implemented not only with hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips and transistors, or programmable hardware devices such as field-programmable gate arrays and programmable logic devices, but also with software executed by various types of processors, or with a combination of the above hardware circuits and software (e.g., firmware).

[0081] It should be noted that the above description of the system and its modules is for convenience only and should not be construed as limiting this specification to the embodiments described. It is understood that those skilled in the art, after understanding the principles of this system, may arbitrarily combine the various modules without departing from these principles to form subsystems connected to other modules. Alternatively, some modules may be split to obtain more modules or multiple units under a single module. Such modifications are all within the scope of this specification.

[0082] One or more embodiments of this specification also provide a computer program product, including computer instructions or computer code, which, when at least a portion of the computer instructions or computer code is executed by a processor, enables the implementation of the image processing method described above.

[0083] One or more embodiments of this specification also provide a computer-readable storage medium storing computer instructions or computer code that, when at least a portion of the computer instructions or computer code is executed by a processor, enables the implementation of the image processing method described above.

[0084] In some embodiments, the aforementioned processor may be a combination of one or more of the following processors: central processing unit (CPU), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), graphics processing unit (GPU), physical processing unit (PPU), digital signal processor (DSP), field-programmable gate array (FPGA), programmable logic device (PLD), programmable logic controller (PLC), reduced instruction set computer (RISC), and microprocessor.

[0085] In some embodiments, the aforementioned storage medium may include one or more combinations of the following: mass storage, removable storage, volatile read-write memory, and read-only memory (ROM). Exemplary mass storage may include disks, optical disks, solid-state drives, etc. Exemplary removable storage may include flash drives, floppy disks, optical disks, memory cards, compressed hard disks, magnetic tapes, etc. Exemplary volatile read-write memory may include random access memory (RAM). Exemplary RAM may include dynamic random access memory (DRAM), dual data rate synchronous dynamic random access memory (DDRSDRAM), static random access memory (SRAM), silicon controlled retrieval memory (T-RAM), and zero-capacitance memory (Z-RAM), etc. Exemplary read-only memory may include masked read-only memory (MROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compressed hard disk read-only memory (CD-ROM), and digital multifunction hard disk read-only memory, etc.

[0086] The beneficial effects that the embodiments of this specification may bring include, but are not limited to: (1) using dual imaging modules to acquire endoscopic images can effectively expand the focusing range or depth of field of the endoscopic system; (2) the two imaging modules have their own optical lenses, and the focal position of the optical lenses can be flexibly selected, so that the focusing range of the two imaging modules is significantly different and adjacent to each other, thus greatly expanding the overall focusing range of the endoscopic system; (3) using injection-molded lenses can make the diameter of the insertion part of the endoscopic system smaller even if dual imaging modules are used; (4) the endoscopic system can work in image selection mode and image synthesis mode to meet the different imaging needs of users; (5) automatically displaying a first image, a second image or a synthesized image with better clarity, improving the overall imaging clarity of the endoscopic system; (6) adjusting the image output frequency of the first imaging module and the second imaging module based on the clarity, reducing system energy consumption while ensuring imaging quality, and effectively avoiding heat accumulation at the tip of the endoscope, thus greatly reducing the impact on the object being examined, etc. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects may be any one or a combination of the above, or any other possible beneficial effects.

[0087] The basic concepts have been described above. It is clear that the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, various modifications, improvements, and corrections may be made to this specification by those skilled in the art. Such modifications, improvements, and corrections are taught in this specification and therefore remain within the concept and scope of the exemplary embodiments described herein.

Claims

1. An endoscope system, characterized in that, It includes a light source, an insertion part, and an image processing part. The insertion part has at least a tip configured to penetrate into the area to be inspected. The tip is provided with a first imaging module and a second imaging module. The light source is used to provide illumination to the area being inspected; The first imaging module is used to acquire the first image; The first imaging module has a first focusing range and a first imaging range; The second imaging module is used to acquire the second image; The second imaging module has a second focus range and a second imaging range; the second focus range and the first focus range are at least partially different, and the second imaging range and the first imaging range have an intersection; The image processing unit is used to select at least one of the first image and the second image and output it, or to obtain a composite image based on the first image and the second image and output it.

2. The endoscope system according to claim 1, characterized in that, It has an image selection mode and an image synthesis mode; the image processing unit is further configured to operate in the image selection mode based on a first instruction to select at least one of the first image and the second image and output it, or operate in the image synthesis mode to obtain a synthesized image based on the first image and the second image and output it.

3. The endoscope system according to claim 1, characterized in that, It has an image selection mode and an image synthesis mode; the image processing unit is further used for: Obtain the sharpness indicator values ​​of the first image and the second image; When the difference in sharpness indicator values ​​between the first image and the second image exceeds a first threshold, the image selection mode is activated to select at least one of the first image and the second image and output it. When the difference in sharpness indicator values ​​between the first image and the second image does not exceed the first threshold, the image synthesis mode is used to obtain and output a synthesized image based on the first image and the second image.

4. The endoscope system according to claim 2 or 3, characterized in that, When operating in the image selection mode, the image processing unit is further configured to output the first image, or output the second image, or output both the first image and the second image simultaneously based on the second instruction.

5. The endoscope system according to claim 2 or 3, characterized in that, When operating in the image selection mode, the image processing unit is further used to output the image with higher clarity between the first image and the second image.

6. The endoscope system according to claim 5, characterized in that, In order to output the image with higher image clarity between the first image and the second image, the image processing unit is further configured to: Divide the first image into N1 image sub-regions, where N1 is an integer greater than 1; calculate the sharpness indicator value for each image sub-region in the first image; Determine the influence weight of each image sub-region in the first image; The sharpness of the first image is determined based on the sharpness indicator value of each image sub-region in the first image and the influence weight; Divide the second image into N2 image sub-regions, where N2 is an integer greater than 1; calculate the sharpness indicator value for each image sub-region in the second image; Determine the influence weight of each image sub-region in the second image; The sharpness of the second image is determined based on the sharpness indicator value of each image sub-region in the second image and the influence weight; Compare the sharpness of the first image and the second image, and output the image with higher sharpness between the first image and the second image; Among them, N1 and N2 may be equal or unequal.

7. The endoscope system according to claim 6, characterized in that, The influence weight of each sub-region in the first image is a constant value or determined based on the subject being photographed; the influence weight of each sub-region in the second image is a constant value or determined based on the subject being photographed.

8. The endoscope system according to claim 7, characterized in that, The image processing unit is also used to determine the influence weight of each image sub-region in the first image or the second image; To determine the influence weights of each image sub-region in the first image or the second image, the image processing unit is further configured to: Identify the first image or the second image to determine the current subject being photographed; Based on the correspondence between the photographed object and the influence weight, the influence weight of each image sub-region in the first image or the second image is determined.

9. The endoscope system according to claim 2 or 3, characterized in that, It also includes the control unit; The control unit is used to control the imaging module corresponding to the image that the image processing unit has not selected for output to stop working when the image processing unit is working in the image selection mode.

10. The endoscope system according to claim 2 or 3, characterized in that, When operating in the image synthesis mode, the image processing unit is further configured to: One or more target image sub-regions are determined from the first image and the second image respectively based on the sharpness; The composite image is obtained by stitching together one or more target image sub-regions.

11. The endoscope system according to claim 10, characterized in that, In order to determine one or more target image sub-regions from the first image and the second image respectively based on sharpness, the image processing unit is further configured to: Register the first image with the second image; The registered first image and the second image are divided into M first image sub-regions and M second image sub-regions, respectively, where M is an integer greater than 1, and the M first image sub-regions correspond one-to-one with the M second image sub-regions; Calculate the sharpness indicator value for each first image sub-region and each second image sub-region respectively; Select the one with the higher sharpness indicator value from the corresponding first image sub-region and second image sub-region as the target image sub-region, and then obtain M target image sub-regions.

12. The endoscope system according to claim 11, characterized in that, In order to register the first image with the second image, the image processing unit is further configured to: Perform one or more of the following operations on at least one of the first image and the second image: scaling, translation, rotation, so that two or more positioning reference points in the first image and two or more positioning reference points in the second image are aligned one by one; Alternatively, reduce the following differences between the first image and the second image: brightness difference and color difference.

13. The endoscope system according to claim 1, characterized in that, It also includes the control unit; The control unit is used to adjust the image output frequency of at least one of the first imaging module and the second imaging module based on the sharpness of the first image and the second image.

14. The endoscope system according to claim 13, characterized in that, The control unit is further used for: Obtain the sharpness indicator values ​​of the first image and the second image; The image output frequency of the imaging module corresponding to the image with a higher sharpness indicator value in the first image and the second image is controlled to be higher than the image output frequency of the imaging module corresponding to the image with a lower sharpness indicator value.

15. The endoscope system according to claim 14, characterized in that, The control unit is further used for: Obtain the sharpness indicator values ​​of the first image and the second image; When the difference in sharpness indicator values ​​between the first image and the second image exceeds a second threshold, the image output frequency of the imaging module corresponding to the image with the higher sharpness indicator value is controlled to be higher than the image output frequency of the imaging module corresponding to the image with the lower sharpness indicator value. When the difference in sharpness indicator values ​​between the first image and the second image does not exceed a second threshold, the image output frequencies of the first imaging module and the second imaging module are made to be the same.

16. The endoscope system according to claim 1, characterized in that, The first imaging module includes a first optical lens and a first image sensor. The first optical lens is used to acquire a first optical image, and the first image sensor is used to receive the first optical image and convert it into the first image. The second imaging module includes a second optical lens and a second image sensor. The second optical lens is used to acquire a second optical image, and the second image sensor is used to receive the second optical image and convert it into the second image. The principal optical axis of the first optical lens is parallel to the principal optical axis of the second optical lens.

17. The endoscopic system according to claim 16, characterized in that, The first optical lens and the second optical lens are fixed-focus lenses.

18. The endoscope system according to claim 16, characterized in that, The distance between the principal optical axis of the first optical lens and the principal optical axis of the second optical lens is less than the radius of the tip end.

19. The endoscopic system according to claim 16, characterized in that, The focal point of the first optical lens is located in the area of ​​7mm to 15mm on its front end surface, and the focal point of the second optical lens is located in the area of ​​2mm to 6mm on its front end surface.

20. An endoscope insertion part, characterized in that, It has at least a tip configured to penetrate into the area to be inspected, the tip being provided with a first imaging module and a second imaging module; The first imaging module is used to acquire the first image; The first imaging module has a first focusing range and a first imaging range; The second imaging module is used to acquire the second image; The second imaging module has a second focus range and a second imaging range; the second focus range and the first focus range are at least partially different, and the second imaging range and the first imaging range have an intersection.

21. An image processing method for an endoscope system, characterized in that, include: Acquire a first image and a second image, the first image and the second image having different focal regions, and the imaging range corresponding to the first image and the imaging range corresponding to the second image having an intersection; Select at least one of the first image and the second image and output it, or obtain a composite image based on the first image and the second image and output it.

22. The image processing method according to claim 21, characterized in that, The step of selecting at least one of the first image and the second image and outputting it, or obtaining and outputting a composite image based on the first image and the second image, includes: Select at least one of the first image and the second image based on the first instruction and output it, or obtain a composite image based on the first image and the second image and output it.

23. The image processing method according to claim 21, characterized in that, The step of selecting at least one of the first image and the second image and outputting it, or obtaining and outputting a composite image based on the first image and the second image, includes: Obtain the sharpness indicator values ​​of the first image and the second image; When the difference in sharpness indicator values ​​between the first image and the second image exceeds a first threshold, at least one of the first image and the second image is selected and output. When the difference in sharpness indicator values ​​between the first image and the second image does not exceed the first threshold, a composite image is obtained based on the first image and the second image and output.

24. The image processing method according to claim 21, characterized in that, The step of selecting and outputting at least one of the first image and the second image includes: The first image, the second image, or both the first and second images can be output based on the second instruction.

25. The image processing method according to claim 21, characterized in that, The step of selecting and outputting at least one of the first image and the second image includes: Output the image with higher image clarity between the first image and the second image.

26. The image processing method according to claim 25, characterized in that, The step of outputting the image with higher image clarity between the first image and the second image includes: The first image is divided into N1 image sub-regions, where N1 is an integer greater than 1; the sharpness indicator value of each image sub-region in the first image is calculated; the influence weight of each image sub-region in the first image is determined; the sharpness of the first image is determined based on the sharpness indicator value and influence weight of each image sub-region in the first image. The second image is divided into N2 image sub-regions, where N2 is an integer greater than 1; the sharpness indicator value of each image sub-region in the second image is calculated; the influence weight of each image sub-region in the second image is determined; the sharpness of the second image is determined based on the sharpness indicator value and influence weight of each image sub-region in the second image. Compare the sharpness of the first image and the second image, and output the image with higher sharpness between the first image and the second image; Among them, N1 and N2 may be equal or unequal.

27. The image processing method according to claim 26, characterized in that, The influence weight of each sub-region in the first image is a constant value or determined based on the subject being photographed; the influence weight of each sub-region in the second image is a constant value or determined based on the subject being photographed.

28. The image processing method according to claim 27, characterized in that, Also includes: Determine the influence weight of each image sub-region in the first image or the second image; The determination of the influence weights of each image sub-region in the first image or the second image further includes: Identify the first image or the second image to determine the current subject being photographed; Based on the correspondence between the photographed object and the influence weight, the influence weight of each image sub-region in the first image or the second image is determined.

29. The image processing method according to claim 21, characterized in that, The process of obtaining and outputting a synthesized image based on the first image and the second image includes: One or more target image sub-regions are determined from the first image and the second image respectively based on the sharpness; The composite image is obtained by stitching together one or more target image sub-regions.

30. The image processing method according to claim 29, characterized in that, The step of determining one or more target image sub-regions from the first image and the second image based on sharpness includes: Register the first image with the second image; The registered first image and the second image are divided into M first image sub-regions and M second image sub-regions, respectively, where M is an integer greater than 1, and the M first image sub-regions correspond one-to-one with the M second image sub-regions; Calculate the sharpness indicator value for each first image sub-region and each second image sub-region respectively; Select the one with the higher sharpness indicator value from the corresponding first image sub-region and second image sub-region as the target image sub-region, and then obtain M target image sub-regions.

31. The endoscope system according to claim 30, characterized in that, The registration of the first image and the second image includes: Perform one or more of the following operations on at least one of the first image and the second image: scaling, translation, rotation, so that two or more positioning reference points in the first image and two or more positioning reference points in the second image are aligned one by one; Alternatively, reduce the following differences between the first image and the second image: brightness difference and color difference.

32. A computer-readable storage medium storing computer code or instructions that, when at least a portion of the computer code or instructions is executed by a processor, enables the implementation of the method as claimed in any one of claims 21 to 31.

33. A computer program product comprising computer code or instructions that, when executed by a processor, enables the implementation of the method as described in any one of claims 21 to 31.

Images