An endoscope system and an image generation method for an endoscope system

Abstract

An endoscope system and an image generation method for the endoscope system, acquiring real-time first and second images; calculating the definition of the first image and the definition of the second image; determining image field translation information related to image field smooth display according to the definition of the first image and the definition of the second image; obtaining a third image for output display according to the first image, the second image and the image field translation information. Different types of definition are calculated based on different image processing methods or the direction information of the movement of the endoscope relative to the target tissue is acquired, the corresponding image field translation information in different image processing methods is obtained, and then the image is processed accordingly, so that the picture jumping is reduced and / or the image misplacement is reduced or even eliminated without increasing additional distance sensors, so that different image processing methods can obtain image observation results that are more consistent with the needs of human eye observation.

Description

Technical Field

[0001] This invention relates to the field of endoscopy, and more specifically to an endoscopy system and an image generation method for an endoscopy system. Background Technology

[0002] With the rapid development of miniaturized sensors and minimally invasive medical technologies, endoscopic minimally invasive or non-invasive medical examinations and treatments have become widely adopted. Endoscopic technology allows imaging components to enter the interior of organisms (such as humans and animals) through natural openings or small surgical incisions, thereby acquiring image information of the examined area and providing more intuitive and accurate diagnostic information. Electronic endoscopes integrate imaging lenses and image sensors at the probe tip, transmitting the captured images to an image processing unit for processing and display on a screen for user observation.

[0003] Traditional electronic endoscopes have limited depth of field due to optical limitations, making it difficult to simultaneously capture clear images of both distant and near objects. Therefore, extended depth-of-field endoscopes based on dual lenses have emerged. These endoscopes acquire images of the object at different working distances using two imaging lenses, displaying the clearer image or fusing the clearer parts of both lenses to ultimately present a clear imaging result.

[0004] However, a significant problem with this method is that the optical paths for near and far focus imaging are physically offset, resulting in spatial misalignment of the images formed by the near and far focus optical paths. This causes the image to jump when switching between near and far focus. Furthermore, the offsets are not consistent at different imaging distances, which makes it easy for misalignment to occur when fusing near and far focus images, thus affecting the final image observation effect. Summary of the Invention

[0005] The main technical problem solved by this invention is how to solve the offset problem of the extended depth-of-field endoscope based on dual lenses, so as to reduce image jitter when switching between near and far views, and significantly reduce or even eliminate the occurrence of image misalignment when fusing near and far images.

[0006] According to a first aspect, one embodiment provides an endoscope system, comprising:

[0007] The guide portion has at least a tip configured to penetrate into the area being inspected;

[0008] An imaging assembly, comprising a first imaging module and a second imaging module, wherein the first imaging module and the second imaging module are disposed at the tip end of the guide portion, the first imaging module being used to acquire a first image of the target tissue within the inspected area, and the second imaging module being used to acquire a second image of the target tissue within the inspected area.

[0009] And a processor, used for:

[0010] Acquire the first and second images in real time;

[0011] Calculate the sharpness of the first image and the sharpness of the second image;

[0012] Based on the sharpness of the first image and the sharpness of the second image, determine the image field of view translation information related to the smooth display of the image field of view;

[0013] A third image is obtained for output display based on the first image, the second image, and the image field of view translation information.

[0014] According to a second aspect, one embodiment provides an endoscope system, comprising:

[0015] The guide portion has at least a tip configured to penetrate into the area being inspected;

[0016] An imaging assembly, comprising a first imaging module and a second imaging module, wherein the first imaging module and the second imaging module are disposed at the tip end of the guide portion, the first imaging module being used to acquire a first image of the target tissue within the inspected area, and the second imaging module being used to acquire a second image of the target tissue within the inspected area.

[0017] And a processor, used for:

[0018] Acquire the first and second images in real time;

[0019] Obtain the current direction information of the movement of the tip of the guide relative to the target tissue;

[0020] Based on the directional information, determine the image field of view translation information related to the smooth display of the image field of view;

[0021] A third image is obtained for output display based on the first image, the second image, and the image field of view translation information.

[0022] According to a third aspect, one embodiment provides an image generation method for an endoscope system, the endoscope system including a guide portion and an imaging assembly, the guide portion having at least a tip configured to penetrate into an area to be examined, the imaging assembly including a first imaging module and a second imaging module disposed at the tip of the guide portion; the first imaging module having a first focus range, the second imaging module having a second focus range, the second focus range and the first focus range intersecting, and the minimum value of the first focus range being greater than the minimum value of the second focus range; the image generation method includes:

[0023] Acquire a first image and a second image in real time, wherein the first image is an image of the target tissue within the inspected area obtained by the first imaging module imaging, and the second image is an image of the target tissue within the inspected area obtained by the second imaging module imaging;

[0024] Calculate the sharpness of the first image and the sharpness of the second image;

[0025] Based on the sharpness of the first image and the sharpness of the second image, determine the image field of view translation information related to the smooth display of the image field of view;

[0026] A third image is obtained for output display based on the first image, the second image, and the image field of view translation information.

[0027] According to a fourth aspect, one embodiment provides an image generation method for an endoscope system, the endoscope system including a guide portion and an imaging assembly, the guide portion having at least a tip configured to penetrate into an area to be examined, the imaging assembly including a first imaging module and a second imaging module disposed at the tip of the guide portion, the first imaging module having a first focusing range, the second imaging module having a second focusing range, the second focusing range and the first focusing range intersecting, and the minimum value of the first focusing range being greater than the minimum value of the second focusing range; the image generation method includes:

[0028] Acquire a first image and a second image in real time, wherein the first image is an image of the target tissue within the inspected area obtained by the first imaging module imaging, and the second image is an image of the target tissue within the inspected area obtained by the second imaging module imaging;

[0029] Obtain the current direction information of the movement of the tip of the guide relative to the target tissue;

[0030] Based on the directional information, determine the image field of view translation information related to the smooth display of the image field of view;

[0031] A third image is obtained for output display based on the first image, the second image, and the image field of view translation information.

[0032] According to the above embodiment of an endoscope system and an image generation method for the endoscope system, firstly, a first image and a second image are acquired in real time by a first imaging module (far-view lens) and a second imaging module (close-view lens) installed on the tip of the guide section; then, different types of sharpness are calculated according to different image processing methods used by the endoscope in actual operation, and image field of view translation information related to smooth display of image field of view in different image processing methods is determined; based on the obtained image field of view translation information, the image is processed accordingly, thereby reducing image jitter and / or reducing or even eliminating image misalignment without adding an additional distance sensor, so that different image processing methods can ultimately obtain image observation results that better meet the observation needs of the human eye;

[0033] In the image switching process, the image translation direction is determined based on the real-time sharpness difference and its changes between the overall sharpness of the first and second images. The image translation amount is calculated by comparing the real-time sharpness difference with the first and second preset thresholds. Based on the overall sharpness of the first and second images, image field-of-view translation information related to smooth image display in the image switching process is determined. The first and second images are translated according to the obtained image field-of-view translation information to obtain a third image for output display. Thus, based on the real-time sharpness differences and their changes in the images acquired by different imaging modules, the overall image is slightly translated, thereby achieving a smooth transition in the image switching process and reducing screen jitter.

[0034] For image fusion processing, the first and second images are partitioned, and a sharpness distribution difference map is obtained based on the difference in local sharpness in the same partition in different images. This results in a reference image and a non-reference image. The pixel translation direction and the pixel translation amount corresponding to each pixel in the non-reference image are also obtained. Based on the local sharpness of the first and second images, the image field of view translation information related to smooth image display in image fusion processing is determined. The pixels in the non-reference image are translated according to the obtained image field of view translation information and filled into the corresponding positions in the reference image to obtain the third image for output display. In this way, the image misalignment is reduced or even eliminated by adjusting the position of individual pixels in the non-reference image. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the structure of an endoscope system;

[0036] Figure 2 A flowchart illustrating how a processor in an endoscope system acquires a third image based on image sharpness;

[0037] Figure 3This is a flowchart for obtaining a third image for output display through image switching processing;

[0038] Figure 4 This is a flowchart illustrating how image fusion processing is used to obtain a third image for output display.

[0039] Figure 5 This is a schematic diagram of the first preset mapping table;

[0040] Figure 6 This is a schematic diagram of obtaining a third image based on an image switching processing method;

[0041] Figure 7 This is a schematic diagram of the first image;

[0042] Figure 8 This is a schematic diagram of the second image;

[0043] Figure 9 This is a schematic diagram of the sharpness distribution difference map;

[0044] Figure 10 This is a schematic diagram of the third image obtained based on image fusion processing.

[0045] Figure 11 A flowchart illustrating how a processor acquires a third image based on directional information of the movement of the guide unit relative to the target tissue;

[0046] Figure 12 This is a flowchart of an image generation method for an endoscope system.

[0047] Figure 13 This is a flowchart of another image generation method for endoscope systems. Detailed Implementation

[0048] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0049] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0050] The serial numbers assigned to components in this document, such as "first" and "second," are used only to distinguish the described objects and have no sequential or technical meaning. The terms "connection" and "linkage" used in this application, unless otherwise specified, include both direct and indirect connections (linkages).

[0051] A dual-lens extended depth-of-field endoscope comprises two lenses: a telephoto lens and a close-up lens. These two lenses have different focus ranges, and their focus ranges overlap in an area known as the depth-of-field overlap region. This overlap region means that within this area, both the telephoto and close-up lenses can capture relatively sharp images. However, due to the optimal focus range of optical lenses, although both imaging lenses capture relatively sharp images within the depth-of-field overlap region, there is still a difference in image sharpness between them. In other words, within the depth-of-field overlap region, one of the images captured by the two imaging lenses is the sharpest. In practical use… During the process, the field of view areas of the two lenses are not completely identical. Different image processing methods can be selected according to the actual situation, including image switching or image fusion. In particular, during image switching, in order to avoid significant changes in the field of view areas corresponding to the images displayed before and after the image switching, and thus avoid phenomena such as the loss of target tissue in the image after switching, a smooth transition between the corresponding images of different lenses needs to be achieved during image switching. In the image fusion process, the difference in the field of view areas of the two lenses can also cause pixel misalignment at the same location in the corresponding images of different lenses. Therefore, when fusing different lenses, it is also necessary to reduce or even eliminate misalignment to ensure the final observation effect.

[0052] In this embodiment of the invention, for different image processing methods during the use of the endoscope, such as switching between near and far images or fusing near and far images, the corresponding image translation amount or pixel translation amount is calculated so that during the image switching process, a smooth transition between images can be achieved based on the image translation amount to reduce image jitter; during the image fusion process, the positions of different pixels can be adjusted based on the pixel translation amount to reduce or even eliminate image misalignment, thereby ensuring the final image observation effect and making it more in line with the observation needs of the human eye.

[0053] Please refer to Figure 1 Some embodiments provide an endoscope system including a guide 10, an imaging component 11, and a processor 12. The guide 10 has at least a tip configured to penetrate into an area to be examined. The imaging component 11 includes a first imaging module and a second imaging module, which are disposed at the tip of the guide. The first imaging module is used to acquire a first image of the target tissue within the area to be examined, and the second imaging module is used to acquire a second image of the target tissue within the area to be examined. Furthermore, the first imaging module corresponds to the endoscope's telephoto lens, and the second imaging module corresponds to the endoscope's near-field lens. The two have different focusing ranges. In this embodiment, the focusing ranges of the two are respectively denoted as the first focusing range and the second focusing range. That is, the first imaging module has a first focusing range, the second imaging module has a second focusing range, and the minimum value of the first focusing range is greater than the minimum value of the second focusing range. The second focusing range and the first focusing range have an intersection, which corresponds to the depth-of-field overlap area.

[0054] In this embodiment, the processor 12 performs image processing (such as image switching or image fusion processing) on ​​the first and second images acquired in real time by the first and second imaging modules to obtain a third image for output display. The processing flow of the processor to obtain the third image based on image clarity in this embodiment is as follows: Figure 2 As shown, the specific steps include:

[0055] Step S100: Acquire the first and second images in real time.

[0056] During the use of the endoscope, real-time image acquisition is performed through the first imaging module and the second imaging module, thereby obtaining the first image and the second image corresponding to the current moment.

[0057] Step S110: Calculate the sharpness of the first image and the sharpness of the second image.

[0058] The sharpness in this embodiment includes overall sharpness and / or local sharpness. For example, when evaluating sharpness, methods such as the Tenengrad function, feature extraction amount, and edge contrast amount can be used.

[0059] Among them, overall sharpness refers to the sharpness of the entire image, while local sharpness refers to the sharpness of different areas of the image. In this embodiment, the first image and the second image are divided into multiple partitions. The size of each partition can be set according to the actual situation. For example, the size of each partition can be set to 64 pixels × 64 pixels. In order to ensure the accuracy and effectiveness of processing, the minimum partition size is specified to be no less than 4 pixels × 4 pixels.

[0060] Then, the sharpness corresponding to each partition in the first image is recorded as the local sharpness of each partition in the first image; the second image is divided into multiple partitions, and the sharpness corresponding to each partition in the second image is recorded as the local sharpness of each partition in the second image; and the number of partitions corresponding to the first image is the same as the number of partitions corresponding to the second image. At this time, the local sharpness corresponding to all partitions in the first image constitutes the sharpness distribution difference map of the first image; the local sharpness corresponding to all partitions in the second image constitutes the sharpness distribution difference map of the second image.

[0061] Step S120: Determine image field of view translation information related to smooth image field of view display based on the sharpness of the first image and the sharpness of the second image.

[0062] In this embodiment, the image field of view translation information related to smooth image field of view display includes the image translation direction and the image translation amount, and / or the pixel translation amount and the pixel translation direction. Image switching processing can be performed based on the image translation direction and the image translation amount, and image fusion processing can be performed based on the pixel translation amount.

[0063] The real-time sharpness difference is obtained based on the overall sharpness of the first image and the overall sharpness of the second image, including:

[0064] Based on the overall sharpness of the current first image and the second image, the image translation direction and the amount of image translation can be calculated so that a smooth transition can be achieved when switching images later. That is: firstly, calculate the difference between the overall sharpness of the first image and the overall sharpness of the second image to obtain the real-time sharpness difference. Let the overall sharpness of the first image be C1 and the overall sharpness of the second image be C2. The difference between the two, C1-C2, is taken as the current real-time sharpness difference.

[0065] The image translation amount is obtained based on the real-time sharpness difference, including:

[0066] In some embodiments, the image translation amount is obtained based on the real-time sharpness difference and a first preset mapping table, such as... Figure 5 As shown, it is used to describe the correspondence between real-time sharpness differences and image translation amounts; the first preset mapping table is pre-set with multiple pairs of real-time sharpness differences and their corresponding image translation amounts; when the real-time sharpness difference cannot be directly found in the first preset mapping table, the image translation amount corresponding to the real-time sharpness difference can be obtained by interpolation of nearby points, so that the real-time sharpness difference and the image translation amount are in a one-to-one correspondence.

[0067] In some embodiments, the acquisition of the image translation amount can also be based on the degree of change of the current real-time clarity difference relative to the real-time clarity difference corresponding to the previous moment. For example, calculate the difference between the current real-time clarity difference A and the real-time clarity difference B corresponding to the previous moment, and then obtain the ratio between the obtained difference and the real-time clarity difference B corresponding to the previous moment. At this time, the obtained ratio is |A - B| / B, and use the obtained ratio as the degree of change of the current real-time clarity difference relative to the real-time clarity difference corresponding to the previous moment; finally, obtain the image translation amount according to the obtained degree of change. For example, preset a maximum translation amount, and use the product of the obtained degree of change and the maximum translation amount as the image translation amount.

[0068] Obtaining the image translation direction according to the current real-time clarity difference and the real-time clarity difference of the previous moment includes:

[0069] Generally, there are three states of the endoscope moving relative to the target tissue: gradually approaching the target tissue, reaching near the target tissue, and gradually moving away from the target tissue. Among them, in the process of the endoscope starting to enter and gradually approaching the target tissue, the clarity of the image corresponding to the long-distance lens is higher than the clarity of the image corresponding to the close-up lens, that is, C1 > C2, and the corresponding real-time clarity difference C1 - C2 is greater than 0. To better meet the observation requirements, the third image for display is default to the first image of the long-distance lens. If image switching processing is required subsequently, the long-distance lens needs to be switched to the close-up lens; during this process, the clarity of the image corresponding to the long-distance lens will gradually decrease, and the clarity of the image corresponding to the close-up lens will gradually increase. At this time, the gap between the corresponding clarities of the two will gradually narrow, and the corresponding real-time clarity difference C1 - C2 will also gradually decrease; continuously judge whether the real-time clarity difference meets the preset conditions. In this embodiment, it is to continuously compare the real-time clarity difference with the first preset threshold and the second preset threshold. Exemplarily, when the method for measuring image clarity is the Tenengrad function, the first preset threshold can be set to 100, and the second preset threshold can be set to 10; until the real-time clarity difference is less than the second preset threshold, switch the long-distance lens to the close-up lens. At this time, the third image for output display is the second image collected by the close-up lens / second imaging module;

[0070] When the endoscope reaches near the target tissue, at this time the target tissue is within the depth-of-field overlap area of the long-distance lens and the close-up lens. At this time, the target tissue is relatively clear in the images corresponding to both the long-distance lens and the close-up lens, that is, at this time the clarity of the first image and the second image is similar within the depth-of-field overlap area. Continuing to approach the target tissue, the clarity of the image corresponding to the close-up lens is gradually higher than the clarity of the long-distance image, that is, C1 < C2, and the corresponding real-time clarity difference C1 - C2 is less than 0. If image switching processing is required subsequently, the close-up lens needs to be switched to the long-distance lens;

[0071] As the endoscope moves further away from the target tissue, especially after leaving the depth-of-field overlap area, the clarity of the image corresponding to the distant lens gradually increases, while the clarity of the image corresponding to the close-up lens gradually decreases. At this time, the difference between the two corresponding clarity gradually increases, and the corresponding real-time clarity difference C1-C2 also gradually increases. The real-time clarity difference is continuously judged to see if it meets the preset conditions until the real-time clarity difference is greater than the first preset threshold, at which point the close-up lens is switched to the distant lens.

[0072] In order to reduce screen jitter during image switching, the images need to be slightly shifted during actual use of the endoscope based on the real-time difference in clarity between the first and second images, so as to achieve a smooth transition between distant and close-up images.

[0073] This embodiment obtains the image translation direction during minute translation based on the changes in real-time sharpness differences over at least three time points. If the imaging module corresponding to the currently displayed image is the first imaging module / distant lens, and the current time is recorded as the i-th time point (i≥3), then if the real-time sharpness difference corresponding to the current i-th time point is less than the real-time sharpness difference corresponding to the (i-1)-th time point, and the real-time sharpness difference corresponding to the (i-1)-th time point is less than the real-time sharpness difference corresponding to the (i-2)-th time point, it indicates that the real-time sharpness difference is continuously decreasing, the difference in sharpness between the images corresponding to the two lenses is gradually narrowing, corresponding to a decrease in the sharpness of the distant image and an increase in the sharpness of the close-up image. In other words, it is a situation where the image is gradually approaching the target tissue. Therefore, if an image switching is required, the distant lens must be switched to the close-up lens, so the image translation direction is the positive direction.

[0074] If the imaging module corresponding to the currently displayed image is the second imaging module / close-up lens, and if the real-time sharpness difference at the current i-th moment is greater than the real-time sharpness difference at the i-1-th moment, and the real-time sharpness difference at the i-1-th moment is greater than the real-time sharpness difference at the i-2-th moment, it means that the real-time sharpness difference is continuously increasing, and the difference in sharpness between the images corresponding to the two lenses is also gradually increasing. This corresponds to the situation where the sharpness of the close-up image decreases, while the sharpness of the distant image increases. In other words, it means that the image is gradually moving away from the target tissue. Therefore, if an image switching is required, the close-up lens must be switched to the distant lens. Hence, the image translation direction is the negative direction.

[0075] It should be noted that the positive / negative direction of image translation is determined by the structure of the lens, specifically by the relative positions of the distant and close-up lenses. For example, when the distant lens is on the left and the close-up lens is on the right, the image translation direction when switching from a distant to a close-up view is the direction of the close-up lens relative to the distant lens, which means the image translation direction is to the right. Conversely, when switching from a close-up to a distant view, the image translation direction is the direction of the distant lens relative to the close-up lens, which means the image translation direction is to the left. In this embodiment, rightward translation is taken as the positive direction, and leftward translation is taken as the negative direction.

[0076] Furthermore, this embodiment can automatically switch images based on the overall clarity of the first and second images, or the user can switch images according to the actual needs observed during use. The image translation direction and amount are obtained based on the lens corresponding to the currently displayed image and the real-time clarity difference between the first and second images. At this time, the image translation direction is the direction of the lens after switching relative to the lens before switching. For example, if the current lens is a distant view, the corresponding image translation direction is positive when switching to a close-up lens; otherwise, the corresponding image translation direction is negative. The image translation amount can be obtained based on the real-time clarity difference and the first preset mapping table, which will not be elaborated here.

[0077] The pixel translation direction and pixel translation amount are obtained based on the local sharpness of the first and second images, including:

[0078] For any given partition, the difference between the local sharpness of that partition in the first image and the local sharpness of that partition in the second image is calculated. This difference is then used as the local sharpness difference of that partition. If the local sharpness difference of that partition is positive, it means that the partition has high sharpness in the first image; if the local sharpness difference of that partition is negative, it means that the partition has high sharpness in the second image. This process yields the local sharpness difference for each partition, and the local sharpness differences for all partitions constitute a sharpness distribution difference map.

[0079] Since organs and tissues are continuous in actual use, there are generally no sudden changes in image sharpness, such as when the distant view is sharper and the foreground view is sharper in the next moment. Therefore, in order to prevent noise from affecting the calculation of pixel translation, this embodiment needs to perform image smoothing on the obtained sharpness distribution difference map or according to the correspondence of adjacent pixels. For ease of description, this embodiment still refers to the processed sharpness distribution difference map as the sharpness distribution difference map. That is to say, the sharpness distribution difference map processed in subsequent steps is actually the smoothed sharpness distribution difference map.

[0080] For example, in this embodiment, the first image is shown in Figure 7, and the second image is shown in Figure 8. Figure 8 As shown, the local sharpness of corresponding partitions in the first image and the second image are subtracted to obtain the local sharpness difference of each partition. The local sharpness differences corresponding to all partitions constitute a sharpness distribution difference map. At this time, the pixel value of each pixel in the sharpness distribution difference map is equal to the local sharpness difference of the partition to which that pixel belongs. In this embodiment, based on Figure 7 and Figure 8 The resulting sharpness distribution difference map is as follows Figure 9 As shown; in Figure 9 In the image, the x and y coordinates of each pixel correspond to the x and y coordinates of the pixel in the first / second image, and the y coordinate of each pixel represents the pixel value in the sharpness distribution difference map.

[0081] Then, based on the obtained sharpness distribution difference map, a reference image and a non-reference image are selected from the first image and the second image. In this embodiment, the image containing more sharp partitions in the first image and the second image is used as the reference image, and the other image is used as the non-reference image.

[0082] In some embodiments, a reference image and a non-reference image are selected based on the positive or negative sign of the local sharpness difference corresponding to different partitions in the sharpness distribution difference map. For example, in all partitions of the sharpness distribution difference map, if the number of partitions with positive local sharpness differences is greater than or equal to the number of partitions with negative local sharpness differences, that is, if the number of relatively sharp partitions in the first image is greater than or equal to the number of relatively sharp partitions in the second image, then the first image is used as the reference image and the second image is used as the non-reference image; otherwise, the second image is used as the reference image and the first image is used as the non-reference image.

[0083] The pixel translation direction is obtained based on the reference image and the non-reference image. The pixel translation direction is the direction of the lens corresponding to the reference image relative to the lens corresponding to the non-reference image. Specifically, when the first image is the reference image and the second image is the non-reference image, the lens corresponding to the reference image is a long shot, and the lens corresponding to the non-reference image is a close shot. If the long shot is on the left and the close shot is on the right, then the lens corresponding to the reference image is located to the left of the lens corresponding to the non-reference image, which is the direction of the long shot relative to the close shot. To reduce or even eliminate the spatial misalignment between the non-reference image and the reference image, the non-reference image needs to be translated to the left first, so the corresponding pixel translation direction is negative. When the second image is the reference image and the first image is the non-reference image, the corresponding pixel translation direction is positive.

[0084] The pixel shift of the corresponding pixel in the non-reference image can be obtained based on the pixel value of each pixel in the sharpness distribution difference map. For example, the pixel shift of each pixel in the non-reference image can be obtained based on the pixel value of each pixel in the sharpness distribution difference map and a second preset mapping table. The second preset mapping table describes the correspondence between the pixel value and the pixel shift of each pixel in the sharpness distribution difference map. The second preset mapping table pre-sets multiple pairs of correspondences between pixel values ​​and pixel shifts. When a pixel value cannot be directly found in the second preset mapping table, the pixel shift corresponding to the pixel value can be obtained by interpolation of neighboring points, so that different pixel values ​​and pixel shifts in the sharpness distribution difference map have a one-to-one correspondence.

[0085] Step S130: Based on the first image, the second image, and the image field of view translation information, obtain the third image for output display.

[0086] For image switching processing, the process of obtaining the third image for output display based on the first image, the second image, and the image translation direction and amount is as follows: Figure 3 As shown, it includes:

[0087] Step S131: Translate the first image and the second image according to the image field of view translation information to obtain the translation regions corresponding to the first image and the second image respectively.

[0088] When the image translation direction is positive, the first and second images are shifted to the right. If the real-time sharpness difference is greater than or equal to a first preset threshold, the real-time sharpness difference is obtained based on the overall sharpness of the first and second images acquired at the next moment, and compared again with the first preset threshold. If the obtained real-time sharpness difference is less than the first preset threshold, the image translation amount is obtained based on the obtained real-time sharpness difference. The obtained image translation amount is the distance of the overall image translation, and the first and second images are translated accordingly, thereby obtaining the translation areas corresponding to the first and second images, such as... Figure 6 As shown, Figure 6 A schematic diagram is shown to obtain a third image based on the corresponding translation regions of the first and second images. Let the size of the first and second images be M×N. If the image translation amount is L1, that is, both the first and second images are translated to the right by a distance of L1, then the translation regions corresponding to the first and second images are P1 and P2, respectively. And let the remaining regions in the first and second images, excluding the translation regions, be S1 and S2, respectively.

[0089] When the image translation direction is negative, the first and second images are shifted to the left. If the real-time sharpness difference is less than or equal to the second preset threshold, the real-time sharpness difference is obtained based on the overall sharpness of the first and second images acquired at the next moment, and compared again with the second preset threshold. If the real-time sharpness difference is greater than the second preset threshold, the image translation amount is obtained based on the change in the real-time sharpness difference. The obtained image translation amount is the distance of the overall image translation, and the first and second images are translated to obtain the translation areas corresponding to the first and second images. If the image translation amount at this time is L2, then in Figure 6 In the first image and the second image, the translation regions correspond to S1 and S2 respectively; then the remaining regions in the first image and the second image other than the translation regions are P1 and P2 respectively.

[0090] Step S132: Obtain the third image for output display based on the translation area corresponding to the first image and the second image.

[0091] When the image translation direction is positive and the real-time sharpness difference is less than the first preset threshold, the third image is composed of the translation region in the first image and the remaining region in the second image excluding the translation region; then the real-time sharpness difference is compared with the second preset threshold. If the real-time sharpness difference is less than the second preset threshold, then the third image is the first image.

[0092] When the image translation direction is positive and the real-time sharpness difference is less than the first preset threshold, the third image used for display consists of the translation region P1 in the first image and the remaining region S2 in the second image excluding the translation region. Then, the real-time sharpness difference is compared with the second preset threshold. If the real-time sharpness difference is greater than or equal to the second preset threshold, the real-time sharpness difference is obtained based on the overall sharpness of the first and second images acquired at the next moment, and compared with the first and second preset thresholds again. If the real-time sharpness difference is less than the second preset threshold, for example, as the endoscope gradually approaches the target tissue, the real-time sharpness difference between the near and far images gradually decreases, making the real-time sharpness difference less than the second preset threshold, then the second image is used as the third image for output display.

[0093] When the image translation direction is negative and the real-time sharpness difference is greater than the second preset threshold, the third image is composed of the translation region in the second image and the remaining region in the first image excluding the translation region; when the image translation direction is negative, the real-time sharpness difference is then compared with the first preset threshold. If the real-time sharpness difference is greater than the first preset threshold, the third image is the first image.

[0094] When the image translation direction is negative and the real-time sharpness difference is greater than the second preset threshold, the third image used for display consists of the translation region S2 in the second image and the remaining region P1 in the first image excluding the translation region. Then, the real-time sharpness difference is compared with the first preset threshold. If the real-time sharpness difference is less than or equal to the first preset threshold, the real-time sharpness difference is obtained based on the overall sharpness of the first and second images acquired at the next moment, and compared again with the first and second preset thresholds. If the real-time sharpness difference is greater than the first preset threshold, for example, as the endoscope gradually moves away from the target tissue, the real-time sharpness difference between the near and far images gradually increases, making the real-time sharpness difference greater than the first preset threshold, then the first image is used as the third image for output display.

[0095] For image fusion processing, the process of obtaining a third image for output display based on the first image, the second image, and the pixel translation direction and pixel translation amount is as follows: Figure 4 As shown, it includes:

[0096] Step S133: Obtain the region to be filled in the reference image.

[0097] The regions to be filled in the reference image are filled based on the pixel translation amounts corresponding to each pixel in the non-reference image, resulting in a third image for output display. The regions to be filled in the reference image are the less clear areas. For example, if the first image is the reference image, and there are many regions with positive local sharpness differences in the sharpness distribution difference map, then the regions with negative local sharpness differences are the less clear regions in the first image, and these regions with negative local sharpness differences can be directly used as the regions to be filled in the reference image. Alternatively, the unclear regions in the reference image can be extracted using traditional methods such as erosion, U-net, or fixed block division, thus obtaining the regions to be filled in the reference image.

[0098] Step S134: Fill the area to be filled in the reference image according to the pixel translation direction and the pixel translation amount corresponding to each pixel in the non-reference image to obtain the third image for output display.

[0099] The area to be filled in the reference image is filled according to the pixel translation amount and pixel translation direction of each pixel in the non-reference image. First, the pixel translation direction is obtained based on the reference image and the non-reference image.

[0100] For example, if the reference image corresponds to the first image and the non-reference image corresponds to the second image, then the pixels in the second image need to be filled into a portion of the first image. Since there is a spatial misalignment between the second and first images, the pixels in the second image need to be translated first, and then the translated pixels in the second image are filled into the corresponding positions in the area to be filled. At this time, the pixel translation direction is the direction of the lens corresponding to the reference image relative to the lens corresponding to the non-reference image, that is, the direction of the distant view relative to the close-up view; that is, the pixel translation direction is to the left. Conversely, the pixel translation direction is to the right. Thus, based on the pixel translation direction and the pixel translation amount of each pixel in the non-reference image, a third image for output display is obtained. In this embodiment, based on... Figure 7 and Figure 8 The third image obtained by combining the first and second images shown is as follows: Figure 10 As shown.

[0101] This embodiment first acquires real-time first and second images through a first imaging module (distant lens) and a second imaging module (close-up lens) mounted on the tip of the guide section; then, it calculates different types of sharpness based on different image processing methods used by the endoscope in actual operation, and determines the image field of view translation information related to the smooth display of the image field of view in different image processing methods; based on the obtained image field of view translation information, the image is processed accordingly, thereby reducing image jitter and / or reducing or even eliminating image misalignment without adding an additional distance sensor, so that different image processing methods can ultimately obtain image observation results that better meet the observation needs of the human eye;

[0102] In the image switching process, since the final goal is to achieve an overall switching between images from different imaging lenses, the sharpness during image switching corresponds to the overall sharpness of the first and second images. Then, the image translation direction is determined based on the real-time sharpness difference and its changes between the first and second images. The image translation amount is calculated by comparing the real-time sharpness difference with a first and a second preset threshold. Based on the overall sharpness of the first and second images, image field-of-view translation information related to smooth image display during image switching is determined. The first and second images are then translated based on this image field-of-view translation information to obtain a third image for output display. Thus, based on the real-time sharpness differences and changes between images acquired by different imaging modules, a slight overall image translation is performed, achieving a smooth transition during image switching and reducing screen jitter.

[0103] Image fusion processing involves fusing the image content corresponding to different imaging lenses. Therefore, the sharpness during image fusion processing corresponds to the local sharpness in different regions of the first and second images. By partitioning the first and second images, a sharpness distribution difference map is obtained based on the difference in local sharpness in the same partition in different images. This yields a reference image and a non-reference image. The pixel translation direction and the pixel translation amount corresponding to each pixel in the non-reference image are also obtained. Based on the local sharpness of the first and second images, the image field of view translation information related to smooth image display in image fusion processing is determined. The obtained image field of view translation information is used to translate the pixels in the non-reference image and fill them into the corresponding positions in the reference image to obtain the third image for output display. This adjustment of the position of individual pixels in the non-reference image reduces or even eliminates image misalignment.

[0104] One embodiment provides an endoscope system including a guide 10 having at least a tip configured to be inserted into an area to be examined; and an imaging assembly 11 including a first imaging module and a second imaging module, the first and second imaging modules being disposed at the tip of the guide 10. The first imaging module is used to acquire a first image of target tissue within the area to be examined, and the second imaging module is used to acquire a second image of target tissue within the area to be examined. The first imaging module corresponds to the endoscope's telephoto lens, and the second imaging module corresponds to the endoscope's near-field lens. They have different focusing ranges. In this embodiment, the focusing ranges are respectively denoted as the first focusing range and the second focusing range. That is, the first imaging module has a first focusing range, the second imaging module has a second focusing range, and the minimum value of the first focusing range is greater than the minimum value of the second focusing range. The second focusing range and the first focusing range intersect, and this intersection corresponds to the depth-of-field overlap region.

[0105] The processor 12 is used to acquire a real-time first image and a second image; acquire the direction information of the current movement of the tip of the guide relative to the target tissue, and then obtain a third image for output display; the processing flow of the processor acquiring the third image based on the direction information of the movement of the guide relative to the target tissue in this embodiment is as follows: Figure 11 As shown, the specific steps include:

[0106] Step S200: Acquire the first and second images in real time.

[0107] During the use of the endoscope, real-time image acquisition is performed through the first imaging module and the second imaging module, thereby obtaining the first image and the second image corresponding to the current moment.

[0108] Step S210: Obtain the current direction information of the movement of the tip of the guide relative to the target tissue.

[0109] The overall sharpness of the first image and the overall sharpness of the second image are obtained; the difference between the overall sharpness of the first image and the overall sharpness of the second image is calculated to obtain the real-time sharpness difference; based on the obtained real-time sharpness difference, the current direction information of the movement of the tip of the guide relative to the target tissue is obtained.

[0110] Here, the current moment is denoted as the i-th moment, where i ≥ 3. If the real-time clarity difference corresponding to the current i-th moment is less than the real-time clarity difference corresponding to the (i-1)-th moment, and the real-time clarity difference corresponding to the (i-1)-th moment is less than the real-time clarity difference corresponding to the (i-2)-th moment, then the current direction of movement of the tip of the guide relative to the target tissue is towards the target tissue. If the real-time clarity difference corresponding to the current i-th moment is greater than the real-time clarity difference corresponding to the (i-1)-th moment, and the real-time clarity difference corresponding to the (i-1)-th moment is greater than the real-time clarity difference corresponding to the (i-2)-th moment, then the current direction of movement of the tip of the guide relative to the target tissue is away from the target tissue.

[0111] Step S220: Based on the obtained direction information, determine the image field of view translation information related to the smooth display of the image field of view.

[0112] In this embodiment, the image field of view translation information related to smooth image field of view display includes the image translation direction and the image translation amount; wherein, the image translation direction is obtained based on the obtained direction information; and the image translation amount is obtained based on the real-time sharpness difference.

[0113] In addition, the image field of view translation information related to smooth image field of view display also includes pixel translation amount and pixel translation direction; wherein, the first image is divided into multiple partitions, and the sharpness corresponding to each partition in the first image is recorded as the local sharpness of each partition in the first image; the second image is divided into multiple partitions, and the sharpness corresponding to each partition in the second image is recorded as the local sharpness of each partition in the second image;

[0114] Based on the difference in local sharpness between corresponding partitions in the first and second images, the local sharpness difference for each partition is obtained, and the local sharpness differences for all partitions constitute a sharpness distribution difference map. According to the obtained sharpness distribution difference map, a reference image and a non-reference image are selected from the first and second images. Specifically, if the number of partitions with positive local sharpness differences is greater than or equal to the number of partitions with negative local sharpness differences in all partitions of the sharpness distribution difference map, then the first image is used as the reference image, and the second image is used as the non-reference image; if the number of partitions with positive local sharpness differences is less than the number of partitions with negative local sharpness differences, then the second image is used as the reference image, and the first image is used as the non-reference image.

[0115] The pixel translation direction is obtained based on the reference image and the non-reference image. When the first image is the reference image and the second image is the non-reference image, the pixel translation direction is negative; when the second image is the reference image and the first image is the non-reference image, the pixel translation direction is positive.

[0116] Then, based on the pixel values ​​of each pixel in the sharpness distribution difference map, the pixel shift amount of the corresponding pixel in the non-reference image is obtained. For example, the pixel shift amount of each pixel in the non-reference image is obtained based on the pixel values ​​of each pixel in the sharpness distribution difference map and the second preset mapping table. The second preset mapping table is used to describe the correspondence between pixel values ​​and pixel shift amounts.

[0117] Step S230: Based on the first image, the second image, and the image field of view translation information, obtain the third image for output display.

[0118] When the image translation direction is positive and the real-time sharpness difference is less than the first preset threshold, the third image is composed of the translated region in the first image and the remaining region in the second image excluding the translated region; when the image translation direction is positive and the real-time sharpness difference is less than the second preset threshold, the third image is the first image; when the image translation direction is negative and the real-time sharpness difference is greater than the second preset threshold, the third image is composed of the translated region in the second image and the remaining region in the first image excluding the translated region; when the image translation direction is negative and the real-time sharpness difference is greater than the first preset threshold, the third image is the second image.

[0119] And / or, obtain the region to be filled in the reference image, and fill the region to be filled in the reference image according to the pixel translation amount corresponding to each pixel point in the non-reference image to obtain a third image for output display.

[0120] This embodiment first acquires a real-time first image and a second image through a first imaging module (long-range lens) and a second imaging module (close-range lens) installed on the tip of the guide section; then, it acquires the current direction information of the movement of the tip of the guide section relative to the target tissue, and processes the image accordingly based on the obtained direction information. Without adding an additional distance sensor, it reduces image jitter and / or reduces or even eliminates image misalignment, so that different image processing methods can ultimately obtain image observation results that better meet the observation needs of the human eye.

[0121] In the image switching process, the direction of endoscope movement relative to the target tissue is determined based on the real-time sharpness difference and its changes between the overall sharpness of the first and second images. This yields the image translation direction during image switching. The image translation amount is calculated by comparing the real-time sharpness difference with the first and second preset thresholds, thus determining the image field of view translation information related to smooth image display during image switching. Based on the obtained image field of view translation information, the first and second images are translated to obtain the third image for output display. Thus, based on the real-time sharpness differences and their changes in the images acquired by different imaging modules, the overall image is slightly translated, thereby achieving a smooth transition during image switching and reducing screen jitter.

[0122] For image fusion processing, the first and second images are partitioned, and a sharpness distribution difference map is obtained based on the difference in local sharpness in the same partition in different images. This results in a reference image and a non-reference image. The pixel translation direction and the pixel translation amount corresponding to each pixel in the non-reference image are also obtained. Based on the local sharpness of the first and second images, the image field of view translation information related to smooth image display in image fusion processing is determined. The pixels in the non-reference image are translated according to the obtained image field of view translation information and filled into the corresponding positions in the reference image to obtain the third image for output display. Image misalignment is reduced or even eliminated by adjusting the position of individual pixels in the non-reference image.

[0123] An image generation method for an endoscope system is disclosed. The endoscope system includes a guide section and an imaging assembly. The guide section has at least a tip configured to be inserted into an area to be examined. The imaging assembly includes a first imaging module and a second imaging module, which are disposed at the tip of the guide section. The first imaging module has a first focusing range, and the second imaging module has a second focusing range. The second focusing range and the first focusing range intersect, and the minimum value of the first focusing range is greater than the minimum value of the second focusing range. The image generation method in this embodiment is as follows: Figure 12 As shown, the specific steps include:

[0124] Step S300: Acquire the first and second images in real time.

[0125] The first image is an image of the target tissue within the inspected area obtained by imaging with the first imaging module, and the second image is an image of the target tissue within the inspected area obtained by imaging with the second imaging module.

[0126] Step S320: Calculate the sharpness of the first image and the sharpness of the second image.

[0127] Step S330: Determine image field of view translation information related to smooth image field of view display based on the sharpness of the first image and the sharpness of the second image.

[0128] Step S340: Based on the first image, the second image, and the image field of view translation information, obtain the third image for output display.

[0129] It should be noted that the image generation method in this embodiment corresponds to the steps in the processing flow of a processor in an endoscope system to acquire a third image based on image clarity, and will not be repeated here.

[0130] This embodiment first acquires a real-time first image and a second image; then, it calculates different types of sharpness based on the different image processing methods used by the endoscope in actual operation, and determines the image field of view translation information related to the smooth display of the image field of view in different image processing methods; based on the obtained image field of view translation information, the image is processed accordingly, thereby reducing image jitter and / or reducing or even eliminating image misalignment without adding an additional distance sensor, so that different image processing methods can ultimately obtain image observation results that better meet the observation needs of the human eye;

[0131] In the image switching process, since the final goal is to achieve an overall switching between images from different imaging lenses, the sharpness during image switching corresponds to the overall sharpness of the first and second images. Then, the image translation direction is determined based on the real-time sharpness difference and its changes between the first and second images. The image translation amount is calculated by comparing the real-time sharpness difference with a first and a second preset threshold. Based on the overall sharpness of the first and second images, image field-of-view translation information related to smooth image display during image switching is determined. The first and second images are then translated based on this image field-of-view translation information to obtain a third image for output display. Thus, based on the real-time sharpness differences and changes between images acquired by different imaging modules, a slight overall image translation is performed, achieving a smooth transition during image switching and reducing screen jitter.

[0132] Image fusion processing involves fusing the image content corresponding to different imaging lenses. Therefore, the sharpness during image fusion processing corresponds to the local sharpness in different regions of the first and second images. By partitioning the first and second images, a sharpness distribution difference map is obtained based on the difference in local sharpness in the same partition in different images. This yields a reference image and a non-reference image. The pixel translation direction and the pixel translation amount corresponding to each pixel in the non-reference image are also obtained. Based on the local sharpness of the first and second images, the image field of view translation information related to smooth image display in image fusion processing is determined. The obtained image field of view translation information is used to translate the pixels in the non-reference image and fill them into the corresponding positions in the reference image to obtain the third image for output display. This adjustment of the position of individual pixels in the non-reference image reduces or even eliminates image misalignment.

[0133] An image generation method for an endoscope system includes a guide section and an imaging assembly. The guide section has at least a tip configured to be inserted into an area to be examined. The imaging assembly includes a first imaging module and a second imaging module disposed at the tip of the guide section. The first imaging module has a first focusing range, and the second imaging module has a second focusing range. The second focusing range and the first focusing range intersect, and the minimum value of the first focusing range is greater than the minimum value of the second focusing range. The image generation method in this embodiment is as follows: Figure 13 As shown, the specific steps include:

[0134] Step S400: Acquire the first and second images in real time.

[0135] The first image is an image of the target tissue within the inspected area obtained by imaging with the first imaging module, and the second image is an image of the target tissue within the inspected area obtained by imaging with the second imaging module.

[0136] Step S410: Obtain the current direction information of the movement of the tip of the guide relative to the target tissue.

[0137] Step S420: Based on the obtained direction information, determine the image field of view translation information related to the smooth display of the image field of view.

[0138] Step S430: Based on the first image, the second image, and the image field of view translation information, obtain the third image for output display.

[0139] It should be noted that the image generation method in this embodiment corresponds to the steps in the processing flow of a processor in an endoscope system that acquires a third image based on the direction information of the movement of the guide relative to the target tissue, and will not be repeated here.

[0140] This embodiment first acquires a real-time first image and a second image; then it acquires the current direction information of the movement of the tip of the guide relative to the target tissue, and processes the image accordingly based on the obtained direction information. Without adding an additional distance sensor, it reduces image jitter and / or reduces or even eliminates image misalignment, so that different image processing methods can ultimately obtain image observation results that better meet the observation needs of the human eye.

[0141] In the image switching process, the direction of endoscope movement relative to the target tissue is determined based on the real-time sharpness difference and its changes between the overall sharpness of the first and second images. This yields the image translation direction during image switching. The image translation amount is calculated by comparing the real-time sharpness difference with the first and second preset thresholds, thus determining the image field of view translation information related to smooth image display during image switching. Based on the obtained image field of view translation information, the first and second images are translated to obtain the third image for output display. Thus, based on the real-time sharpness differences and their changes in the images acquired by different imaging modules, the overall image is slightly translated, thereby achieving a smooth transition during image switching and reducing screen jitter.

[0142] For image fusion processing, the first and second images are partitioned, and a sharpness distribution difference map is obtained based on the difference in local sharpness in the same partition in different images. This results in a reference image and a non-reference image. The pixel translation direction and the pixel translation amount corresponding to each pixel in the non-reference image are also obtained. Based on the local sharpness of the first and second images, the image field of view translation information related to smooth image display in image fusion processing is determined. The pixels in the non-reference image are translated according to the obtained image field of view translation information and filled into the corresponding positions in the reference image to obtain the third image for output display. Image misalignment is reduced or even eliminated by adjusting the position of individual pixels in the non-reference image.

[0143] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0144] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. An endoscope system, characterized in that, include: The guide portion has at least a tip configured to penetrate into the area being inspected; An imaging assembly, comprising a first imaging module and a second imaging module, wherein the first imaging module and the second imaging module are disposed at the tip end of the guide portion, the first imaging module being used to acquire a first image of the target tissue within the inspected area, and the second imaging module being used to acquire a second image of the target tissue within the inspected area. And a processor, used for: Acquire the first and second images in real time; Calculate the sharpness of the first image and the sharpness of the second image; Based on the sharpness of the first image and the sharpness of the second image, determine the image field of view translation information related to the smooth display of the image field of view; A third image is obtained for output display based on the first image, the second image, and the image field of view translation information.

2. The system as described in claim 1, characterized in that, The first imaging module has a first focus range, the second imaging module has a second focus range, the second focus range and the first focus range have an intersection, and the minimum value of the first focus range is greater than the minimum value of the second focus range.

3. The system as described in claim 1, characterized in that, The calculation of the sharpness of the first image and the sharpness of the second image includes: The sharpness includes overall sharpness and / or local sharpness, wherein the first image is divided into multiple partitions, and the sharpness corresponding to each partition in the first image is recorded as the local sharpness of each partition in the first image; the second image is divided into multiple partitions, and the sharpness corresponding to each partition in the second image is recorded as the local sharpness of each partition in the second image; the number of partitions corresponding to the first image is the same as the number of partitions corresponding to the second image.

4. The system as described in claim 1, characterized in that, The step of determining image field-of-view translation information related to smooth image display based on the sharpness of the first image and the sharpness of the second image includes: The image field of view translation information related to smooth image display includes image translation direction and image translation amount, and / or pixel translation amount and pixel translation direction; wherein, the difference between the overall sharpness of the first image and the overall sharpness of the second image is calculated to obtain the real-time sharpness difference; the image translation amount is obtained based on the obtained real-time sharpness difference; the image translation direction is obtained based on the current real-time sharpness difference and the real-time sharpness difference at the previous moment, and the image translation direction includes positive and negative directions; Based on the difference in local sharpness between corresponding partitions in the first image and the second image, the local sharpness difference corresponding to each partition is obtained, and the local sharpness differences corresponding to all partitions constitute a sharpness distribution difference map; based on the obtained sharpness distribution difference map, a reference image and a non-reference image are selected from the first image and the second image; the pixel translation direction is obtained based on the reference image and the non-reference image; based on the pixel value of each pixel in the sharpness distribution difference map, the pixel translation amount of the corresponding pixel in the non-reference image is obtained.

5. The system as described in claim 1, characterized in that, The step of obtaining a third image for output display based on the first image, the second image, and the image field-of-view translation information includes: The first image and the second image are translated according to the image field of view translation information to obtain the translation area corresponding to the first image and the second image. A third image for output display is obtained according to the translation area corresponding to the first image and the second image. And / or, obtain the region to be filled in the reference image, and fill the region to be filled in the reference image according to the pixel translation amount and pixel translation direction corresponding to each pixel point in the non-reference image, to obtain the third image for output display.

6. The system as described in claim 5, characterized in that, The step of obtaining the third image for output display based on the translation regions corresponding to the first and second images includes: When the image translation direction is positive and the real-time sharpness difference is less than the first preset threshold, the third image is composed of the translation region in the first image and the remaining region in the second image excluding the translation region; when the image translation direction is positive and the real-time sharpness difference is less than the second preset threshold, the third image is the first image. When the image translation direction is negative and the real-time sharpness difference is greater than the second preset threshold, the third image is composed of the translation region in the second image and the remaining region in the first image excluding the translation region; when the image translation direction is negative and the real-time sharpness difference is greater than the first preset threshold, the third image is the second image.

7. An endoscope system, characterized in that, include: The guide portion has at least a tip configured to penetrate into the area being inspected; An imaging assembly, comprising a first imaging module and a second imaging module, wherein the first imaging module and the second imaging module are disposed at the tip end of the guide portion, the first imaging module being used to acquire a first image of the target tissue within the inspected area, and the second imaging module being used to acquire a second image of the target tissue within the inspected area. And a processor, used for: Acquire the first and second images in real time; Obtain the current direction information of the movement of the tip of the guide relative to the target tissue; Based on the directional information, determine the image field of view translation information related to the smooth display of the image field of view; A third image is obtained for output display based on the first image, the second image, and the image field of view translation information.

8. The system as described in claim 7, characterized in that, The step of obtaining the current direction information of the movement of the tip of the guide relative to the target tissue includes: Obtain the overall sharpness of the first image and the overall sharpness of the second image; calculate the difference between the overall sharpness of the first image and the overall sharpness of the second image to obtain the current real-time sharpness difference; If the real-time clarity difference at the current i-th moment is less than the real-time clarity difference at the i-1-th moment, and the real-time clarity difference at the i-1-th moment is less than the real-time clarity difference at the i-2-th moment, then the current direction of the tip of the guide relative to the target tissue is towards the target tissue. If the real-time clarity difference at the current i-th moment is greater than the real-time clarity difference at the (i-1)-th moment, and the real-time clarity difference at the (i-1)-th moment is greater than the real-time clarity difference at the (i-2)-th moment, then the current direction of movement of the tip of the guide relative to the target tissue is away from the target tissue.

9. An image generation method for an endoscope system, the endoscope system including a guide portion and an imaging assembly, the guide portion having at least a tip portion configured to be inserted into an area to be examined, the imaging assembly including a first imaging module and a second imaging module, the first imaging module and the second imaging module being disposed at the tip portion of the guide portion. The first imaging module has a first focus range, the second imaging module has a second focus range, the second focus range and the first focus range intersect, and the minimum value of the first focus range is greater than the minimum value of the second focus range; characterized in that, The image generation method includes: Acquire a first image and a second image in real time, wherein the first image is an image of the target tissue within the inspected area obtained by the first imaging module imaging, and the second image is an image of the target tissue within the inspected area obtained by the second imaging module imaging; Calculate the sharpness of the first image and the sharpness of the second image; Based on the sharpness of the first image and the sharpness of the second image, determine the image field of view translation information related to the smooth display of the image field of view; A third image is obtained for output display based on the first image, the second image, and the image field of view translation information.

10. An image generation method for an endoscope system, the endoscope system comprising a guide portion and an imaging assembly, the guide portion having at least a tip configured to penetrate into an area to be examined, the imaging assembly comprising a first imaging module and a second imaging module disposed at the tip of the guide portion, the first imaging module having a first focusing range, the second imaging module having a second focusing range, the second focusing range and the first focusing range intersecting, and the minimum value of the first focusing range being greater than the minimum value of the second focusing range, characterized in that... The image generation method includes: Acquire a first image and a second image in real time, wherein the first image is an image of the target tissue within the inspected area obtained by the first imaging module imaging, and the second image is an image of the target tissue within the inspected area obtained by the second imaging module imaging; Obtain the current direction information of the movement of the tip of the guide relative to the target tissue; Based on the directional information, determine the image field of view translation information related to the smooth display of the image field of view; A third image is obtained for output display based on the first image, the second image, and the image field of view translation information.

Images