Image processing system and image processing method

Abstract

The application discloses an image processing system and an image processing method. The image processing system comprises an image generating device and an image processing device; the image processing device is used for acquiring n groups of to-be-processed image groups; for any group of to-be-processed image groups taken at an intermediate time, interpolation processing is performed based on two adjacent to-be-processed image groups of a current group of to-be-processed image groups, so that a group of interpolation images corresponding to the current group of to-be-processed image groups is obtained; and each to-be-processed image group and each interpolation image group are subjected to image fusion, so that a target fusion image of the photographed object is obtained. The technical scheme disclosed by the application realizes reduction of system imaging delay, thereby improving the display effect of an endoscope video.

Description

Technical Field

[0001] This invention relates to the field of endoscopic imaging technology, and more particularly to an image processing system and an image processing method. Background Technology

[0002] Traditional techniques acquire images of lesions by emitting visible and near-infrared light towards the lesion site. To observe tissue metabolism during surgery and avoid cutting blood vessels, it is necessary to simultaneously acquire metabolic and vascular morphology information of the lesion site. However, traditional techniques cannot simultaneously include both metabolic and vascular morphology information in the acquired images, requiring the overlay of visible light images containing vascular morphology information and infrared light images containing metabolic information.

[0003] However, existing technologies typically employ two methods to overlay visible light and near-infrared images acquired by an endoscope: The first method uses multiple CMOS sensors to simultaneously capture white light and near-infrared images, then fuses these images to obtain the overlaid image. This method offers high performance and can achieve high frame rate video streams, but it requires a complex optical system to separate the white light and near-infrared light, and at least two CMOS sensors to complete the imaging and image registration. This results in a large and costly optical imaging module. The second method uses a single CMOS sensor, with the light source generating white light and near-infrared light at intervals. After acquiring two adjacent frames of white light and near-infrared light images, the overlaid image is obtained through image fusion and output. This method features a simple, small, and low-cost optical imaging system, but its drawbacks are also significant: it requires multiple frames to output a single frame, increasing the imaging system latency and resulting in poor endoscopic video display quality. Summary of the Invention

[0004] This invention provides an image processing system and method that immediately fuses and displays the local images upon acquisition, thereby solving the problem of poor output endoscopic video quality due to imaging delay in existing technologies. This reduces system imaging delay and improves the display effect of endoscopic video.

[0005] In a first aspect, embodiments of the present invention provide an image processing system, the system comprising: an image generating device and an image processing device; wherein...

[0006] The image generation device is used to sequentially capture images of different local positions of the subject using two light sources, generating n sets of images to be processed of the subject; wherein, a set of images to be processed is the result of a single continuous capture by any one light source, and the single continuous capture result includes at least one frame of the image to be processed, with a preset interval between each frame of the image to be processed; odd-numbered sets of images to be processed are the capture results of the first light source, and even-numbered sets of images to be processed are the capture results of the second light source;

[0007] The image processing device is used to acquire n groups of images to be processed; for any group of images to be processed captured at an intermediate time, interpolation processing is performed based on two adjacent groups of images to be processed in the current group to obtain an interpolated image group corresponding to the current group to be processed; and each group of images to be processed and each interpolated image group are image fused to obtain a target fused image of the captured object.

[0008] Optionally, the image generation device includes a light source controller, a first light source, a second light source, and a light capturing device;

[0009] The light source controller is used to project light with different center wavelengths in adjacent groups and light with the same center wavelength in groups separated by a distance.

[0010] The first light source is used to project light of a first center wavelength onto the object being photographed under the control of the light source controller;

[0011] The second light source is used to project light of a second center wavelength onto the object being photographed under the control of the light source controller;

[0012] The light-capturing device is used to capture light on the subject and generate an image to be processed.

[0013] Optionally, the image processing device includes a first memory, a second memory, and a processor; the first memory includes k first logic units, and the second memory includes k second logic units; k is a positive integer greater than a preset threshold.

[0014] The first logic unit is used to store the image capture result of the first light source generated by the image generation device;

[0015] The second logic unit is used to store the image capture result of the second light source generated by the image generation device;

[0016] The processor is configured to perform the following steps in the image processing device: acquiring multiple frames of images to be processed, generating multiple groups of images to be processed, and performing image fusion on each group of images to be processed and the interpolated image group to generate a target fused image corresponding to the subject being photographed.

[0017] Optionally, the processor includes a module for generating a group of images to be processed, a module for obtaining an interpolated group of images, and a module for obtaining a target fused image; wherein,

[0018] The image acquisition module is used to acquire n sets of images to be processed;

[0019] The interpolated image group acquisition module is used to perform interpolation processing on any group of images to be processed captured at an intermediate time, based on two adjacent groups of images to be processed in the current group, to obtain the interpolated image group corresponding to the current group of images to be processed.

[0020] The target fusion image acquisition module is used to fuse the images to be processed and the interpolated images to obtain the target fusion image of the photographed object.

[0021] Optionally, the interpolated image group acquisition module includes a pixel mean data acquisition unit and an interpolated image group acquisition unit; wherein,

[0022] The pixel mean data acquisition unit is used to determine the pixel mean data of two corresponding frames of images to be processed in two adjacent groups of images to be processed.

[0023] The interpolated image group obtaining unit is used to perform data interpolation processing on the pixel mean data to obtain the interpolated images of the corresponding two frames of images to be processed, and to obtain the interpolated image group of the current group of images to be processed based on each of the interpolated images.

[0024] Optionally, the target fused image acquisition module includes a local fused image acquisition unit and a target fused image acquisition unit; wherein,

[0025] The local fusion image acquisition unit is used to perform image fusion on the current group of images to be processed and the interpolated image group corresponding to the current group of images to be processed for any group of images to be processed, so as to obtain the local fusion image group of the current group of images to be processed.

[0026] The target fusion image acquisition unit is used to determine the encoding order of each group of local fusion image groups, and to stitch the local fusion image groups together based on the encoding order to obtain the target fusion image of the photographed object.

[0027] Optionally, if the image group to be processed is the first frame of the image group to be processed, then each frame of the image group to be processed is output before the target fused image is output.

[0028] Optionally, the image to be processed includes m rows / columns of pixel data, where m is a positive integer less than a preset threshold.

[0029] Optionally, the image processing system is an endoscopic imaging system, and correspondingly, the first light source in the image processing system is a visible light source and the second light source is a near-infrared light source, or the first light source is a near-infrared light source and the second light source is a visible light source.

[0030] Secondly, embodiments of the present invention also provide an image processing method, the method comprising:

[0031] Obtain n sets of images to be processed; wherein, each set of images to be processed is a continuous shooting result of any light source, and a continuous shooting result includes at least one frame of images to be processed, and each frame of images to be processed is shot at a preset interval time. Odd-numbered sets of images to be processed are shooting results of the first light source, and even-numbered sets of images to be processed are shooting results of the second light source.

[0032] For any group of images to be processed captured at an intermediate time, interpolation is performed based on the two adjacent groups of images to be processed in the current group to obtain the interpolated image group corresponding to the current group to be processed.

[0033] The images to be processed and the interpolated images are fused together to obtain the target fused image of the photographed object.

[0034] Thirdly, embodiments of the present invention also provide an electronic device, comprising:

[0035] At least one processor; and

[0036] A memory communicatively connected to the at least one processor; wherein,

[0037] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image processing method according to any embodiment of the present invention.

[0038] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions that are used to cause a processor to execute the image processing method described in any embodiment of the present invention.

[0039] The image processing system provided in this embodiment of the invention includes an image generation device 1 and an image processing device 2. The image processing device 2 immediately performs image fusion and displays the local image when it acquires the local image, so as to solve the problem that the output endoscope video effect is poor due to the imaging delay in the prior art, thereby reducing the system imaging delay and improving the display effect of the endoscope video.

[0040] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of the structure of an image processing system according to an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of the structure of another image processing system provided according to an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the structure of another image processing system provided according to an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the structure of an image processing apparatus according to an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of the structure of another image processing system provided according to an embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram of the structure of another image processing system provided according to an embodiment of the present invention;

[0048] Figure 7 This is a schematic diagram of an image processing method provided according to an embodiment of the present invention;

[0049] Figure 8 This is a flowchart of another image processing method provided according to an embodiment of the present invention;

[0050] Figure 9 This is a schematic diagram of the structure of an electronic device that implements the image processing method of the present invention. Detailed Implementation

[0051] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0052] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0053] This invention provides an image processing system. Figure 1 This is a schematic diagram of an image processing system provided in an embodiment of the present invention. This embodiment is applicable to processing images captured from different light sources to generate fused images.

[0054] In existing technologies, to observe tissue metabolism and morphological location during surgery, it is necessary to simultaneously acquire metabolic and morphological information of the lesion site. Since images acquired using existing technologies cannot simultaneously include both metabolic and morphological information of the lesion site, image fusion processing is required to obtain a fused image from the acquired images containing these two types of information. Furthermore, the fused image is output sequentially to enable simultaneous observation of tissue metabolism and morphological location during surgery.

[0055] Currently, there are generally two methods for obtaining fused images. The first method uses multiple CMOS sensors to simultaneously capture white light and near-infrared images, then fuses these images to obtain the superimposed image. This method offers good performance and can obtain high frame rate video streams, but it requires a complex optical system to separate white light and near-infrared light, and at least two CMOS sensors to complete the imaging and image registration. This results in a large and costly optical imaging module. The second method uses a single CMOS sensor. The light source generates white light and near-infrared light at intervals. After acquiring two adjacent frames of white light and near-infrared light images, the superimposed image is obtained through image fusion and output. This method has a simple optical imaging system structure, small size, and low cost, but its disadvantages are also obvious: it requires multiple frames to output a single frame, increasing the imaging system latency and resulting in poor display quality of the endoscopic video.

[0056] Based on the above-mentioned technical problems, the technical solution of this invention provides an image processing system that immediately performs image fusion and displays the local images upon acquisition, thereby solving the problem of poor output endoscopic video quality due to imaging delay in the prior art, reducing system imaging delay, and thus improving the display effect of endoscopic video.

[0057] like Figure 1 As shown, the image processing system includes: an image generating device 1 and an image processing device 2; wherein,

[0058] The image generation device 1 is used to sequentially capture images of different local positions of the subject using two light sources, generating n sets of images of different local positions of the subject to be processed; wherein, a set of images to be processed is the result of a single continuous capture by any one light source, the single continuous capture result includes at least one frame of the image to be processed, each frame of the image to be processed is captured at a preset interval, odd-numbered sets of images to be processed are the capture results of the first light source, and even-numbered sets of images to be processed are the capture results of the second light source;

[0059] The image processing device 2 is used to acquire n groups of images to be processed; for any group of images to be processed captured at an intermediate time, interpolation processing is performed based on two adjacent groups of images to be processed in the current group to obtain an interpolated image group corresponding to the current group to be processed; and each group of images to be processed and each interpolated image group are image fused to obtain a target fused image of the captured object.

[0060] The image processing system can be understood as a system that uses two light sources to capture images of different local locations of the subject in a cyclical manner, processes each image to obtain a locally fused image, and then stitches these images together to obtain the target fused image of the subject. This system comprehensively processes and obtains tissue morphology and metabolic information of the subject within a preset time period. The subject can be a human or a lesion site in an animal. In practical applications, tissue morphology information can be understood as the visually observable shape and location of the lesion site; morphological information reflects the tissue morphology and location of the lesion site. Furthermore, to clearly understand the tissue metabolism of the lesion site, it is also necessary to simultaneously acquire the tissue metabolic information of the lesion site.

[0061] In this embodiment, the image processing system includes an image generation device 1 and an image processing device 2.

[0062] Specifically, the subject is divided into columns, rows, or blocks to obtain multiple local positions. The two light sources in the image generation device 1 are used in a loop to sequentially capture images of different local positions of the subject, generating n sets of images of different local positions of the subject to be processed.

[0063] In the actual process of photographing different local positions of the subject, each local position is photographed sequentially according to the division results to ensure that the entire subject is completely photographed in the division order. For example, the subject is divided into rows, resulting in multiple rows of local positions. During the photographing process, a first light source is used to photograph the first row of local positions, and a second light source is used to photograph the second row of local positions; the first light source is used to photograph the third row of local positions in a cyclical manner, and the second light source is used to photograph the fourth row of local positions; optionally, the above cyclical order is followed to photograph each row of local positions of the subject until all local positions of the subject have been photographed, resulting in n sets of images of the subject to be processed.

[0064] In this embodiment, a set of images to be processed is the result of a continuous shooting from any light source. A continuous shooting result includes at least one frame of images to be processed. Each frame of images to be processed is shot at a preset interval. Odd-numbered sets of images to be processed are the shooting results from the first light source, and even-numbered sets of images to be processed are the shooting results from the second light source.

[0065] Specifically, images are taken at different local locations of the subject to obtain individual frames of images to be processed at each local location. These frames are then stitched together to obtain a single image representing the complete subject. In other words, the individual frames of images at different local locations can be understood as local image data of different local locations within a single image representing the subject.

[0066] The advantage of this embodiment, which involves capturing and processing a portion of the subject, is its faster processing speed. Compared to capturing and processing an entire image, this embodiment has a shorter image processing delay, resulting in better image output and display in practical applications.

[0067] Specifically, when each group of images to be processed contains one frame of image to be processed, the corresponding process of generating n groups of images to be processed from different local positions of the subject by using two light sources in a loop can be as follows: using the first light source to shoot the first local position for a preset interval time to obtain the first frame of image to be processed, which is the first group of images to be processed; using the second light source to shoot the second local position for a preset interval time to obtain the second frame of image to be processed, which is the second group of images to be processed; using the first light source again to shoot the third local position to obtain the third group of images to be processed; using the second light source again to shoot the fourth local position to obtain the fourth group of images to be processed, and continuing to shoot in the above loop until all local positions of the subject have been shot, resulting in multiple groups of images to be processed.

[0068] Optionally, when each group of images to be processed contains multiple frames of images to be processed, the corresponding process of using two types of light sources to capture images of n different local positions of the subject can also be as follows: using a first light source to continuously capture images of multiple preset local positions at multiple preset intervals, obtaining multiple frames of images to be processed corresponding to multiple different local positions, and using these multiple frames of images to be processed as the first group of images to be processed; then using a second light source to continue capturing images of multiple preset local positions in the remaining local positions at multiple preset intervals, obtaining multiple frames of images to be processed corresponding to multiple different local positions, and using these multiple frames of images to be processed as the second group of images to be processed; then using the first light source and the second light source again to continuously capture images of the remaining local positions until all local positions have been captured, obtaining multiple groups of images to be processed corresponding to each local position.

[0069] It should be noted that during the imaging process using both the first and second light sources, the number of image frames captured by the two light sources can differ in each cycle. The effect is that the fused image obtained after image interpolation based on the different number of frames can achieve a balance in the quality of the fused image from the different light sources. The reason for this balance is that in practical applications, the requirement for metabolic information during surgery is simply that it be visible. Furthermore, the display of metabolic information does not require particularly high resolution due to the staining and absorption characteristics of the contrast agent itself. However, morphological information needs to be clearly and carefully observed during surgery to avoid accidentally damaging other tissues, thus requiring higher resolution. The display of morphological information mainly depends on the light source used for imaging. Therefore, in this case, processing images with more frames containing morphological information and fewer frames containing metabolic information can yield a fused image with balanced image quality, thereby improving the image display effect during surgery.

[0070] Based on the above implementation method, in this embodiment, a frame of image to be processed includes m rows / columns of pixel data, where m is a positive integer less than a preset threshold. Specifically, the pixel data contained in the image to be processed varies depending on the way the subject is divided. For example, if the subject is divided into 1078 local locations, then for a frame of image to be processed with 1078 rows / columns, the image to be processed has 1 row / column; if the subject is divided into 539 local locations, then for a frame of image to be processed with 1078 rows / columns, the image to be processed has 2 rows / columns.

[0071] It should also be noted that the first light source can be a photographic light source that emits excitation light in a first preset wavelength band, designed to obtain tissue morphology information; the second light source can be a photographic light source that emits excitation light in a second preset wavelength band, designed to obtain tissue metabolic information.

[0072] In practical applications, to obtain the tissue morphology information of the subject being photographed, a first light source can emit light of a first preset wavelength band, that is, illuminate the subject with light of the first preset wavelength band, thereby capturing the light on the subject for imaging, thus obtaining the directly observable morphological and positional information of the subject. Optionally, in this embodiment, the wavelength band of the first light source can be the visible light band of 400-650nm.

[0073] Furthermore, since the tissue metabolic information of the subject needs to be displayed with the help of a developing agent, the developing agent needs to be injected into the subject beforehand. Then, a second light source is used to emit excitation light of a second preset wavelength. The excitation light of the second preset wavelength is the excitation light of the developing agent corresponding to the wavelength. Further, the subject is irradiated based on the excitation light of the second preset wavelength. This excitation light can excite the developing agent in the subject, so that the developing agent emits light carrying information under the excitation light. By capturing the light carrying information and forming an image, the tissue metabolic information of the subject can be obtained. Then, the physiological and pathological information of the subject can be obtained based on the tissue metabolic information.

[0074] It should be noted that different types of developers correspond to different excitation wavelengths. For example, when the developer is indocyanine green, the corresponding excitation wavelength is 780nm-811nm. Optionally, the developer can also be other chemical substances. For example, when using Rhodamine series dyes as developers, the corresponding excitation wavelength is 520-600nm; when using cyanine dyes (Cy series) as developers, the corresponding excitation wavelength is 550-780nm; and when using Alexa Fluor series dyes as developers, the corresponding excitation wavelength is 340-680nm. Of course, the above examples are only illustrative of the technical solution of this embodiment and should not be construed as limiting the technical solution of this embodiment. This embodiment does not limit the selection of the developer or the selection of its excitation wavelength.

[0075] It should also be noted that the image generation device 1 is connected to the image processing device 2, and is used to transmit the n frames of images to be processed generated by the image generation device 1 to the image processing device 2, so that the image processing device 2 can group the above-mentioned n frames of images to obtain multiple image groups to be processed, and perform image fusion processing on each image group to obtain the target fused image. Optionally, the two devices can be electrically connected or communication connected. In other words, the connection method between the two devices only needs to realize the image transmission function. This embodiment does not limit the specific connection method.

[0076] Based on the above implementation method, the image generation device 1, after obtaining n frames of images to be processed, also transmits the obtained n frames of images to be processed to the image processing device 2, so that the image processing device 2 performs image processing on the images to be processed and obtains the processed target fused image.

[0077] Specifically, the image processing device 2 receives n frames of images to be processed transmitted by the image generation device 1, and generates multiple groups of images to be processed based on the received n frames of images to be processed; for any group of images to be processed captured at an intermediate time, interpolation processing is performed based on two adjacent groups of images to be processed in the current group of images to be processed to obtain an interpolated image group corresponding to the current group of images to be processed; the images to be processed and the interpolated image groups are fused to obtain a target fused image of the captured object.

[0078] In existing technologies, to observe tissue metabolism during surgery and avoid cutting blood vessels, it is necessary to simultaneously acquire metabolic information and vascular morphology information of the lesion site. The traditional processing method involves acquiring two adjacent frames—a first frame containing morphological information and a second frame containing metabolic information—and then fusing them to obtain a fused result. However, when processing and outputting the two complete images, there is a significant delay between the image output time and the image acquisition time. During this delay, the subject may move, and if the surgeon operates on the subject based on the output image, they may accidentally operate on other tissues, such as blood vessels, leading to surgical risks. This embodiment does not process the entire image corresponding to the complete subject; instead, it captures, processes, and outputs a portion of the image data from a single image. Specifically, the technical solution of this embodiment can be exemplarily described as acquiring a first row of pixel data, processing it, and then outputting the first row of pixel data. Simultaneously, a second row of pixel data can be acquired and processed, allowing the output of the second row of pixel data to follow immediately after the first row is output. Since processing row data takes less time than processing the entire image, the technical solution in this embodiment can effectively reduce image output latency. Furthermore, to reduce image ghosting during the processing of different frames of images, this embodiment also performs interpolation followed by fusion, thereby reducing image ghosting and ensuring clear image output while minimizing latency.

[0079] The image processing system provided in this embodiment of the invention includes an image generation device 1 and an image processing device 2. The image processing device 2 immediately performs image fusion and displays the local image when it acquires the local image, so as to solve the problem that the output endoscope video effect is poor due to the imaging delay in the prior art, thereby reducing the system imaging delay and improving the display effect of the endoscope video.

[0080] Figure 2 This is a schematic diagram of another image processing system provided in an embodiment of the present invention. See also... Figure 2 Based on the above embodiments, optionally, the image generation device 1 includes a light source controller 11, a first light source 12, a second light source 13, and a light capturing device 14; the light source controller 11 is used to project light of different wavelengths in adjacent groups and light of the same wavelength in alternate groups; the first light source 12 is used to project light of the first wavelength onto the subject under the control of the light source controller 11; the second light source 13 is used to project light of the second wavelength onto the subject under the control of the light source controller 11; the light capturing device 14 is used to capture and photograph the light on the subject and generate an image to be processed.

[0081] For example, a first light source is used to continuously photograph multiple local positions of the subject, resulting in a first set of photographic results; a second light source is used to continuously photograph multiple local positions of the remaining local positions of the subject, resulting in a second set of photographic results; the first light source is used again to photograph multiple local positions of the remaining local positions of the subject, resulting in a third set of photographic results. Here, projecting light of different wavelengths in adjacent sets can be understood as the light sources corresponding to the first and second sets of photographic results being different, i.e., the projection wavelengths of the light are different. Projecting light of the same wavelength in alternate sets can be understood as the light sources corresponding to the first and third sets of photographic results being the same, i.e., the projection wavelengths of the light are the same.

[0082] In this embodiment, the light source controller 11 is connected to the first light source 12 and the second light source 13 respectively, and is used to project light of different wavelengths in adjacent groups and light of the same wavelength in groups separated by distance. The light source controller 11 can be understood as a control switch for the light source, that is, it can be used to control whether the current light source emits light of a preset wavelength in the current group.

[0083] Specifically, the first light source 12 can be controlled to emit light of a first preset wavelength in odd-numbered arrays, and the first light source 12 responds to the control command of the light source controller 11 to emit light of the first preset wavelength in odd-numbered arrays; the second light source 13 can be controlled to emit light of a second preset wavelength in even-numbered arrays, and the second light source 13 responds to the control command of the light source controller 11 to emit light of the second preset wavelength in even-numbered arrays.

[0084] For example, the light source controller 11 controls the power supply of the first light source 12 to be turned on in odd numbers. When the first light source 12 detects that it is turned on, it emits light of a first preset wavelength and illuminates the subject with the light of the first preset wavelength. In even numbers, it controls the power supply of the second light source 13 to be turned on. When the second light source 13 detects that it is turned on, it emits light of a second preset wavelength and illuminates the subject with the light of the second preset wavelength.

[0085] Specifically, the light-capturing device 14 captures and photographs the light on the subject and generates an image to be processed. Optionally, based on the illumination frequencies of the two light sources on the subject, the light-capturing device captures light in the first preset wavelength band in odd-numbered arrays to obtain the photographing result of the first light source 12; and captures light in the second preset wavelength band in even-numbered arrays to obtain the photographing result of the second light source 13.

[0086] For example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor can be used as a light-capturing device 14 to capture light on the subject and form an image, resulting in n frames of images to be processed.

[0087] Based on the above embodiments, the image generation device 1 in this embodiment may further include a light source, a first filter, a second filter, a filter controller, and a light capturing device 14. Specifically, the light source emits light in a third preset wavelength band; wherein the third preset wavelength band includes the aforementioned first preset wavelength band and second preset wavelength band. Specifically, the light source continuously emits light in the third preset wavelength band, and the filter controller controls the first filter to operate in odd-numbered arrays, so that it can only transmit light in the first preset wavelength band to the subject; and controls the second filter to operate in even-numbered arrays, so that it can only transmit light in the second preset wavelength band to the subject. Then, the light capturing device 14 captures the light on the subject to form an image, obtaining n frames of images to be processed.

[0088] It should be noted that the two methods of obtaining the images to be processed described above in the embodiments of the present invention are exemplary descriptions of the technical solutions of this embodiment. The technical solutions of this embodiment can also obtain the above n frames of images to be processed through other existing image acquisition methods. There are no limitations on the acquisition method and acquisition device.

[0089] Figure 3 This is a schematic diagram of another image processing system provided in an embodiment of the present invention. See also... Figure 3 Based on the above embodiments, optionally, the image processing device 2 includes a first memory 21, a second memory 22, and a processor 23; the first memory 21 includes k first logic units 211, and the second memory 22 includes k second logic units 22121; k is a positive integer greater than a preset threshold; the first logic units 211 are used to store the shooting results of the first light source 12 generated by the image generating device 1; the second logic units 221 are used to store the shooting results of the second light source 13 generated by the image generating device 1; the processor 23 is used to execute the steps in the image processing device 2 of acquiring multiple groups of images to be processed, and performing image fusion on the multiple groups of images to be processed and the interpolated image groups to generate a target fused image corresponding to the shooting object.

[0090] In this embodiment of the invention, the first memory 21 and the second memory 22 are respectively connected to the light-capturing device 14 and are used to store the images to be processed generated by the light-capturing device 14. Specifically, the first memory 21 is used to receive and store the shooting results of the first light source 12 in odd-numbered arrays; the second memory 22 is used to receive and store the shooting results of the second light source 13 in even-numbered arrays.

[0091] In practical applications, both the first memory 21 and the second memory 22 can be non-volatile memories. In the technical solution of this embodiment, the two adjacent image groups to be processed in the current group of images to be processed need to be interpolated, and then the interpolated image group and the current group of images to be processed are fused. Therefore, for any frame of image to be processed, multiple frames need to be repeatedly stored in the memory so that the current frame of image to be processed can be reused for image fusion processing. Based on this, in this embodiment, the first memory 21 can store each frame of the odd-numbered image group of images to be processed from the first light source 12 in k logical units, and correspondingly, the second memory 22 can store each frame of the even-numbered image group of images to be processed from the two light sources 13 in k logical units. Optionally, in this embodiment, k can be 3.

[0092] See examples Figure 4 , Figure 4 This is a schematic diagram of the structure of the image processing apparatus 2 provided in an embodiment of the present invention. Image interpolation and image fusion processing can be performed based on the functional modules in the image processing apparatus 2 to obtain the target fused image of each group of images to be processed. For example, in the following exemplary description, each group of images to be processed includes one frame of image to be processed. Specifically, at time T0, the shooting result of the first light source 12 is acquired, i.e., the first frame of image to be processed, and the first frame of image to be processed is repeatedly stored in three first logic units 211, for example... Figure 6 In FIFO A, FIFO B, and FIFO C; at time T1, the image captured by the second light source 13, i.e., the second frame of the image to be processed, is acquired and repeatedly stored in the three second logic units 221, for example... Figure 4 In FIFO D, FIFO E and FIFO F; at time T2, the shooting result of the first light source 12 is acquired cyclically, that is, the third frame of the image to be processed, and the third frame of the image to be processed is repeatedly stored in the three first logic units 211; at time T3, the shooting result of the second light source 13 is acquired cyclically, that is, the fourth frame of the image to be processed, and the fourth frame of the image to be processed is repeatedly stored in the three first logic units 211; then the shooting results of the two light sources generated by the light capturing device 14 are alternately stored in turn until all the images to be processed are stored.

[0093] In practical applications, due to memory access delays, processor 23 can only begin retrieving the image to be processed in the third frame, meaning it can only retrieve the first frame of the image to be processed stored in time T0 at time T2. Since processor 23 can only retrieve one frame of the image to be processed at time T2, it cannot perform image interpolation or image fusion processing on that frame. However, due to the first-in, first-out (FIFO) nature of the memory, the first frame of the image to be processed will be retrieved from FIFO A at time T2, so that subsequent frames of the image to be processed stored in FIFO A can be retrieved at time T3. Furthermore, at time T3, processor 23 retrieves the third frame of the image to be processed stored at time T2 from FIFO A. Since image interpolation requires at least two images, processor 23 also needs to retrieve the first frame of the image to be processed stored at time T0 from FIFO B. Then, processor 23 performs image interpolation processing on the first frame and the third frame to be processed to obtain the interpolated image corresponding to the second frame. Furthermore, at time T3, processor 23 also needs to acquire the shooting result containing another light source, that is, acquire the second frame of the image to be processed stored at time T1 from FIFO D, and fuse the interpolated image corresponding to the second frame and the second frame to be processed to obtain the target fused image at time T3.

[0094] Furthermore, at time T4, processor 23 acquires the fourth frame of the image to be processed stored at time T3 in FIFO D, and the second frame of the image to be processed in FIFO E. Then, processor 23 performs image interpolation processing on the second and fourth frames of the image to be processed to obtain the interpolated image corresponding to the third frame. Further, at time T4, processor 23 also needs to acquire the shooting result containing another light source, that is, acquire the third frame of the image to be processed in FIFO B, and fuse the interpolated image corresponding to the third frame with the third frame of the image to be processed to obtain the target fused image at time T3. Simultaneously, at time T4, the first frame of the image to be processed stored at time T0 is retrieved to facilitate the retrieval and processing of subsequent images at later times. Optionally, processor 23 sequentially retrieves the images to be processed stored in each logic unit according to the above retrieval order, and performs image interpolation and image processing to obtain the target fused image.

[0095] Figure 5 This is a schematic diagram of another image processing system provided in an embodiment of the present invention. See also... Figure 5Based on the above embodiments, optionally, the processor 23 includes a to-be-processed image group generation module 231, an interpolated image group acquisition module 232, and a target fusion image acquisition module 233; wherein, the to-be-processed image acquisition module is used to acquire n to-be-processed image groups; the interpolated image group acquisition module 232 is used to perform interpolation processing on any to-be-processed image group captured at an intermediate time, based on two adjacent to-be-processed image groups of the current to-be-processed image group, to obtain the interpolated image group corresponding to the current to-be-processed image group; the target fusion image acquisition module 233 is used to perform image fusion of each to-be-processed image group and each interpolated image group to obtain the target fusion image of the captured object.

[0096] In this embodiment, the processor 23 can be a smart terminal device or a cloud server. The processor 23 only needs to have the functions of image processing and video rendering. This embodiment does not limit the form of the processor 23. The processor 23 includes a to-be-processed image group generation module 231, an interpolated image group acquisition module 232, and a target fused image acquisition module 233.

[0097] Specifically, the image group generation module 231 acquires the images to be processed stored in the image generation device 1. Specifically, it may involve: storing each frame of the images to be processed in the first logic unit 211 and the second logic unit 221 of the memory according to the acquisition time, and determining the image group corresponding to each frame of the images to be processed based on the data stored in each logic unit.

[0098] Specifically, the interpolated image group acquisition module 232 performs image interpolation processing on the acquired image group to be processed to obtain the interpolated image group. See [link to documentation] for details. Figure 6 Please provide an explanation.

[0099] Figure 6 This is a schematic diagram of another image processing system provided in an embodiment of the present invention. See also... Figure 6 Based on the above embodiments, optionally, the interpolated image group acquisition module 232 includes a pixel mean data acquisition unit 2321, an interpolated image acquisition unit, and an interpolated image group acquisition unit 2322; wherein, the pixel mean data acquisition unit 2321 is used to determine the pixel mean data of two corresponding frames of images to be processed in two adjacent image groups; the interpolated image group acquisition unit 2322 is used to perform data interpolation processing on the pixel mean data to obtain the interpolated images of the corresponding two frames of images to be processed, and based on each of the interpolated images, to obtain the interpolated image group of the current group of images to be processed.

[0100] Specifically, the pixel mean data acquisition unit 2321 is used to obtain the pixel mean data of each pixel in the two images to be interpolated. Specifically, for any two corresponding frames of images to be processed in the two adjacent image groups, two sets of pixel data of each corresponding pixel in the two frames to be processed are obtained, and the two sets of pixel data are averaged to obtain the pixel mean data of the two sets of pixel data.

[0101] Specifically, the interpolated image group obtaining unit 2322 is used to obtain the difference image group corresponding to the image group to be processed. Specifically, it can be to obtain a preset interpolation algorithm, perform data interpolation processing on the pixel mean data based on the interpolation algorithm to obtain the interpolated images of the two frames to be processed, and obtain the interpolated image group of the current image group to be processed based on the interpolated images corresponding to the two frames to be processed in each of the two adjacent image groups.

[0102] It should be noted that any existing image interpolation algorithm can be used for the fusion process. This embodiment does not limit the interpolation algorithm used in the interpolation process.

[0103] Optionally, based on the above implementation method, the target fusion image acquisition module 233 can be further described in detail. Figure 6 Please provide an explanation. See also... Figure 6 Based on the above embodiments, optionally, the target fusion image acquisition module 233 includes a local fusion image acquisition unit 2331 and a target fusion image acquisition unit 2332; wherein, the local fusion image acquisition unit 2331 is used to perform image fusion on the image group to be processed in the current group and the interpolated image group corresponding to the current group of image groups to be processed for any group of image groups to be processed, to obtain a local fusion image group of the current group of image groups to be processed; the target fusion image acquisition unit 2332 is used to determine the encoding order of each group of local fusion image groups respectively, and to stitch the images of each group of local fusion image groups based on each encoding order, to obtain a target fusion image of the photographed object.

[0104] Specifically, the local fusion image acquisition unit 2331 is used to perform fusion processing on the image to be processed and the interpolated image to obtain a local fusion image. Specifically, it can acquire a fusion algorithm and perform fusion processing on each frame of the image to be processed in the current group of images to be processed and each frame of the interpolated image group corresponding to the current group of images to be processed to obtain a local fusion image.

[0105] It should be noted that any existing image fusion algorithm can be used in the fusion process, and this embodiment does not limit the fusion algorithm used in the fusion process.

[0106] Specifically, the target fusion image acquisition unit 2332 is used to stitch together each group of local fusion images to obtain a target fusion image. Specifically, it may involve determining the local position of the photographed object corresponding to each frame of the image to be processed in each group of images to be processed; determining the encoding order of each frame of the local fusion image in each group of local fusion image corresponding to each frame of the image to be processed based on the local position of each frame of the image to be processed; and stitching together the fusion images in each group of local fusion image based on the encoding order to obtain the target fusion image of the photographed object.

[0107] It should be noted that the target fused image acquisition unit 2332 can perform image stitching before obtaining the local fused images corresponding to all frames of images to be processed. That is, it stitches images based on existing local fused images and continues stitching after obtaining subsequent local fused images until the fused images corresponding to all images to be processed are obtained, at which point stitching stops, and the target fused image is obtained. Based on the above rendering method, the information change process of the captured object within a preset time period can be observed in real time, further reducing latency. Optionally, image stitching can also be performed together after obtaining the local fused images of all images to be processed to obtain the target fused image. Based on the above stitching method, the complete fused result of the captured object can be directly obtained. The above method of obtaining the target fused image is only an exemplary introduction to the technical solution of this embodiment, and this embodiment does not limit it.

[0108] Based on the above implementation method, if the image group to be processed is the first frame image group to be processed, then each frame image to be processed in the image group to be processed is also output before the target fused image is output.

[0109] Specifically, when the first set of images to be processed is obtained, since there is only one set of images to be processed, image interpolation and image fusion processing cannot be performed. The first set of images to be processed can be output first, and then local fused images containing fusion information can be continuously output.

[0110] See also the examples. Figure 4 Since the processor 23 can only retrieve one frame of the image to be processed at time T2, and there are no other frames of the image to be processed that can be used for image interpolation and image fusion processing, the first frame of the image to be processed retrieved at time T0 is directly output at time T2, so that the image display in the subsequent output images is smoother, thereby further improving the image display effect.

[0111] Based on the above embodiments, it should be noted that the image processing system in this embodiment can be an endoscopic imaging system. The first light source 12 in the image processing system is a visible light source, and the second light source 13 is a near-infrared light source. Accordingly, the image captured by the first light source 12 is a visible light image, and the image captured by the second light source 13 is a near-infrared light image.

[0112] In practical applications, endoscopic techniques have emerged to enable clear observation of lesions within the patient's body, allowing for precise surgical treatment. Endoscopes expand the surgical field, enabling surgeons to observe lesions more clearly during the procedure. To ensure accurate treatment of lesions while avoiding damage to surrounding tissues, it is necessary to obtain morphological and metabolic information about the lesion.

[0113] Specifically, by injecting indocyanine green into the lesion site, and then using a near-infrared light source in the 780nm-811nm band to emit near-infrared light to the lesion site, and capturing the light on the subject, a near-infrared light image of the lesion site is obtained to acquire metabolic information of the lesion site.

[0114] Specifically, visible light is emitted to the lesion site using a visible light source in the 400-650nm wavelength range, and the visible light image of the lesion site is obtained by capturing the light on the subject, thereby acquiring morphological information of the lesion site.

[0115] Based on the above implementation method, in this embodiment, the first light source can also be a near-infrared light source, and the second light source can be a white light source. Accordingly, the image captured by the first light source is a near-infrared light image, and the image captured by the second light source is a visible light image. That is, in practical applications, the first frame is captured using near-infrared light to obtain metabolic information of the lesion site, and the second frame is captured using visible light to obtain morphological information of the lesion site. These images are then fused to obtain fused information.

[0116] It should be noted that this embodiment does not limit the priority order of shooting the object by the two light sources.

[0117] Furthermore, visible light images at each time moment, as well as the preceding and following near-infrared light images of the visible light images, are acquired. The two near-infrared light images are interpolated to obtain an interpolated near-infrared light image. This interpolated near-infrared light image and the visible light image are then fused to obtain a fused image. The process also involves acquiring near-infrared light images at each time moment, acquiring the preceding and following visible light images of the near-infrared light images, interpolating the preceding and following visible light images to obtain an interpolated visible light image, and fusing the interpolated visible light image and the infrared light image to obtain a fused image. The fused images are then output, thus solving the problem of image ghosting in the fused images obtained in the prior art, which leads to poor output fused image quality and improving the image quality of the fused image.

[0118] Based on the above implementation method, see Figure 7 Furthermore, the technical solution of this invention also provides a preferred embodiment. Figure 7 This is a schematic diagram illustrating an image processing method provided in an embodiment of the present invention, which involves capturing images of a subject to obtain multiple sets of images to be processed, and generating a target fused image based on these multiple sets of images. In this embodiment, each set of images to be processed is presented as a frame. Exemplarily, the processing method includes:

[0119] Specifically, the subject is divided into rows to obtain multiple rows of local positions. In this exemplary embodiment, the subject is subdivided into four rows of local positions. Two light sources are used to sequentially capture images of different local positions of the subject. Specifically, the first light source is used to capture images of the first row of local positions to obtain the first frame of the image to be processed, i.e., the first row of local image data to be processed; the second light source is used to capture images of the second row of local positions to obtain the second row of local image data to be processed; the first light source is used to capture images of the third row of local positions to obtain the third row of local image data to be processed; and the second light source is used to capture images of the fourth row of local positions to obtain the fourth row of local image data to be processed. This yields the result of a complete scan of the subject. In practical applications, if it is necessary to capture and display the information of the subject for a long time, it is necessary to continue using the two light sources to capture images of the subject in a loop until the capture is completed.

[0120] Furthermore, during the acquisition of partial image data to be processed from the subject, image interpolation is performed on the first and third rows of partial image data to obtain interpolated image data corresponding to the second row of partial image data. This interpolated image data is then fused with the second row of partial image data to obtain fused image data. Further, fused image data for the third row of partial image data is obtained using the above method. For the first and fourth rows of partial image data, since there are no adjacent sets of image data for interpolation, corresponding interpolated data cannot be obtained. Therefore, when fusing the two rows of partial image data, a preset transparent image can be acquired for fusion processing to obtain fused image data corresponding to the two rows of partial image data. Alternatively, the captured partial image data can be directly used as its corresponding fused image data. Further, according to the corresponding local positions of each frame of fused image data on the subject, they are fused to obtain a complete fused image of the subject.

[0121] It's important to note that the reason for not directly using the first and second-to-last rows of local image data from the next frame for interpolation in the last row of local image data is that the corresponding local positions in the first and second-to-last rows differ significantly, resulting in substantial differences in the data content between the two rows. Consequently, the interpolated result would not correspond to the last row of local image data, leading to inaccurate interpolation and thus inaccurate fusion results. Similarly, the reason for not interpolating the first local image data is the same.

[0122] It should also be noted that the above processing method immediately performs image fusion and displays the local images upon acquisition, thereby solving the problem of poor output endoscopic video quality caused by imaging delay in existing technologies, reducing system imaging delay, and thus improving the display effect of endoscopic video.

[0123] Figure 8 This is a flowchart of another image processing method provided in an embodiment of the present invention. This embodiment is applicable to the case of processing captured images to generate fused images. This method can be executed by an image processing device 2, which can be implemented in hardware and / or software, and can be configured in an image processing system. Figure 8 As shown, the method includes:

[0124] S110. Obtain n groups of images to be processed.

[0125] Among them, a group of images to be processed is the result of a continuous shooting from any light source. A continuous shooting result includes at least one frame of the image to be processed. Each frame of the image to be processed is shot at a preset interval. Odd-numbered groups of images to be processed are the shooting results from the first light source, and even-numbered groups of images to be processed are the shooting results from the second light source.

[0126] S120. For any group of images to be processed captured at an intermediate time, perform interpolation processing based on two adjacent groups of images to be processed in the current group to obtain the interpolated image group corresponding to the current group of images to be processed.

[0127] S130. The images to be processed and the interpolated images are fused to obtain the target fused image of the subject.

[0128] The image processing method provided in this invention solves the problem of poor output endoscopic video quality caused by imaging delay in the prior art by immediately performing image fusion and displaying the local image capture results upon acquisition. This reduces system imaging delay and improves the display effect of endoscopic video.

[0129] The image processing method provided in the embodiments of the present invention can be executed by the image processing system provided in any embodiment of the present invention. The image processing system has the corresponding functional modules and beneficial effects of the execution system generation method.

[0130] Figure 9 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0131] like Figure 9As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0132] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0133] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as image processing methods.

[0134] In some embodiments, the image processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the image processing method by any other suitable means (e.g., by means of firmware).

[0135] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] Computer programs for implementing the image processing methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0137] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0138] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0139] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0140] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0141] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0142] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image processing system, characterized in that, include: Image generation apparatus and image processing apparatus; wherein, The image generation device is used to sequentially capture images of different local positions of the subject using two light sources, generating n sets of images to be processed of different local positions of the subject; wherein, a set of images to be processed is a continuous shooting result of any one light source, a continuous shooting result includes at least one frame of the image to be processed, each frame of the image to be processed is captured at a preset interval, odd-numbered sets of images to be processed are the shooting results of the first light source, and even-numbered sets of images to be processed are the shooting results of the second light source. The image processing device is used to acquire n groups of images to be processed; for any group of images to be processed captured at an intermediate time, interpolation processing is performed based on two adjacent groups of images to be processed in the current group to obtain an interpolated image group corresponding to the current group to be processed; and each group of images to be processed and each interpolated image group are image fused to obtain a target fused image of the captured object.

2. The system according to claim 1, characterized in that, The image generation device includes a light source controller, a first light source, a second light source, and a light capturing device; The light source controller is used to project light with different center wavelengths in adjacent groups and light with the same center wavelength in groups separated by a distance. The first light source is used to project light of a first center wavelength onto the object being photographed under the control of the light source controller; The second light source is used to project light of a second center wavelength onto the object being photographed under the control of the light source controller; The light-capturing device is used to capture light on the subject and generate an image to be processed.

3. The system according to claim 1, characterized in that, The image processing device includes a first memory, a second memory, and a processor; the first memory includes k first logic units, and the second memory includes k second logic units; k is a positive integer greater than a preset threshold. The first logic unit is used to store the image capture result of the first light source generated by the image generation device; The second logic unit is used to store the image capture result of the second light source generated by the image generation device; The processor is configured to perform the following steps in the image processing device: acquiring multiple frames of images to be processed, generating multiple groups of images to be processed, and performing image fusion on each group of images to be processed and the interpolated image group to generate a target fused image corresponding to each group of images to be processed.

4. The system according to claim 3, characterized in that, The processor includes a module for generating a group of images to be processed, a module for obtaining an interpolated group of images, and a module for obtaining a target fused image; wherein... The image group generation module is used to obtain n image groups to be processed; The interpolated image group acquisition module is used to perform interpolation processing on any group of images to be processed captured at an intermediate time, based on two adjacent groups of images to be processed in the current group, to obtain the interpolated image group corresponding to the current group of images to be processed. The target fusion image acquisition module is used to fuse the images to be processed and the interpolated images to obtain the target fusion image of the photographed object.

5. The system according to claim 4, characterized in that, The interpolated image group acquisition module includes a pixel mean data acquisition unit and an interpolated image group acquisition unit; wherein... The pixel mean data acquisition unit is used to determine the pixel mean data of two corresponding frames of images to be processed in two adjacent groups of images to be processed. The interpolated image group obtaining unit is used to perform data interpolation processing on the pixel mean data to obtain the interpolated images of the corresponding two frames of images to be processed, and to obtain the interpolated image group of the current group of images to be processed based on each of the interpolated images.

6. The system according to claim 4, characterized in that, The target fused image acquisition module includes a local fused image acquisition unit and a target fused image acquisition unit; wherein... The local fusion image acquisition unit is used to perform image fusion on the current group of images to be processed and the interpolated image group corresponding to the current group of images to be processed for any group of images to be processed, so as to obtain the local fusion image group of the current group of images to be processed. The target fusion image acquisition unit is used to determine the encoding order of each group of local fusion image groups, and to stitch the local fusion image groups together based on the encoding order to obtain the target fusion image of the photographed object.

7. The system according to claim 1, characterized in that, If the image group to be processed is the first frame of the image group to be processed, then each frame of the image group to be processed is output before the target fused image is output.

8. The system according to claim 1, characterized in that, The image frame to be processed includes m rows / columns of pixel data, where m is a positive integer less than a preset threshold.

9. The system according to any one of claims 1-8, characterized in that, The image processing system is an endoscopic imaging system. Accordingly, the first light source in the image processing system is a visible light source and the second light source is a near-infrared light source, or the first light source is a near-infrared light source and the second light source is a visible light source.

10. An image processing method, characterized in that, include: Obtain n sets of images to be processed; wherein, each set of images to be processed is a continuous shooting result of any light source, and a continuous shooting result includes at least one frame of images to be processed, and each frame of images to be processed is shot at a preset interval time. Odd-numbered sets of images to be processed are shooting results of the first light source, and even-numbered sets of images to be processed are shooting results of the second light source. For any group of images to be processed captured at an intermediate time, interpolation is performed based on the two adjacent groups of images to be processed in the current group to obtain the interpolated image group corresponding to the current group to be processed. The images to be processed and the interpolated images are fused together to obtain the target fused image of the photographed object.

Images