Image processing system and image processing method

Abstract

The application discloses an image processing system and an image processing method. The image processing system comprises an image generating device and an image processing device. The image generating device is used for continuously shooting a shooting object by using two light sources and generating a plurality of image groups to be processed. The image group to be processed comprises a first image, a second image and a third image. The image processing device is used for acquiring the plurality of image groups to be processed. For any image group to be processed, the third image and the first image in the current image group to be processed are subjected to image interpolation processing to obtain an interpolated image of the current image group to be processed. The interpolated image and the second image are subjected to image fusion processing to obtain a target fusion image corresponding to the current image group to be processed. The technical scheme disclosed by the application realizes the improvement of the image quality of the fusion image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of endoscopic imaging, and in particular to an image processing system and an image processing method. BACKGROUND

[0002] In the prior art, visible light and near-infrared light are emitted to a lesion site to obtain an image of the lesion site. In order to observe the metabolic condition of the tissue and avoid cutting into blood vessels during surgery, metabolic information and blood vessel shape information of the lesion site need to be obtained at the same time. However, the image obtained by the prior art cannot simultaneously include the metabolic information and the blood vessel shape information of the lesion site, and therefore it is necessary to perform image superposition on a visible light image containing blood vessel shape information and an infrared light image containing metabolic information.

[0003] However, in order to achieve superposition of the visible light image and the infrared light image collected by the endoscope, the prior art usually has the following two methods: the first method is to use multiple CMOS sensors to simultaneously capture white light and near-infrared images, and to obtain a superimposed image through image fusion. This method has good performance and can obtain a high-frame-rate video stream, but a complex optical system is needed to separate the white light and the near-infrared light, and at least two CMOS sensors are needed to complete imaging, and the images need to be registered, so the optical imaging module is large in size and high in cost. The second method is to use a single CMOS sensor, and the light source generates white light and near-infrared light in frames. After two adjacent frames of white light and near-infrared light images are captured, a superimposed image is obtained through image fusion and output. This method has a simple structure, small size and low cost, but it has a significant disadvantage: when capturing images in motion, there will be a certain motion deviation between the two adjacent frames of images, which will cause the white light and near-infrared light images in the superimposed image to not coincide, resulting in a poor output image. SUMMARY

[0004] The present application provides an image processing system and an image processing method, which fuse different frames of images after image interpolation to solve the problem of ghosting in the fused image obtained by the prior art, thereby improving the image quality of the fused image.

[0005] In a first aspect, an image processing system is provided, which comprises an image generation device and an image processing device, wherein the image generation device is configured to continuously capture an object by using two light sources to generate a plurality of image groups to be processed, and the image processing device is configured to perform image processing on the image groups to be processed to generate a superimposed image.

[0006] The image generation device is configured to continuously capture an object by using two light sources to generate a plurality of image groups to be processed, and the image processing device is configured to perform image processing on the image groups to be processed to generate a superimposed image.

[0007] The image processing apparatus is configured to acquire a plurality of image groups to be processed; for any one of the image groups to be processed, performing image interpolation processing on the third image and the first image in the current image group to be processed to obtain an interpolated image corresponding to the current image group to be processed; and performing image fusion processing on the interpolated image and the second image to obtain a target fusion image corresponding to the current image group to be processed.

[0008] Optionally, the image generation apparatus comprises a light source controller, a first light source, a second light source and a light capturing device.

[0009] The light source controller is configured to project light of different wavebands in adjacent frames and project light of the same waveband in alternate frames.

[0010] The first light source is configured to project light of a first waveband to the shooting object under the control of the light source controller.

[0011] The second light source is configured to project light of a second waveband to the shooting object under the control of the light source controller.

[0012] The light capturing device is configured to capture the light on the shooting object and generate an image to be processed.

[0013] Optionally, the image processing apparatus comprises a first memory, a second memory and a processor; the first memory comprises k first logical units, and the second memory comprises k second logical units; k is a positive integer greater than a preset threshold.

[0014] The first logical unit is configured to store the shooting result of the first light source generated by the image generation apparatus.

[0015] The second logical unit is configured to store the shooting result of the second light source generated by the image generation apparatus.

[0016] The processor is configured to perform the steps of acquiring a plurality of image groups to be processed and performing image fusion on the plurality of image groups to generate target fusion images corresponding to the plurality of image groups to be processed, respectively.

[0017] Optionally, the processor comprises an image group to be processed acquisition module, an interpolated image obtaining module and a target fusion image obtaining module; wherein,

[0018] The image group to be processed acquisition module is configured to acquire a plurality of image groups to be processed.

[0019] The interpolated image obtaining module is configured to, for any one of the image groups to be processed, perform image interpolation processing on the third image and the first image in the current image group to be processed to obtain an interpolated image corresponding to the current image group to be processed.

[0020] a target fusion image obtaining module configured to perform image fusion processing on the interpolation image and the second image to obtain a target fusion image corresponding to the current image group to be processed.

[0021] Optionally, the interpolation image obtaining module comprises an image feature acquisition unit, a feature offset determination unit and an interpolation image obtaining unit, wherein

[0022] the image feature acquisition unit is configured to acquire a first image feature corresponding to the first image and a third image feature corresponding to the third image respectively;

[0023] the feature offset determination unit is configured to perform feature convolution processing on the third image feature and the first image feature to obtain a feature offset of the current image group to be processed;

[0024] the interpolation image obtaining unit is configured to perform image interpolation on the third image and the first image based on the feature offset to obtain an interpolation image of the current image group to be processed.

[0025] Optionally, the interpolation image obtaining unit comprises a third offset image obtaining subunit, a first offset image obtaining subunit and an interpolation image obtaining subunit, wherein

[0026] the third offset image obtaining subunit is configured to perform image offset on the third image based on the feature offset to obtain a third offset image corresponding to the third image;

[0027] the first offset image obtaining subunit is configured to perform image offset on the first image based on the feature offset to obtain a first offset image corresponding to the first image;

[0028] the interpolation image obtaining subunit is configured to acquire a preset interpolation parameter, perform interpolation processing on the first offset image and the third offset image based on the interpolation parameter to obtain an interpolation image of the current image group to be processed.

[0029] Optionally, the processor further comprises an image acquisition time point determination module and an image output time point determination module, wherein

[0030] the image acquisition time point determination module is configured to acquire an image acquisition time point of the third image in the current image group to be processed for any image group to be processed;

[0031] the image output time point determination module is configured to determine an image output time point of the target fusion image based on the image acquisition time point.

[0032] Optionally, if the to-be-processed image group is a first to-be-processed image group, the current to-be-processed image group outputs the first image before outputting the target fusion image.

[0033] Optionally, the image processing system is an endoscope imaging system, and correspondingly, the first light source in the image processing system is a visible light source, and the second light source is a near-infrared light source, or the first light source is a near-infrared light source, and the second light source is a white light source.

[0034] In a second aspect, an embodiment of the present application further provides an image processing method, which comprises: acquiring a plurality of to-be-processed image groups; wherein the to-be-processed image group comprises a first image, a second image and a third image, the first image and the third image are the shooting results of a first light source, and the second image is the shooting result of a second light source;

[0035] For any to-be-processed image group, the third image and the first image in the current to-be-processed image group are subjected to image interpolation processing to obtain an interpolated image of the current to-be-processed image group;

[0036] The interpolated image and the second image are subjected to image fusion processing to obtain a target fusion image corresponding to the current to-be-processed image group.

[0037] In a third aspect, an embodiment of the present application further provides an electronic device, which comprises:

[0038] at least one processor; and

[0039] a memory in communication connection with the at least one processor; wherein

[0040] The memory stores a computer program which can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the image processing method according to any one of the embodiments of the present application.

[0041] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a computer instruction, and the computer instruction is used to enable a processor to execute the image processing method according to any one of the embodiments of the present application when executed.

[0042] The technical scheme of the embodiment of the present application specifically comprises an image generation device and an image processing device, the image processing device obtains an interpolation image containing morphological information by performing image interpolation on a first frame of to-be-processed image and a third frame of to-be-processed image containing morphological information, and then obtains a target fusion result by fusing the interpolation image containing morphological information and a second frame of to-be-processed image containing metabolic information, thereby solving the problem of ghosting in the fusion image obtained in the prior art, leading to poor fusion image effect, and realizing the improvement of the image quality of the fusion image.

[0043] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0045] Figure 1 is a structural schematic diagram of an image processing system provided according to an embodiment of the present application;

[0046] Figure 2 is a schematic diagram of a to-be-processed image group provided according to an embodiment of the present application;

[0047] Figure 3 is a schematic diagram of a to-be-processed image group provided according to an embodiment of the present application;

[0048] Figure 4 is a structural schematic diagram of another image processing system provided according to an embodiment of the present application;

[0049] Figure 5 is a structural schematic diagram of another image processing system provided according to an embodiment of the present application;

[0050] Figure 6 is a structural schematic diagram of an image generation device provided according to an embodiment of the present application;

[0051] Figure 7 is a structural schematic diagram of another image processing system provided according to an embodiment of the present application;

[0052] Figure 8 is a structural schematic diagram of another image processing system provided according to an embodiment of the present application;

[0053] Figure 9 is a structural schematic diagram of another image processing system according to Embodiment Three of the present application;

[0054] Figure 10 is a structural schematic diagram of another image processing system according to Embodiment Three of the present application;

[0055] Figure 11 is a flow chart of an image processing method according to Embodiment One of the present application;

[0056] Figure 12 is a structural schematic diagram of an electronic device implementing the image processing method according to Embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0058] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion.

[0059] The embodiments of the present application provide an image processing system. Figure 1 A structural schematic diagram of an image processing system according to an embodiment of the present application is provided. The present embodiment can be applied to the case of processing images of different light sources taken by a camera to generate a fusion image.

[0060] In the prior art, in order to observe the tissue metabolic condition and the tissue morphological position condition during surgery, it is necessary to simultaneously acquire the metabolic information and the morphological information of the lesion site. The image obtained based on the prior art cannot simultaneously include the metabolic information and the morphological information of the lesion site, so the obtained images respectively containing the two kinds of information need to be subjected to image fusion processing to obtain a fusion image. Further, the fusion image is sequentially outputted to realize the observation of the tissue metabolic condition and the tissue morphological position condition during surgery.

[0061] Currently, there are two methods for obtaining a fused image. The first method is to use multiple CMOS sensors to simultaneously capture white light and near-infrared images, and to obtain the superimposed image through image fusion. This method has good performance and can obtain a high-frame-rate video stream, but requires a complex optical system to separate the white light and near-infrared light, and at least two CMOS sensors to complete imaging, and the images need to be registered. The optical imaging module is large in size and high in cost. The second method is to use a single CMOS sensor, and the light source generates white light and near-infrared light in frames. After two adjacent frames of white light and near-infrared light images are captured, the superimposed image is obtained through image fusion and output. This method has a simple optical imaging system structure, small size and low cost, but has the obvious disadvantage that when capturing images in motion, there is a certain motion deviation between the two adjacent frames of images, which causes the white light and near-infrared light images of the superimposed image to not coincide, resulting in a poor output image effect.

[0062] Based on the above technical problems, the technical scheme of the embodiment of the present application provides an image processing system, which performs image interpolation on the previous image frame and the next image frame of the current time image frame, and then fuses the interpolated image with the current time image frame, and takes the fusion result as the output frame corresponding to the current time, thereby solving the problem of ghosting in the fused image obtained in the prior art, improving the image quality of the fused image, and improving the image quality of the fused image.

[0063] As shown in Figure 1 The image processing system includes an image generation device 1 and an image processing device 2. The image generation device 1 is used to continuously capture a shooting object using two light sources, and generates a plurality of image groups to be processed. The image group to be processed includes a first image, a second image and a third image. The first image and the third image are the shooting results of the first light source, and the second image is the shooting result of the second light source. The image processing device 2 is used to obtain a plurality of image groups to be processed. For any image group to be processed, the third image and the first image in the current image group to be processed are subjected to image interpolation processing to obtain an interpolated image of the current image group to be processed. The interpolated image and the second image are subjected to image fusion processing to obtain a target fused image corresponding to the current image group to be processed.

[0064] In the embodiment, the image processing system can be understood as a comprehensive processing system for obtaining the tissue morphology information and the tissue metabolism information of the shooting object in a preset time period by continuously shooting the shooting object by using the light sources and processing the shooting results to obtain the target fusion image. The shooting object can be a human body or a lesion site of an animal. In actual application, the tissue morphology information can be understood as the morphology and position of the lesion site that can be directly observed, and the morphology information can be used to reflect the tissue morphology and position of the lesion site. In order to clearly know the tissue metabolism of the lesion site, the tissue metabolism information of the lesion site also needs to be obtained.

[0065] Specifically, the image processing system can continuously shoot the shooting object by using two light sources, and perform image interpolation processing and image fusion processing on the shooting results in a preset time period to obtain a fusion image, so as to obtain the tissue morphology information and the tissue metabolism information of the shooting object in the preset time period. The image processing system includes an image generation device 1 and an image processing device 2.

[0066] Specifically, the image generation device 1 is used to continuously shoot the shooting object by using two light sources in turn, wherein each light source is used to shoot for a preset interval, and n frames of to-be-processed images of the shooting object are obtained after continuous shooting in a preset time period. According to the shooting time of each frame of to-be-processed image, the multiple frames of to-be-processed images are grouped to obtain multiple to-be-processed image groups. In the embodiment, the to-be-processed image groups include a first image, a second image and a third image, the first image and the third image are the shooting results of the first light source, and the second image is the shooting result of the second light source.

[0067] It needs to be specifically pointed out that the first image, the second image and the third image in the embodiment can be any three continuous to-be-processed images, for example, refer to FIG. 1B. Figure 2 Figure 2 Each matrix block in the embodiment is a to-be-processed image group, and each matrix includes three continuous to-be-processed images, that is, the first image, the second image and the third image. Alternatively, the first image, the second image and the third image can also be three discontinuous to-be-processed images, for example, refer to FIG. 1C. Figure 3 Figure 3 The three to-be-processed images represented by the same filler in the embodiment are the first image, the second image and the third image.

[0068] It also needs to be specifically pointed out that the first light source 12 can be a shooting light source that can emit first preset waveband excitation light for obtaining tissue morphology information, and the second light source 13 can be a shooting light source that can emit second preset waveband excitation light for obtaining tissue metabolism information.

[0069] ​​In actual application, in order to obtain the morphological information of the photographed object, the first light source can be used to emit light of the first preset wave band, i.e. the first light source is used to irradiate the photographed object with light of the first preset wave band, and then the light on the photographed object is captured to form an image, so that the morphological and positional information of the photographed object which can be directly observed can be obtained. Optionally, the wave band of the first light source in the embodiment can be the visible light wave band of 400-650 nm.

[0070] Further, since the metabolic information of the photographed object needs to be displayed by means of a developing agent, the developing agent needs to be injected into the photographed object in advance, and then the second light source is used to emit excitation light of the second preset wave band, the excitation light of the second preset wave band is the excitation light of the wave band corresponding to the developing agent, and further, the photographed object is irradiated based on the excitation light of the second preset wave band, the excitation light can excite the developing agent in the photographed object, so that the developing agent emits light carrying information under the excitation of the excitation light, and the light carrying information is captured to form an image, so that the metabolic information of the photographed object can be obtained, and further, the physiological information and pathological information of the photographed object can be obtained according to the metabolic information.

[0071] It should be noted that different types of developing agents correspond to different wave bands of excitation light. For example, when the developing agent is indocyanine green, the wave band of the excitation light is 780-811 nm. Optionally, the developing agent can also be other chemical substances, for example, when a dye of the rhodamine series is used as the developing agent, the excitation wave band of the excitation light is 520-600 nm; when a cyanine (Cy) series dye is used as the developing agent, the excitation wave band of the excitation light is 550-780 nm; and when an Alexa Fluor series dye is used as the developing agent, the excitation wave band of the excitation light is 340-680 nm. Of course, the above examples are only illustrative and cannot be regarded as a limitation of the embodiment. The embodiment does not limit the selection of the developing agent and the selection of the wave band of the excitation light.

[0072] It should be further noted that the image generation device 1 is connected with the image processing device 2, and is used to transmit the n frames of images to be processed generated by the image generation device 1 to the image processing device 2, so that the image processing device 2 performs image processing on the n frames of images to be processed to obtain a target fusion image. Optionally, the two devices can be electrically connected or communicatively connected, in other words, the connection mode between the two devices can realize the image transmission function, and the specific connection mode is not limited in the embodiment.

[0073] The image processing apparatus 2 is configured to receive the n frames of to-be-processed images transmitted by the image generating apparatus 1, and on the basis of the n frames of to-be-processed images, generate a plurality of to-be-processed image groups based on the n frames of to-be-processed images; for any to-be-processed image group, perform image interpolation processing on the third image and the first image in the current to-be-processed image group to obtain an interpolated image of the current to-be-processed image group; and perform image fusion processing on the interpolated image and the second image to obtain a target fusion image corresponding to the current to-be-processed image group.

[0074] On the basis of the above embodiment, the image generating apparatus 1, on the basis of the n frames of to-be-processed images, further transmits the n frames of to-be-processed images to the image processing apparatus 2, so that the image processing apparatus 2 performs image processing on the to-be-processed images to obtain a target fusion image after processing.

[0075] In the prior art, the method for obtaining an image containing morphological information and metabolic information at the same time is to fuse a first frame image containing morphological information and a second frame image containing metabolic information after collecting the two adjacent frames to obtain a fusion result. However, because there is a time delay between the first frame and the second frame, there is a motion deviation between the first frame image and the second frame image, in other words, there is a displacement between the image contents of the first frame image and the second frame image, so that there is image ghosting between the fusion results of the two frames, which specifically means that the photographed objects in the two frames cannot completely coincide, so that the image quality of the target fusion image is poor. In view of the above technical problem, the process of obtaining the target fusion image by the image processing apparatus 2 in the embodiment is to perform image interpolation on the first frame to-be-processed image and the third frame to-be-processed image containing morphological information to obtain an interpolated image containing morphological information, and then perform fusion on the interpolated image containing morphological information and the second frame to-be-processed image containing metabolic information to obtain a target fusion result. Because the three frames in the embodiment are continuously photographed, the interpolation result based on the first frame and the third frame can be regarded as the photographed result of the second frame, so that the target fusion result obtained in the embodiment can be understood as the image fusion result of the image containing metabolic information and the image containing morphological information in the same frame, and there will be no time delay between the obtained target fusion images, so that there will be no image ghosting between the obtained fusion results, and the image quality of the obtained target fusion image is good.

[0076] The image processing system provided by the embodiment specifically comprises an image generation device 1 and an image processing device 2. The image processing device 2 obtains an interpolation image containing morphological information by performing image interpolation on a first frame of to-be-processed image and a third frame of to-be-processed image containing morphological information, and further obtains a target fusion result by fusing the interpolation image containing morphological information and a second frame of to-be-processed image containing metabolic information. The image processing system solves the problem that the fusion image obtained in the prior art has image ghosting, resulting in poor fusion image effect, and improves the image quality of the fusion image.

[0077] Figure 4 Another structural schematic diagram of an image processing system provided by the embodiment is shown in the figure. Figure 4 On the basis of the above embodiment, optionally, the image generation device 1 comprises a light source controller 11, a first light source 12, a second light source 13 and a light line capturing device 14. The light source controller 11 is configured to project light lines of different wave bands in adjacent frames and project light lines of the same wave band in every other frame. The first light source 12 is configured to project light lines of a first wave band to a shooting object under the control of the light source controller 11. The second light source 13 is configured to project light lines of a second wave band to the shooting object under the control of the light source controller 11. The light line capturing device 14 is configured to capture the light lines on the shooting object and generate to-be-processed images.

[0078] In the embodiment, the light source controller 11 is connected to the first light source 12 and the second light source 13, respectively, and is configured to project light lines of different wave bands in adjacent frames and project light lines of the same wave band in every other frame. The light source controller 11 can be understood as a control switch of the light source, that is, the light source controller 11 can be used to control whether the current light source emits light lines of a preset wave band in the current frame.

[0079] Specifically, the first light source 12 can be controlled to emit light lines of a first preset wave band in odd-numbered frames, and the first light source 12 emits the light lines of the first preset wave band in the odd-numbered frames in response to a control instruction of the light source controller 11. The second light source 13 can be controlled to emit light lines of a second preset wave band in even-numbered frames, and the second light source 13 emits the light lines of the second preset wave band in the even-numbered frames in response to a control instruction of the light source controller 11.

[0080] For example, the light source controller 11 controls the power supply of the first light source 12 to be turned on in odd-numbered frames, and the first light source 12 emits light lines of a first preset wave band when the power supply is turned on and irradiates the light lines of the first preset wave band to a shooting object. The light source controller 11 controls the power supply of the second light source 13 to be turned on in even-numbered frames, and the second light source 13 emits light lines of a second preset wave band when the power supply is turned on and irradiates the light lines of the second preset wave band to the shooting object.

[0081] Specifically, the light capturing device 14 captures the light on the shooting object and generates the to-be-processed images. Optionally, based on the illumination frequency of the two light sources on the shooting object, the light capturing device captures the light of the first preset wave band in odd frames to obtain the shooting result of the first light source 12, and captures the light of the second preset wave band in even frames to obtain the shooting result of the second light source 13.

[0082] For example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor can be used as the light capturing device 14 to capture the light on the shooting object and generate n frames of to-be-processed images.

[0083] Based on the above-mentioned embodiments, the image generating device 1 in the technical scheme of the present embodiment can further include a light source, a first filter, a second filter, a filter controller, and the light capturing device 14. Specifically, the light source emits light of a third preset wave band, wherein the third preset wave band includes the first preset wave band and the second preset wave band. Specifically, the light source continuously emits light of the third preset wave band, the filter controller controls the first filter to work in odd frames so that the first filter can only transmit the light of the first preset wave band to the shooting object, and controls the second filter to work in even frames so that the second filter can only transmit the light of the second preset wave band to the shooting object. Then, the light capturing device 14 captures the light on the shooting object to generate n frames of to-be-processed images.

[0084] It should be noted that the above-mentioned two methods of obtaining to-be-processed images are exemplary descriptions of the technical scheme of the present embodiment, and the technical scheme of the present embodiment can also obtain the above-mentioned n frames of to-be-processed images through other existing image acquisition methods, and the acquisition method and acquisition device are not limited.

[0085] Figure 5 Another structural schematic diagram of an image processing system provided by the present embodiment is provided. Referring to Figure 5 Based on the above-mentioned embodiments, the image processing device 2 can include a first memory 21, a second memory 22, and a processor 23. The first memory 21 includes k first logical units 211, and the second memory 22 includes k second logical units 221. k is a positive integer greater than a preset threshold. The first logical unit 211 is used to store the shooting result of the first light source 12 generated by the image generating device 1. The second logical unit 221 is used to store the shooting result of the second light source 13 generated by the image generating device 1. The processor 23 is used to execute the steps of obtaining a plurality of to-be-processed image groups and performing image fusion on the plurality of to-be-processed image groups to generate a plurality of target fusion images corresponding to the plurality of to-be-processed image groups.

[0086] In this embodiment of the invention, the first memory 21 and the second memory 22 are respectively connected to the light-capturing device 14 and are used to store the images to be processed generated by the light-capturing device 14. Specifically, the first memory 21 is used to receive and store the shooting results of the first light source 12 in odd-numbered frames; the second memory 22 is used to receive and store the shooting results of the second light source 13 in even-numbered frames.

[0087] In practical applications, both the first memory 21 and the second memory 22 can be non-volatile memories. In the technical solution of this embodiment, the two adjacent frames of the image to be processed in the current frame need to be interpolated, and then the interpolated image and the current frame of the image to be processed are fused. Therefore, for any frame of the image to be processed, multiple frames need to be repeatedly stored in the memory so that the current frame of the image to be processed can be reused for image fusion processing. Based on this, in this embodiment, the first memory 21 used to store the shooting results of the first light source 12 can store each frame of the image to be processed in k logical units, and correspondingly, the second memory 22 used to store the shooting results of the second light source 13 can store each frame of the image to be processed in k logical units. Optionally, in this embodiment, k can be 3.

[0088] See examples Figure 6 , Figure 6 This is a schematic diagram of the image processing apparatus provided in an embodiment of the present invention. Image interpolation and image fusion processing can be performed based on the functional modules in the above-described image processing apparatus to obtain the target fused image of each image group to be processed. For example, in the following exemplary description, three consecutive frames of images in the image group to be processed in this embodiment are used as an example. Specifically, at time T0, the shooting result of the first light source 12 is acquired, i.e., the first frame of the image to be processed, and the first frame of the image to be processed is repeatedly stored in three first logic units 211, for example... Figure 6 In FIFO A, FIFO B, and FIFO C; at time T1, the image captured by the second light source 13, i.e., the second frame of the image to be processed, is acquired and repeatedly stored in the three second logic units 221, for example... Figure 6 In FIFO D, FIFO E and FIFO F; at time T2, the shooting result of the first light source 12 is acquired cyclically, that is, the third frame of the image to be processed, and the third frame of the image to be processed is repeatedly stored in the three first logic units 211; at time T3, the shooting result of the second light source 13 is acquired cyclically, that is, the fourth frame of the image to be processed, and the fourth frame of the image to be processed is repeatedly stored in the three first logic units 211; then the shooting results of the two light sources generated by the light capturing device 14 are alternately stored in turn until all the images to be processed are stored.

[0089] Since the memory used in the embodiment has access delay, the processor 23 can only start to take out the image to be processed outside at the third frame, that is, the first frame image to be processed stored at T0 can be taken out at T2. Since the processor 23 can only take out one frame of image to be processed at T2, the image interpolation and image fusion processing cannot be performed on the frame of image to be processed. However, since the memory has the characteristic of first-in first-out, the first frame image to be processed is taken out from FIFO A at T2, so that the image to be processed stored subsequently in FIFO A can be taken out at T3. Further, at T3, the processor 23 takes out the third frame image to be processed stored at T2 from FIFO A, and two images are required for image interpolation, so the processor 23 also needs to take out the first frame image to be processed stored at T0 from FIFO B, and then the processor 23 performs image interpolation processing on the first frame image to be processed and the third frame image to be processed to obtain the interpolation image corresponding to the second frame. Further, at T3, the processor 23 also needs to obtain the shooting result containing another light source, that is, the second frame image to be processed stored at T1 is obtained from FIFO D, and the interpolation image corresponding to the second frame and the second frame image to be processed are fused to obtain the target fusion image at T3.

[0090] Further, at T4, the processor 23 obtains the fourth frame image to be processed stored at T3 from FIFO D, and obtains the second frame image to be processed from FIFO E. Then the processor 23 performs image interpolation processing on the second frame image to be processed and the fourth frame image to be processed to obtain the interpolation image corresponding to the third frame. Further, at T4, the processor 23 also needs to obtain the shooting result containing another light source, that is, the third frame image to be processed is obtained from FIFO B, and the interpolation image corresponding to the third frame and the third frame image to be processed are fused to obtain the target fusion image at T3. At the same time, the first frame image to be processed stored at T0 is also taken out at T4, so that the subsequent image to be processed can be taken out at the subsequent moment for processing. Optionally, the processor 23 takes out the image to be processed stored in each logical unit in turn based on the above taking-out sequence, and performs image interpolation and image processing to obtain the target fusion image.

[0091] Figure 7 Another structure schematic diagram of an image processing system provided by the embodiment of the application is shown in FIG. 4. Referring to FIG. 4, the image processing system comprises a processor 23, a plurality of FIFOs 21, and a plurality of logical units 22. Figure 7On the basis of the above-mentioned embodiments, optionally, the processor 23 comprises a to-be-processed image group acquisition module 231, an interpolation image obtaining module 232, and a target fusion image obtaining module 233; wherein the to-be-processed image group acquisition module 231 is configured to acquire a plurality of to-be-processed image groups; the interpolation image obtaining module 232 is configured to, for any to-be-processed image group, perform image interpolation processing on a third image and a first image in the current to-be-processed image group to obtain an interpolation image of the current to-be-processed image group; and the target fusion image obtaining module 233 is configured to perform image fusion processing on the interpolation image and a second image to obtain a target fusion image corresponding to the current to-be-processed image group.

[0092] In the present embodiment, the processor 23 can be a smart terminal device or a cloud server, and the processor 23 only needs to have the functions of image processing and video rendering, and the form of the processor 23 is not limited in the present embodiment. The processor 23 comprises a to-be-processed image group acquisition module 231, an interpolation image obtaining module 232, and a target fusion image obtaining module 233.

[0093] The to-be-processed image group acquisition module 231 acquires the to-be-processed images stored in the image generation device 1. Specifically, according to the time of acquisition, each frame of to-be-processed image is respectively stored in the first logical unit 211 and the second logical unit 21, and three frames of to-be-processed images taken out from each first logical unit 211 and each second logical unit 221 are taken as one to-be-processed image group.

[0094] The interpolation image obtaining module 232 performs image interpolation processing on the acquired to-be-processed images to obtain interpolation images.

[0095] Specifically, the target fusion image obtaining module 233 performs fusion processing on the interpolation images and the to-be-processed images to obtain fusion images. Specifically, the fusion coefficients corresponding to each frame of to-be-processed image can be obtained. Specifically, different fusion coefficients can be set in advance according to different display requirements. The current display requirement is acquired, and the fusion coefficient corresponding to the display requirement is determined.

[0096] For example, in the present embodiment, the display video wants to more clearly display the metabolic information of the photographed object, so the fusion coefficient of each frame of to-be-processed image reflecting the metabolic information is set to a large value, for example, 0.7, and the fusion coefficient of each frame of to-be-processed image reflecting the morphological information is set to a small value, for example, 0.3.

[0097] Since the current frame of to-be-processed image at different times can be the result of shooting by different light sources, the specific values of the first fusion coefficient and the second fusion coefficient need to be determined according to each time, which is not limited in the present embodiment.

[0098] Optionally, based on the above-mentioned embodiments, the interpolation image obtaining module 232 can specifically refer to Figure 8 It is illustrated. Figure 8 Another structural schematic diagram of an image processing system provided by an embodiment of the present application is provided. Referring to Figure 8 Optionally, based on the above-mentioned embodiments, the interpolation image obtaining module 232 includes an image feature obtaining unit 2321, a feature offset determining unit 2322 and an interpolation image obtaining unit 2323; the image feature obtaining unit 2321 is configured to obtain a first image feature corresponding to the first image and a third image feature corresponding to the third image respectively; the feature offset determining unit 2322 is configured to perform feature convolution processing on the third image feature and the first image feature to obtain a feature offset of the current image group to be processed; and the interpolation image obtaining unit 2323 is configured to perform image interpolation on the third image and the first image based on the feature offset to obtain an interpolation image of the current image group to be processed.

[0099] Specifically, the image feature obtaining unit 2321 is configured to obtain an image feature. Specifically, a first image feature corresponding to the first image is obtained, and a third image feature corresponding to the third image is obtained. The process of obtaining the image feature can be that a pixel histogram of the image is determined based on each pixel point in the image, and the image feature in the image is represented by the curve trend of the histogram. Optionally, the image feature can also be determined based on a neural network model, which is not limited.

[0100] Specifically, the feature offset determining unit 2322 is configured to determine a feature offset of the image. Specifically, the first image feature and the third image feature can also be subjected to feature convolution processing to obtain the feature offset. In other words, the change amount of the shooting object in the first image and the third image is determined, and the change amount is represented by the feature offset.

[0101] Specifically, the interpolation image obtaining unit 2323 determines the interpolation image of the first image and the third image based on the feature offset, which can specifically refer to Figure 9 It is illustrated.

[0102] Figure 9 Another structural schematic diagram of an image processing system provided by an embodiment of the present application is provided. Referring to Figure 9On the basis of the above embodiment, optionally, the interpolation image obtaining unit 2323 comprises a first offset image obtaining subunit 23231, a third offset image obtaining subunit 23232 and an interpolation image obtaining subunit 23233; wherein the first offset image obtaining subunit 23231 is configured to perform image offset on the first image based on the feature offset to obtain a first offset image corresponding to the first image; the third offset image obtaining subunit 23232 is configured to perform image offset on the third image based on the feature offset to obtain a third offset image corresponding to the third image; and the interpolation image obtaining subunit 23233 is configured to obtain a preset interpolation parameter, and perform interpolation processing on the first offset image and the third offset image based on the interpolation parameter to obtain an interpolation image of the current image group to be processed.

[0103] Specifically, the first offset image obtaining subunit 23231 is configured to obtain the first offset image. Specifically, the first offset image can be obtained by performing image offset on the first image based on the feature offset. Optionally, since there is a feature offset between the first image and the third image, there is 1 / 2 feature offset between the first image and the second image, so the first offset image can be obtained by adding 1 / 2 feature offset to the first image.

[0104] Specifically, the third offset image obtaining subunit 23232 is configured to obtain the third offset image. Specifically, the third offset image can be obtained by performing image offset on the third image based on the feature offset. Optionally, since there is a feature offset between the first image and the third image, there is 1 / 2 feature offset between the third image and the second image, so the third offset image can be obtained by subtracting 1 / 2 feature offset from the third image.

[0105] Specifically, the interpolation image obtaining subunit 23233 is configured to obtain the interpolation image. Specifically, since the first offset image and the third offset image represent images at the same moment, the interpolation image of the current image group to be processed can be obtained by performing interpolation processing on the first offset image and the third offset image based on a preset interpolation parameter.

[0106] Figure 10 Another structural schematic diagram of an image processing system provided by the embodiment of the present application is provided. Referring to Figure 10 On the basis of the above embodiment, optionally, the processor 23 further comprises an image acquisition time determination module 234 and an image output time determination module 235; wherein the image acquisition time determination module 234 is configured to acquire the image acquisition time of the third image in the current image group to be processed for any image group to be processed; and the image output time determination module 235 is configured to determine the image output time of the target fusion image based on the image acquisition time.

[0107] In practical application, on the basis of obtaining the target fusion image of each to-be-processed image group, the obtained target fusion image can be outputted to realize that the metabolic information and morphological information of the photographed object can be continuously observed.

[0108] Specifically, the processor 23 can determine the output time of the target fusion image based on the acquisition time of each frame of image to realize image output. Alternatively, in the embodiment, the processor 23 can determine the image acquisition time of the to-be-processed image group based on the image acquisition time determination module, and determine the image output time of the target fusion image based on the image output time determination module 235.

[0109] Specifically, the image acquisition time determination module 234 can acquire each frame of to-be-processed image in the current to-be-processed image group, and take the acquisition time of the last frame of to-be-processed image as the image acquisition time of the current to-be-processed image group, so the image acquisition time of the third image acquired from the first storage 21 or the second storage 22 is taken as the image acquisition time of the current to-be-processed image group. Further, since the processor 23 has a fast processing speed in image interpolation and image fusion, the delay of image processing can be ignored, that is, the image output time determination module 235 can take the acquisition time of the current to-be-processed image group as the image output time of the current to-be-processed image group. Further, each frame of target fusion image is outputted in turn based on the output time of each frame of target fusion image.

[0110] For example, referring back to Figure 6 For the first to-be-processed image group, the first to-be-processed image group includes three continuous frames of to-be-processed image, so the acquisition time of the third frame is taken as the image acquisition time, that is, T3, and the image acquisition time is taken as the image output time of the target fusion image of the first to-be-processed image group, that is, T3. Further, for the second to-be-processed image group, the second to-be-processed image group includes the second frame of to-be-processed image, the third frame of to-be-processed image and the fourth frame of to-be-processed image, the acquisition time of the fourth frame of to-be-processed image is taken as the image acquisition time of the second to-be-processed image group, that is, T4, and T4 is taken as the image output time of the second to-be-processed image group. Alternatively, the image output time of each to-be-processed image group is determined in turn, and the image output of the target fusion image is realized.

[0111] Based on the above embodiment, if the to-be-processed image group is the first to-be-processed image group, the current to-be-processed image group further outputs the first image before outputting the target fusion image.

[0112] Specifically, when the first frame of the to-be-processed image is acquired, since there is only one frame of the to-be-processed image, the image interpolation and the image fusion processing cannot be performed, and the first frame of the to-be-processed image can be output first, and then the target fusion image containing the fusion information is continuously output.

[0113] Example of continued participation Figure 6 Since the processor 23 can only take out one frame of the to-be-processed image at the T2 moment, there is no other frame of the to-be-processed image that can be used for image interpolation and image fusion processing, based on this, the first frame of the to-be-processed image taken out at the T0 moment is directly output at the T2 moment, so that the image display screen in the subsequent output image is smoother, so as to further improve the image display effect.

[0114] On the basis of the above embodiment, it needs to be explained that the image processing system in the embodiment can be an endoscope imaging system, the first light source 12 in the image processing system is a visible light source, and the second light source 13 is a near-infrared light source. Correspondingly, the shooting result of the first light source 12 is a visible light image, and the shooting result of the second light source 13 is a near-infrared light image.

[0115] In actual application, in order to clearly observe the lesion site in the patient's body, so as to accurately perform surgical treatment on the lesion site, endoscope technology appears. The endoscope can expand the surgical field of view, so that the doctor can more clearly observe the image of the lesion site during the operation. In order to accurately process the lesion site during the operation, but at the same time, it is necessary to avoid cutting other tissues around the lesion site, so it is necessary to obtain the morphological information of the lesion site and the metabolic information of the lesion site.

[0116] Specifically, by injecting indocyanine green to the lesion site, and then using a near-infrared light source with a wavelength of 780-811 nm to emit near-infrared light to the lesion site, and capturing the light of the shooting object, a near-infrared light image of the lesion site is obtained, so as to obtain the metabolic information of the lesion site.

[0117] Specifically, by using a visible light source with a wavelength of 400-650 nm to emit visible light to the lesion site, and capturing the light of the shooting object to obtain a visible light image of the lesion site, the morphological information of the lesion site is obtained.

[0118] On the basis of the above embodiment, the first light source in the embodiment can also be a near-infrared light source, and the second light source is a white light source. Correspondingly, the shooting result of the first light source is a near-infrared light image, and the shooting result of the second light source is a visible light image. That is, in actual application, the first frame of the near-infrared light is shot to obtain the metabolic information of the lesion site, the second frame of the visible light is shot to obtain the morphological information of the lesion site, and then the image fusion is performed to obtain the fusion information.

[0119] It should be noted that the embodiment does not limit the shooting priority order of the two light sources on the shooting object.

[0120] Further, the visible light image at each time, the previous frame near-infrared light image and the next frame near-infrared light image of the visible light image are acquired, the two frames of near-infrared light images are interpolated to obtain an interpolated near-infrared light image, and then the interpolated near-infrared light image and the visible light image are image fused to obtain a fused image, and the previous frame visible light image and the next frame visible light image of the near-infrared light image are acquired, the previous frame visible light image and the next frame visible light image are image interpolated to obtain an interpolated visible light image, and the interpolated visible light image and the infrared light image are fused to obtain a fused image, and then the fused images are output, thereby solving the problem that the fused image obtained in the prior art has image ghosting, resulting in poor output fused image effect, and improving the image quality of the fused image.

[0121] Figure 11 The embodiment of the present application also provides a flowchart of an image processing method, which can be applied to the case of image processing of a photographed image to generate a fused image. The method can be executed by an image processing device 2, which can be realized in the form of hardware and / or software and can be configured in an image processing system. As shown in the figure, the method comprises the following steps. Figure 11

[0122] S110, a plurality of to-be-processed image groups are acquired; wherein the to-be-processed image group comprises a first image, a second image and a third image.

[0123] S120, for any to-be-processed image group, the third image and the first image in the current to-be-processed image group are image interpolated to obtain an interpolated image of the current to-be-processed image group.

[0124] S130, the interpolated image and the second image are image fused to obtain a target fused image corresponding to the current to-be-processed image group.

[0125] The image processing method provided by the embodiment of the present application can obtain an interpolated image containing morphological information by image interpolating the first frame to-be-processed image and the third frame to-be-processed image containing morphological information, and then fuse the interpolated image containing morphological information and the second frame to-be-processed image containing metabolic information to obtain a target fusion result, thereby solving the problem that the fused image obtained in the prior art has image ghosting, resulting in poor output fused image effect, and improving the image quality of the fused image.

[0126] ​The image processing method provided by the embodiment of the present application can be executed by the image processing system provided by any embodiment of the present application, and the image processing system has the corresponding function modules and beneficial effects of the system generation method.

[0127] Figure 12 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0128] As shown in Figure 12 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0129] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, a loudspeaker, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0130] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, and the like. The processor 11 performs various methods and processes described above, such as an image processing method.

[0131] In some embodiments, the image processing method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the image processing method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the image processing method by any other suitable means, such as by means of firmware.

[0132] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, input devices, and output devices.

[0133] Computer programs used to implement the image processing method of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor, implements the functions / operations specified in the flow charts and / or block diagrams. The computer program can execute entirely on a machine, partly on a machine, partly on a machine and partly on a remote machine or entirely on a remote machine or server.

[0134] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0135] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0136] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0137] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0138] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0139] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An image processing system, characterized in that, include: Image generation apparatus and image processing apparatus; wherein, The image generation device is used to continuously photograph the subject using two light sources to generate multiple image groups to be processed; wherein, the image group to be processed includes a first image, a second image, and a third image, the first image and the third image are the results of shooting with the first light source, and the second image is the result of shooting with the second light source; The image processing device is used to acquire multiple groups of images to be processed; for any group of images to be processed, the third image and the first image in the current group of images to be processed are subjected to image interpolation processing to obtain an interpolated image of the current group of images to be processed; the interpolated image and the second image are subjected to image fusion processing to obtain a target fused image corresponding to the current group of images to be processed.

2. The system according to claim 1, characterized in that, The image generation device includes a light source controller, a first light source, a second light source, and a light capturing device; The light source controller is used to project light of different wavelengths in adjacent frames and light of the same wavelength in every other frame. The first light source is used to project light of a first wavelength onto the object being photographed under the control of the light source controller; The second light source is used to project a second band of light onto the object being photographed under the control of the light source controller; The light-capturing device is used to capture light on the subject and generate an image to be processed.

3. The system according to claim 1, characterized in that, The image processing device includes a first memory, a second memory, and a processor; the first memory includes k first logic units, and the second memory includes k second logic units; k is a positive integer greater than a preset threshold. The first logic unit is used to store the image capture result of the first light source generated by the image generation device; The second logic unit is used to store the image capture result of the second light source generated by the image generation device; The processor is configured to perform the steps of acquiring multiple groups of images to be processed in the image processing device, and performing image fusion on the multiple groups of images to be processed to generate target fused images corresponding to the multiple groups of images to be processed respectively.

4. The system according to claim 3, characterized in that, The processor includes a module for acquiring a group of images to be processed, a module for acquiring interpolated images, and a module for acquiring a target fused image; wherein... The image group acquisition module is used to acquire multiple image groups to be processed; The interpolated image acquisition module is used to perform image interpolation processing on the third image and the first image in the current image group to be processed for any image group to be processed, so as to obtain the interpolated image of the current image group to be processed. The target fusion image acquisition module is used to perform image fusion processing on the interpolated image and the second image to obtain the target fusion image corresponding to the current image group to be processed.

5. The system according to claim 4, characterized in that, The interpolated image acquisition module includes an image feature acquisition unit, a feature offset determination unit, and an interpolated image acquisition unit; wherein... The image feature acquisition unit is used to acquire the first image feature corresponding to the first image and the third image feature corresponding to the third image, respectively. The feature offset determination unit is used to perform feature convolution processing on the third image feature and the first image feature to obtain the feature offset of the current image group to be processed. The interpolated image acquisition unit is used to perform image interpolation on the third image and the first image based on the feature offset to obtain the interpolated image of the current image group to be processed.

6. The system according to claim 5, characterized in that, The interpolated image acquisition unit includes a third offset image acquisition subunit, a first offset image acquisition subunit, and an interpolated image acquisition subunit; wherein... The third offset image acquisition subunit is used to perform image offset on the third image based on the feature offset to obtain the third offset image corresponding to the third image; The first offset image acquisition subunit is used to perform image offset on the first image based on the feature offset to obtain the first offset image corresponding to the first image; The interpolated image acquisition subunit is used to obtain preset interpolation parameters, and perform interpolation processing on the first offset image and the third offset image based on the interpolation parameters to obtain the interpolated image of the current image group to be processed.

7. The system according to claim 3, characterized in that, The processor further includes an image acquisition time determination module and an image output time determination module; wherein... The image acquisition time determination module is used to obtain the image acquisition time of the third image in any of the image groups to be processed. The image output time determination module is used to determine the image output time of the target fused image based on the image acquisition time.

8. The system according to claim 1, characterized in that, If the image group to be processed is the first image group to be processed, then the first image is output before the target fused image is output.

9. The system according to any one of claims 1-8, characterized in that, The image processing system is an endoscopic imaging system. Accordingly, the first light source in the image processing system is a visible light source and the second light source is a near-infrared light source, or the first light source is a near-infrared light source and the second light source is a white light source.

10. An image processing method, characterized in that, include: Multiple image groups to be processed are acquired; wherein, the image groups to be processed include a first image, a second image, and a third image, the first image and the third image are the results of shooting with a first light source, and the second image is the result of shooting with a second light source; For any group of images to be processed, perform image interpolation on the third image and the first image in the current group of images to be processed to obtain the interpolated image of the current group of images to be processed. The interpolated image and the second image are fused together to obtain the target fused image corresponding to the current image group to be processed.

Images