Color field sequential imaging method and system based on guided interpolation

Abstract

The present invention discloses a color field sequential imaging method and system based on guided interpolation, which is applied to the field of image intelligent imaging technology. The method includes: irradiating the target area with a light source according to the light source timing and collecting images; constructing a motion equation; interpolating through the motion equation to obtain an interpolated image of the moment when the color light source is not irradiated between the corresponding moments of two collected images; superimposing the interpolated image and the collected image at the same moment to form an imaging image. While maintaining the size and structure of the existing electronic endoscope unchanged, the present invention achieves the purpose of having a high frame rate at a high resolution through a new light source control, image interpolation algorithm and synchronous exposure control. Compared with the traditional Bayer imaging chip, it can obtain a higher resolution and clearer color and texture details; compared with the color field sequential imaging chip, it can obtain a higher frame rate and a smoother picture.

Description

Technical Field

[0001] The present invention relates to the technical field of image intelligent imaging, and in particular to a color field sequential imaging method and system based on guided frame insertion. Background Art

[0002] The pixels of traditional color image sensors (such as CMOS sensors) are arranged in Bayer format, and R, G, and B represent red, green, and blue pixels respectively. Assuming that the number of pixels of the image sensor is M*N, M is the number of rows, and N is the number of columns, then according to the arrangement of the pixels of the sensor, the number of R pixels in the pixels output by the sensor is M*N / 4, the number of G pixels is M*N / 2, and the number of B pixels is M*N / 4. Then, the 3-channel RGB image is reconstructed through the interpolation algorithm. Due to the existence of the interpolation algorithm, some errors may be introduced in the process of estimating the missing color components, resulting in reduced clarity of image details, especially at the color edges. In addition, the Bayer pattern is sensitive to light conditions. In the case of insufficient light or large changes in light, problems such as color distortion or increased noise may occur.

[0003] Another type of sensor uses a black and white CMOS imaging chip, which cyclically emits three monochromatic light sources of red, green and blue to illuminate the target. The chip is synchronously exposed with the light source to obtain the red, green and blue monochromatic light image signals of the target scene, and finally synthesizes a color image. This method can accurately capture and represent colors, thereby providing a higher color resolution. However, since images of different color components need to be captured sequentially in time, the imaging speed is limited. If the object moves during the imaging process, it may cause the images of different color components to be misaligned or blurred, affecting the imaging quality. Since images of multiple color components need to be captured, the amount of data generated may be very large, which places high demands on storage and transmission. In summary, how to provide a solution with high-definition real-time imaging capability to improve the clarity and frame rate of the output image has become a problem to be solved by those skilled in the art. Summary of the invention

[0004] In order to at least overcome the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide a color field sequential imaging method and system based on guided interpolation.

[0005] In a first aspect, the present application provides a color field sequential imaging method based on guided interpolation, characterized in that it includes:

[0006] The target area is illuminated by light according to the light sequence, and an image at each moment in the light sequence is collected as a collected image; each moment in the light sequence corresponds to one light illumination, and the moments of illumination with the first color light are separated by an interval moment; the second color light source and the third color light source are alternately illuminated in the adjacent interval moments;

[0007] Constructing a motion equation based on two adjacent images captured at the time of illumination by the first color light source;

[0008] Interpolating two adjacent captured images corresponding to the same color light source at the same time by using the motion equation to obtain an interpolated image of the time when the color light source is not irradiated between the corresponding time of the two captured images;

[0009] The interpolation image and the acquisition image at the same time are superimposed to form an imaging image.

[0010] In a possible implementation, collecting an image at each moment in the light source time sequence as a collected image includes:

[0011] When the first color is green, the second color is red, and the third color is blue, collecting an image at each moment in the light source time sequence as a collected image;

[0012] When the first color is magenta, the second color is cyan, and the third color is yellow, an image at each moment in the light source sequence is collected and the colors are inverted to form the collected image.

[0013] In a possible implementation, constructing the motion equation according to two adjacent captured images at the time of illumination by the first color light source includes:

[0014] Acquire a captured image at a moment of irradiation with a first color light source as a first captured image, and acquire a captured image at a moment of irradiation with a first color light source next to the first captured image as a second captured image;

[0015] The motion of the pixel change from the first acquired image to the second acquired image is estimated by the optical flow method to form a motion equation corresponding to the interval time between the first acquired image and the second acquired image.

[0016] In a possible implementation, interpolating two adjacent captured images corresponding to the same color light source using the motion equation to obtain an interpolated image at a time when the two captured images are not illuminated by the color light source at the corresponding time includes:

[0017] When interpolating the collected image illuminated by the first color light source, the motion equation corresponding to the interval time is selected to perform image interpolation to form an interpolated image corresponding to the interval time;

[0018] When interpolating the acquired images illuminated by the second color light source or the third color light source:

[0019] When the time when the color light source is not irradiated is an interval time, the motion equation corresponding to the interval time is selected to perform image interpolation to form an interpolation image corresponding to the interval time;

[0020] When the moment when the color light source is not irradiated is the next moment or the previous moment of the corresponding moment of the collected image, the motion equation corresponding to the interval moment corresponding to the collected image is selected to perform image interpolation to form an interpolated image corresponding to the moment.

[0021] In a possible implementation manner, acquiring the interpolation image includes:

[0022] When the moment when the color light source is not irradiated is the next moment of the corresponding moment of the collected image, the collected image is interpolated to the next moment by using the selected motion equation to obtain the interpolated image;

[0023] When the moment when the color light source is not irradiated is the previous moment of the corresponding moment of the collected image, the collected image is interpolated to the previous moment by using the selected motion equation to obtain the interpolated image;

[0024] When the time when the color light source is not irradiated is the interval time, the interpolated image is obtained by interpolating the collected image closest to the interval time to the interval time through the selected motion equation.

[0025] In a possible implementation manner, superimposing the interpolation image and the acquisition image at the same time to form an imaging image includes:

[0026] Registering all the interpolated images at the same moment to the acquired image at that moment;

[0027] The registered interpolation image and the acquired image are synthesized according to the Bayer pattern to form an image to be processed whose resolution is twice that of the interpolation image;

[0028] A demosaicing operation is performed on the image to be processed to form the three-channel imaging image.

[0029] In a second aspect, the present application also provides a color field sequential imaging system based on guided interpolation, comprising:

[0030] The acquisition unit is configured to illuminate the target area with a light source according to a light source timing sequence, and to acquire an image at each moment in the light source timing sequence as an acquired image; each moment in the light source timing sequence corresponds to a light source illumination, and the moments of illumination with the first color light source are separated by an interval moment; in adjacent interval moments, the second color light source and the third color light source are illuminated alternately in sequence;

[0031] A construction unit, configured to construct a motion equation according to two adjacent images collected at the time of illumination by the first color light source;

[0032] An interpolation unit is configured to interpolate two adjacent captured images corresponding to the same color light source through the motion equation to obtain an interpolated image of a moment when the color light source is not irradiated between the two captured images corresponding to the moment;

[0033] The superposition unit is configured to superimpose the interpolation image and the acquisition image at the same time to form an imaging image.

[0034] In a possible implementation, the acquisition unit is further configured to:

[0035] When the first color is green, the second color is red, and the third color is blue, an image at each moment in the light source time sequence is collected as a collected image

[0036] When the first color is magenta, the second color is cyan, and the third color is yellow, an image at each moment in the light source sequence is collected and the colors are inverted to form the collected image.

[0037] In a possible implementation, the construction unit is further configured to:

[0038] Acquire a captured image at a moment of irradiation with a first color light source as a first captured image, and acquire a captured image at a moment of irradiation with a first color light source next to the first captured image as a second captured image;

[0039] The motion of the pixel change from the first acquired image to the second acquired image is estimated by the optical flow method to form a motion equation corresponding to the interval time between the first acquired image and the second acquired image.

[0040] In a possible implementation manner, the superposition unit is further configured as follows:

[0041] Superimposing the interpolation image and the acquisition image at the same time to form an imaging image includes:

[0042] Registering all the interpolated images at the same moment to the acquired image at that moment;

[0043] The registered interpolation image and the acquired image are synthesized according to the Bayer pattern to form an image to be processed whose resolution is twice that of the interpolation image;

[0044] A demosaicing operation is performed on the image to be processed to form the three-channel imaging image.

[0045] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0046] While maintaining the size and structure of the existing electronic endoscope unchanged, the present invention achieves the goal of having a high frame rate at a high resolution through a new light source control, image interpolation algorithm and synchronous exposure control. Compared with the traditional Bayer imaging chip, it can obtain a higher resolution and clearer color and texture details; compared with the color field sequential imaging chip, it can obtain a higher frame rate and a smoother picture. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0048] Figure 1 This is a schematic diagram of the steps of the method of the embodiment of the present application;

[0049] Figure 2 This is a schematic diagram of the process in the case of primary color light source in Example 3 of the present application;

[0050] Figure 3 This is a schematic diagram of the process in the case of a subtractive three-primary color light source in an embodiment of the present application;

[0051] Figure 4 This is a schematic diagram of a 4K three-channel embodiment of the present application;

[0052] Figure 5 This is a schematic diagram of the method steps of an embodiment of the present application. DETAILED DESCRIPTION

[0053] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the embodiment of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0054] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0055] Please refer to Figure 1 , which is a flow chart of a color field sequential imaging method based on guided interpolation provided in an embodiment of the present invention. Furthermore, the color field sequential imaging method based on guided interpolation may specifically include the contents described in the following steps S1 to S4.

[0056] S1: irradiating the target area with light according to the light sequence, and collecting images at each moment in the light sequence as collected images; each moment in the light sequence corresponds to one light irradiation, and the moments of irradiation with the first color light source are separated by an interval moment; in adjacent interval moments, the second color light source and the third color light source are irradiated alternately in sequence;

[0057] S2: constructing a motion equation according to two adjacent images captured at the time of illumination by the first color light source;

[0058] S3: interpolating two adjacent captured images corresponding to the same color light source using the motion equation to obtain an interpolated image of the time when the color light source is not irradiated between the two captured images;

[0059] S4: superimposing the interpolation image and the acquisition image at the same time to form an imaging image.

[0060] When the embodiment of the present application is implemented, the applicant's application number is 202411124987.8. The prior application provides a method of achieving clear imaging of moving objects by interpolation and superposition. However, in scientific practice, the applicant has found that when the images corresponding to each color are interpolated separately, due to the interpolation error, the superimposed image is still prone to blurring. Based on this, in the embodiment of the present application, the ratio of the first color light source, the second color light source, and the third color light source is selected as 2:1:1, wherein the interval between the irradiation moments of the first color light source is one moment, that is, the interval moment, and the second color light source and the third color light source are distributed in these interval moments in turn, and each interval moment is irradiated by the second color light source or the third color light source. At this time, in the sampled acquired images, the acquired images when the first color light source is irradiated are all separated by an interval moment, and the acquired images when the second color light source and the third color light source are irradiated are all separated by three moments. For example, taking red light source, green light source and blue light source as an example, the ratio of green light source, red light source and blue light source is 2:1:1, and the light source cycle can be carried out in the following ways: green-red-green-blue, green-blue-green-red, red-green-blue-green, blue-green-red-green, etc. The sensor is exposed and imaged according to the same clock signal as the light source.

[0061] In an embodiment of the present application, the motion equation between the image frames corresponding to two moments can be constructed through the captured images of the first color light source. The construction process is based on the motion estimation of the pixels between the two images, and the optical flow method is generally used for estimation. Since the target motion state is not affected by color, the motion equation constructed through the captured images corresponding to the first color light source in the present application can be applied to the interpolation of the captured images of the other two color light sources, so that the motion equations of the images corresponding to the three colors can be unified, thereby avoiding the blurring of the images after superposition. After the interpolation is completed, the corresponding image processing is performed, and the interpolated images or captured images corresponding to the three colors at the same moment can be superimposed to form the final imaging image.

[0062] In a possible implementation, collecting an image at each moment in the light source time sequence as a collected image includes:

[0063] When the first color is green, the second color is red, and the third color is blue, collecting an image at each moment in the light source time sequence as a collected image;

[0064] When the first color is magenta, the second color is cyan, and the third color is yellow, an image at each moment in the light source sequence is collected and the colors are inverted to form the collected image.

[0065] When the embodiment of the present application is implemented, as a preferred technical solution, the three primary colors are used as the light source for illumination. The inventors found that for the same target area, the image generated by the green light source is more sensitive to motion, so using green as the first color can effectively improve the accuracy of the subsequent motion equation. As another preferred technical solution, the three subtractive primary colors are used for light source illumination. The inventors found in scientific practice that the illumination intensity of monochromatic light is lower than the illumination intensity of composite light, and the use of the following three subtractive primary colors can effectively improve this situation: Cyan: a mixture of blue light and green light. Magenta: a mixture of red light and blue light. Yellow: a mixture of red light and green light. Such light intensity is stronger, the sensor signal is stronger, and low image clarity is avoided at low illumination. At the same time, when the collected image is formed by the three subtractive primary colors, the collected image needs to be inverted, wherein the acquired magenta image is inverted to obtain a green image, the acquired cyan image is inverted to obtain a red image, and the acquired yellow image is inverted to obtain a blue image.

[0066] In a possible implementation, constructing the motion equation according to two adjacent captured images at the time of illumination by the first color light source includes:

[0067] Acquire a captured image at a moment of irradiation with a first color light source as a first captured image, and acquire a captured image at a moment of irradiation with a first color light source next to the first captured image as a second captured image;

[0068] The motion of the pixel change from the first acquired image to the second acquired image is estimated by the optical flow method to form a motion equation corresponding to the interval time between the first acquired image and the second acquired image.

[0069] When implementing the embodiment of the present application, the optical flow method is used to generate the motion equation, such as the Lucas-Kanade algorithm, and the motion equation is generated by using two adjacent frames of captured images illuminated by the first color. Since the optical flow method actually fits the motion trajectory of the object corresponding to the pixel point, the motion equation between the first captured image and the second captured image actually represents the path of each pixel point in the first captured image moving from the first captured image to the second captured image. The generation process is a relatively mature technology, and the embodiment of the present application does not make much limitation. Since there is only one moment between the first captured image and the second captured image, namely the interval moment, it is necessary to correspond the interval moment to the motion equation at this time for subsequent use.

[0070] In a possible implementation, interpolating two adjacent captured images corresponding to the same color light source using the motion equation to obtain an interpolated image at a time when the two captured images are not illuminated by the color light source at the corresponding time includes:

[0071] When interpolating the collected image illuminated by the first color light source, the motion equation corresponding to the interval time is selected to perform image interpolation to form an interpolated image corresponding to the interval time;

[0072] When interpolating the acquired images illuminated by the second color light source or the third color light source:

[0073] When the time when the color light source is not irradiated is an interval time, the motion equation corresponding to the interval time is selected to perform image interpolation to form an interpolation image corresponding to the interval time;

[0074] When the moment when the color light source is not irradiated is the next moment or the previous moment of the corresponding moment of the collected image, the motion equation corresponding to the interval moment corresponding to the collected image is selected to perform image interpolation to form an interpolated image corresponding to the moment.

[0075] When the embodiment of the present application is implemented, during the image interpolation process, since there is a moment between the captured images illuminated by the first color light source, namely the interval moment, the motion equation corresponding to the interval moment can be directly selected for interpolation. For the captured images corresponding to the second color light source and the third color light source, there are three moments between every two captured images, so there will be three situations when interpolation is required, namely, the moment requiring interpolation is the next second of the captured image, the moment requiring interpolation is the interval moment, and the moment requiring interpolation is the previous second of the captured image. For the moment requiring interpolation is the interval moment, it is only necessary to select the motion equation corresponding to the interval moment for interpolation, and when the moment when the color light source is not irradiated is the next moment or the previous moment of the corresponding moment of the captured image, it is necessary to select the motion equation corresponding to the interval moment where the captured image is located for interpolation. In this way, the motion equation that is most suitable for the moment requiring interpolation can be found for interpolation, which effectively improves the accuracy of the interpolation calculation.

[0076] In a possible implementation manner, acquiring the interpolation image includes:

[0077] When the moment when the color light source is not irradiated is the next moment of the corresponding moment of the collected image, the collected image is interpolated to the next moment by using the selected motion equation to obtain the interpolated image;

[0078] When the moment when the color light source is not irradiated is the previous moment of the corresponding moment of the collected image, the collected image is interpolated to the previous moment by using the selected motion equation to obtain the interpolated image;

[0079] When the time when the color light source is not irradiated is the interval time, the interpolated image is obtained by interpolating the collected image closest to the interval time to the interval time through the selected motion equation.

[0080] In a possible implementation manner, superimposing the interpolation image and the acquisition image at the same time to form an imaging image includes:

[0081] Registering all the interpolated images at the same moment to the acquired image at that moment;

[0082] The registered interpolation image and the acquired image are synthesized according to the Bayer pattern to form an image to be processed whose resolution is twice that of the interpolation image;

[0083] A demosaicing operation is performed on the image to be processed to form the three-channel imaging image.

[0084] When the embodiment of the present application is implemented, during the superposition process, since there is a captured image captured at each moment, it is necessary to align the interpolated image to the captured image to improve the imaging accuracy. The specific alignment method uses point features or line features for image alignment, which belongs to the prior art and is not limited in the embodiment of the present application. After the alignment is completed, the interpolated image and the captured image need to be superimposed according to the Bayer pattern to form a to-be-processed image with twice the resolution. For example, when RGB is selected as the three colors, the red, green, and blue pixels at each time point are combined according to one of the modes of RG / GB, BG / GR, GR / BG, and GB / RG during the superposition process to obtain an image with both length and width being the original image. Figure 2 After the superposition is completed, the demosaicing process can be performed to form a three-channel color imaging image.

[0085] In a possible implementation, take RGB as an example for three colors:

[0086] See also Figure 2 When applied to endoscope lens technology, the light source is switched every 10ms in the order of green, red, green, and blue. At this time, a sensor with a resolution of 2K is used to synchronize with the light source, and imaging is performed every 10ms to obtain 2K images of green, red, green, and blue in sequence. The green image is used as a reference image and the optical flow algorithm is used for motion estimation. The Lucas-Kanade algorithm can be used here to calculate the inter-frame motion equation using two adjacent green frames of images. The motion equation f1(x, y) is calculated by G1 and G2, the motion equation f2(x, y) is calculated by G2 and G3, and the motion equation f3(x, y) is calculated by G3 and G4, and so on.

[0087] When interpolating the green channel: using G1 and G2 as references, use the motion equation f1(x, y) to interpolate to obtain Gb; using G2 and G3 as references, use the motion equation f2(x, y) to interpolate to obtain Gd; using G3 and G4 as references, use the motion equation f3(x, y) to interpolate to obtain Gf; and so on;

[0088] When interpolating the red channel: take R1 and R3 as references, use the motion equation f1(x, y) to interpolate to get Rc; take R1 and R3 as references, use the motion equation f2(x, y) to interpolate to get Rd; take R1 and R3 as references, use the motion equation f3(x, y) to interpolate to get Re; and so on;

[0089] When interpolating the blue channel: using B2 and B4 as references, use the motion equation f2(x,y) to interpolate to get Be; using B2 and B4 as references, use the motion equation f3(x,y) to interpolate to get Bf; using B2 and B4 as references, use the motion equation f4(x,y) to interpolate to get Bg; and so on;

[0090] There are no complete red, green and blue three-channel images at the first three time points, and it is impossible to synthesize a three-channel color image, so no subsequent processing is performed; the registration starts from the fourth time point, taking B2 as a reference, Gd and Rd are registered to B2; taking G3 as a reference, Ge and Re are registered to G3; taking R3 as a reference, Gf and Bf are registered to R3; and so on; the registered red image, green image, and blue image are synthesized according to the Bayer mode, and the red, green, and blue pixels at each time point are combined in the GR / BG mode to obtain a 4K Bayer image; the synthesis method is as follows Figure 4 Finally, the 4k Bayer image is demosaiced. The NEDI algorithm can be used here to obtain a 4K three-channel color image, which is finally formed as shown below. Figure 5 Image shown.

[0091] In a possible implementation, take the three subtractive primary colors as an example:

[0092] See also Figure 3 When applied to endoscope lens technology, the light source is switched every 10ms in the order of magenta, cyan, magenta, and yellow; a sensor with a resolution of 2K is used to synchronize with the light source, and imaging is performed every 10ms to obtain 2K images of magenta, cyan, magenta, and yellow in sequence; the acquired magenta image is inverted to obtain a green image, the acquired cyan image is inverted to obtain a red image, and the acquired yellow image is inverted to obtain a blue image, thereby obtaining a 2K image in a green, red, green, and blue cycle; the subsequent processing process is consistent with the process of the above embodiment, and the embodiment of the present application will not be repeated.

[0093] Based on the same inventive concept, the present application also provides a color field sequential imaging system based on guided interpolation, comprising:

[0094] The acquisition unit is configured to illuminate the target area with a light source according to a light source timing sequence, and to acquire an image at each moment in the light source timing sequence as an acquired image; each moment in the light source timing sequence corresponds to a light source illumination, and the moments of illumination with the first color light source are separated by an interval moment; in adjacent interval moments, the second color light source and the third color light source are illuminated alternately in sequence;

[0095] A construction unit, configured to construct a motion equation according to two adjacent images collected at the time of illumination by the first color light source;

[0096] An interpolation unit is configured to interpolate two adjacent captured images corresponding to the same color light source through the motion equation to obtain an interpolated image of a moment when the color light source is not irradiated between the two captured images corresponding to the moment;

[0097] The superposition unit is configured to superimpose the interpolation image and the acquisition image at the same time to form an imaging image.

[0098] In a possible implementation, the acquisition unit is further configured to:

[0099] When the first color is green, the second color is red, and the third color is blue, an image at each moment in the light source time sequence is collected as a collected image

[0100] When the first color is magenta, the second color is cyan, and the third color is yellow, an image at each moment in the light source sequence is collected and the colors are inverted to form the collected image.

[0101] In a possible implementation, the construction unit is further configured to:

[0102] Acquire a captured image at a moment of irradiation with a first color light source as a first captured image, and acquire a captured image at a moment of irradiation with a first color light source next to the first captured image as a second captured image;

[0103] The motion of the pixel change from the first acquired image to the second acquired image is estimated by the optical flow method to form a motion equation corresponding to the interval time between the first acquired image and the second acquired image.

[0104] In a possible implementation manner, the superposition unit is further configured as follows:

[0105] Superimposing the interpolation image and the acquisition image at the same time to form an imaging image includes:

[0106] Registering all the interpolated images at the same moment to the acquired image at that moment;

[0107] The registered interpolation image and the acquired image are synthesized according to the Bayer pattern to form an image to be processed whose resolution is twice that of the interpolation image;

[0108] A demosaicing operation is performed on the image to be processed to form the three-channel imaging image.

[0109] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0110] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0111] The units described as separate components may or may not be physically separated. As units, it is obvious that a person of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0112] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a grid device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0114] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A color field sequential imaging method based on guided interpolation, characterized in that: include: The target area is illuminated by light according to the light sequence, and an image at each moment in the light sequence is collected as a collected image; each moment in the light sequence corresponds to one light illumination, and the moments of illumination with the first color light are separated by an interval moment; the second color light source and the third color light source are alternately illuminated in the adjacent interval moments; Constructing a motion equation based on two adjacent images captured at the time of being illuminated by the first color light source; Interpolating two adjacent captured images corresponding to the same color light source at the same time by using the motion equation to obtain an interpolated image of the time when the color light source is not irradiated between the corresponding time of the two captured images; Superimposing the interpolation image and the acquisition image at the same time to form an imaging image; Constructing the motion equation based on the captured images at two adjacent moments illuminated by the first color light source includes: Acquire a captured image at a moment of irradiation with a first color light source as a first captured image, and acquire a captured image at a moment of irradiation with a first color light source next to the first captured image as a second captured image; Performing motion estimation on pixel changes from the first acquired image to the second acquired image by using an optical flow method to form a motion equation corresponding to an interval time between the first acquired image and the second acquired image; Interpolating two adjacent captured images corresponding to the same color light source by using the motion equation to obtain an interpolated image of the time when the two captured images are not illuminated by the color light source at the corresponding time includes: When interpolating the collected image illuminated by the first color light source, the motion equation corresponding to the interval time is selected to perform image interpolation to form an interpolated image corresponding to the interval time; When interpolating the acquired images illuminated by the second color light source or the third color light source: When the time when the color light source is not irradiated is an interval time, the motion equation corresponding to the interval time is selected to perform image interpolation to form an interpolation image corresponding to the interval time; When the moment when the color light source is not irradiated is the next moment or the previous moment of the corresponding moment of the collected image, the motion equation corresponding to the interval moment corresponding to the collected image is selected to perform image interpolation to form an interpolated image corresponding to the moment.

2. The color field sequential imaging method based on guided interpolation according to claim 1, characterized in that: Collecting the image at each moment in the light source time sequence as the collected image includes: When the first color is green, the second color is red, and the third color is blue, collecting an image at each moment in the light source time sequence as a collected image; When the first color is magenta, the second color is cyan, and the third color is yellow, an image at each moment in the light source sequence is collected and the colors are inverted to form the collected image.

3. The color field sequential imaging method based on guided interpolation according to claim 1, characterized in that: The acquisition of the interpolation image includes: When the moment when the color light source is not irradiated is the next moment of the corresponding moment of the collected image, the collected image is interpolated to the next moment by using the selected motion equation to obtain the interpolated image; When the moment when the color light source is not irradiated is the previous moment of the corresponding moment of the collected image, the collected image is interpolated to the previous moment by using the selected motion equation to obtain the interpolated image; When the time when the color light source is not irradiated is the interval time, the interpolated image is obtained by interpolating the collected image closest to the interval time to the interval time through the selected motion equation.

4. The color field sequential imaging method based on guided interpolation according to claim 1, characterized in that: Superimposing the interpolation image and the acquisition image at the same time to form an imaging image includes: Registering all the interpolated images at the same moment to the acquired image at that moment; The registered interpolation image and the acquired image are synthesized according to the Bayer pattern to form an image to be processed whose resolution is twice that of the interpolation image; A demosaicing operation is performed on the image to be processed to form the three-channel imaging image.

5. A color field sequential imaging system based on guided interpolation using the method according to any one of claims 1 to 4, characterized in that: include: The acquisition unit is configured to illuminate the target area with a light source according to a light source timing sequence, and to acquire an image at each moment in the light source timing sequence as an acquired image; each moment in the light source timing sequence corresponds to a light source illumination, and the moments of illumination with the first color light source are separated by an interval moment; in adjacent interval moments, the second color light source and the third color light source are illuminated alternately in sequence; A construction unit, configured to construct a motion equation according to two adjacent images collected at the time of illumination by the first color light source; An interpolation unit is configured to interpolate two adjacent captured images corresponding to the same color light source through the motion equation to obtain an interpolated image of a moment when the color light source is not irradiated between the two captured images corresponding to the moment; The superposition unit is configured to superimpose the interpolation image and the acquisition image at the same time to form an imaging image.

6. The color field sequential imaging system based on guided interpolation according to claim 5, characterized in that: The acquisition unit is further configured to: When the first color is green, the second color is red, and the third color is blue, collecting an image at each moment in the light source time sequence as a collected image; When the first color is magenta, the second color is cyan, and the third color is yellow, an image at each moment in the light source sequence is collected and the colors are inverted to form the collected image.

7. The color field sequential imaging system based on guided interpolation according to claim 5, characterized in that: The construction unit is further configured to: Acquire a captured image at a moment of irradiation with a first color light source as a first captured image, and acquire a captured image at a moment of irradiation with a first color light source next to the first captured image as a second captured image; The motion of the pixel change from the first acquired image to the second acquired image is estimated by the optical flow method to form a motion equation corresponding to the interval time between the first acquired image and the second acquired image.

8. The color field sequential imaging system based on guided interpolation according to claim 5, characterized in that: The superposition unit is further configured as: Superimposing the interpolation image and the acquisition image at the same time to form an imaging image includes: Registering all the interpolated images at the same moment to the acquired image at that moment; The registered interpolation image and the acquired image are synthesized according to the Bayer pattern to form an image to be processed whose resolution is twice that of the interpolation image; A demosaicing operation is performed on the image to be processed to form the three-channel imaging image.

Images