Dual-spectrum imaging control system and method, storage medium and device

Through the dual-spectral imaging control system, the multi-spectral light source and image processing technology are used to realize real-time comparison of multiple spectral imaging of the endoscope, solving the problem of only single spectral imaging in the existing technology, and improving the diagnostic ability of early cancer .

CN118924213APending Publication Date: 2024-11-12ZHUHAI SHIXIN MEDICAL TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410892625.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-04
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing endoscopes can only display one spectral imaging at the same time, and cannot compare multiple spectral imaging in real time, resulting in difficulty in diagnosing early cancers in flatness.

Method used

A dual-spectral imaging control system was designed to emit light in various different modes of light via multi-spectral light sources. After the image sensor acquires the image signal, the signal conversion module copies and processes it into multiple signals. The main control module builds a multi-channel image processing unit. , remove even or odd frames and perform splicing to achieve real-time display of multiple spectral imaging.

Benefits of technology

It realizes multiple spectral imaging and displays simultaneously, which facilitates real-time comparison by doctors and improves the diagnostic efficiency and accuracy of the lesions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118924213A_ABST
    Figure CN118924213A_ABST
Patent Text Reader

Abstract

The invention discloses a dual-spectrum imaging control system and method, a storage medium and a device, and relates to the technical field of endoscopes. The system comprises a multispectral light source, an image sensor, a signal conversion module, a main control module and a display module, the multispectral light source can emit light in various different modes; the image sensor is used for acquiring a first image signal under the irradiation of the multispectral light source; the signal conversion module is used for copying the first image signal into a plurality of second image signals; the main control module comprises a plurality of image processing units, a plurality of video output units, an image splicing unit and an output port. According to the system provided by the embodiment of the invention, images of multiple spectrums can be presented at the same time, endoscope images of multiple different spectrums can be presented in real time, doctors can conveniently compare, observe and find focuses, and diagnosis and treatment of lesions are facilitated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of endoscopes, and in particular to a dual-spectrum imaging control system and method, a storage medium, and a device. Background Art

[0002] Ordinary endoscopes can detect lesions with obvious changes in the shape or color of diseased tissues, but cannot show the shape of the glandular duct openings of diseased tissues and subtle changes in blood vessels at all levels, and are difficult to diagnose flat early cancers. However, narrow-band light imaging endoscopes can show the superficial capillaries of the digestive tract mucosa and the collective large blood vessels in the deep mucosa.

[0003] However, endoscopes with narrow-band light imaging on the market can only display one spectral imaging at the same time. When doctors need to compare other spectral imaging, they need to switch to other spectral imaging. The two spectral imaging cannot be compared in real time, which is not conducive to doctor examination. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a dual-spectrum imaging control system and method, storage medium, and device, which can simultaneously display two spectral imaging, so that doctors can compare them in real time.

[0005] On the one hand, a dual-spectrum imaging control system according to an embodiment of the present invention includes:

[0006] Multi-spectral light sources, which can emit a variety of different modes of light;

[0007] An image sensor, used for acquiring a first image signal under the illumination of the multi-spectral light source;

[0008] A signal conversion module, used for copying the first image signal into a plurality of second image signals; the plurality of second image signals correspond to a plurality of different modes of light;

[0009] A main control module, the main control module is used to control the multi-spectral light source to alternately emit a plurality of different modes of light; the main control module includes a plurality of image processing units, a plurality of video output units, an image stitching unit and an output port, the plurality of image processing units and the plurality of video output units correspond to the plurality of second image signals one by one, each of the image processing units is used to perform image processing on the corresponding second image signal, and each of the video output units is used to discard even frames or odd frames of the corresponding second image signal after image processing to obtain a corresponding video signal; the image stitching unit is used to stitch the video signals corresponding to the plurality of second image signals to obtain a stitching signal;

[0010] A display module is used to obtain the spliced ​​signal through the output port and display the spliced ​​image.

[0011] According to some embodiments of the present invention, the multi-spectrum light source is capable of emitting two modes of light.

[0012] According to some embodiments of the present invention, there are two image processing units, one of which adopts ISP parameters of light in one mode, and the other of which adopts ISP parameters of light in another mode.

[0013] According to some embodiments of the present invention, there are two video output units, each of the video output units includes a FIFO, and the FIFOs of the two video output units are connected via a signal inverter.

[0014] According to some embodiments of the present invention, the image stitching unit includes a video processing subsystem, a video data transmission module and a video stitching module which are connected in sequence.

[0015] On the other hand, a dual-spectrum imaging control method according to an embodiment of the present invention is applied to the dual-spectrum imaging control system described in the above-mentioned embodiment, and the method includes:

[0016] The main control module controls the multi-spectrum light source to emit light in multiple different modes alternately;

[0017] The image sensor acquires a first image signal under the illumination of the multi-spectral light source, and sends the first image signal to a signal conversion module;

[0018] The signal conversion module copies the first image signal into a plurality of second image signals; the plurality of second image signals correspond to a plurality of different modes of light;

[0019] Each image processing unit performs image processing on the corresponding second image signal;

[0020] Each video output unit discards even-numbered frames or odd-numbered frames of the corresponding second image signal after image processing to obtain a corresponding video signal;

[0021] The image splicing unit splices the video signals corresponding to the plurality of second image signals to obtain a spliced ​​signal;

[0022] The output port outputs the spliced ​​signal to the display module for display.

[0023] According to some embodiments of the present invention, the step of the main control module controlling the multi-spectrum light source to alternately emit a plurality of different modes of light comprises:

[0024] The main control module controls the multi-spectral light source to switch and emit two modes of light every other frame of image.

[0025] According to some embodiments of the present invention, the step of each image processing unit performing image processing on the corresponding second image signal includes:

[0026] The image processing unit performs at least one of dead point correction, black level compensation, lens shading correction, AWB gain control, Bayer noise reduction, color space conversion, color correction, gamma correction, demosaicing, brightness / chrominance noise filtering, brightness noise reduction, bilateral filtering, non-local mean denoising, edge enhancement, false color suppression, saturation control, and contrast control on the corresponding second image signal.

[0027] On the other hand, according to the storage medium of the embodiment of the present invention, the storage medium stores computer executable instructions, and the computer executable instructions are used to enable a computer to execute the dual-spectrum imaging control method described in the above embodiment.

[0028] On the other hand, an endoscope device according to an embodiment of the present invention includes the dual-spectrum imaging control system described in the above embodiment.

[0029] The dual-spectrum imaging control system and method, storage medium, and device according to the embodiments of the present invention have at least the following beneficial effects: by controlling the multi-spectrum light source to alternately switch to emit a variety of different modes of light, the image sensor can obtain image signals under different spectra; then the image signal of one image sensor is copied into multiple copies through the signal conversion module, and a multi-channel image processing unit is built through the main control module to process the multiple image signals, and after removing the even frames and odd frames of the multiple image signals, the multiple image signals are spliced, so that images of multiple different spectra can be presented on the display module at the same time, and endoscopic imaging of multiple different spectra can be presented in real time, which is convenient for doctors to compare and observe to find lesions, and is helpful for the diagnosis and treatment of lesions.

[0030] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0032] Figure 1 A schematic diagram of the structure of a dual-spectrum imaging control system according to an embodiment of the present invention;

[0033] Figure 2 is a spectrum diagram of white light according to an embodiment of the present invention;

[0034] Figure 3 is a spectrum diagram of narrow-band light according to an embodiment of the present invention;

[0035] Figure 4 is a schematic diagram of image processing according to an embodiment of the present invention;

[0036] Figure 5 is a schematic diagram of a video output unit and an image stitching unit according to an embodiment of the present invention;

[0037] Figure 6 is a switching timing diagram of white light and narrow-band light according to an embodiment of the present invention;

[0038] Figure 7 A flow chart of the steps of the dual-spectrum imaging control method according to an embodiment of the present invention;

[0039] Reference numerals:

[0040] A multi-spectral light source 100 , an image sensor 200 , a signal conversion module 300 , a main control module 400 , and a display module 500 . DETAILED DESCRIPTION

[0041] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limitations on the present application. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0042] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0043] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present invention and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0044] Reference to "embodiments" in the present invention means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0045] Ordinary endoscopes can detect lesions with obvious changes in the shape or color of diseased tissues, but cannot show the shape of the glandular duct openings of diseased tissues and subtle changes in blood vessels at all levels, and are difficult to diagnose flat early cancers. However, narrow-band light imaging endoscopes can show the superficial capillaries of the digestive tract mucosa and the collective large blood vessels in the deep mucosa.

[0046] However, endoscopes with narrow-band light imaging on the market can only display one spectral imaging at the same time. When doctors need to compare other spectral imaging, they need to switch to other spectral imaging. Multiple spectral imaging cannot be compared in real time, which is not conducive to doctor examination.

[0047] To this end, the embodiments of the present invention provide a dual-spectrum imaging control system and method, storage medium, and device, which control a multi-spectrum light source to alternately switch to emit a variety of different modes of light, so that the image sensor can obtain image signals under different spectra; then the image signal of one image sensor is copied into multiple copies through a signal conversion module, and a multi-channel image processing unit is built through a main control module to process the multi-channel image signals, and after removing the even frames and odd frames of the multi-channel image signals, the multiple images are spliced, so that images of multiple different spectra can be presented on the display module at the same time, and endoscopic imaging of multiple different spectra can be presented in real time, which is convenient for doctors to compare and observe to find lesions, and is helpful for the diagnosis and treatment of lesions.

[0048] The following describes in detail the dual-spectrum imaging control system and method, storage medium, and device according to the embodiments of the present invention in conjunction with the accompanying drawings.

[0049] On the one hand, the embodiment of the present invention provides a dual-spectrum imaging control system, such as Figure 1As shown, the system includes a multi-spectral light source 100, an image sensor 200, a signal conversion module 300, a main control module 400 and a display module 500; wherein the multi-spectral light source 100 can emit light of multiple different modes; the image sensor 200 is used to obtain a first image signal under the illumination of the multi-spectral light source 100; the signal conversion module 300 is used to copy the first image signal into multiple second image signals, and the multiple second image signals correspond to the multiple different modes of light; the main control module 400 is used to control the multi-spectral light source 100 to alternately emit light of multiple different modes; the main control module 400 includes Multiple image processing units, multiple video output units, image stitching units and output ports, the multiple image processing units, the multiple video output units correspond one-to-one to the multiple second image signals, each image processing unit is used to perform image processing on the corresponding second image signal, each video output unit is used to discard the even frames or discard the odd frames of the corresponding second image signal after image processing to obtain the corresponding video signal; the image stitching unit is used to stitch the video signals corresponding to the multiple second image signals to obtain a stitching signal; the output port is used to output the stitching signal to the display module 500 for display.

[0050] Specifically, the multi-spectral light source 100 can emit a variety of different lights, such as white light, narrow-band light, infrared light, ultraviolet light, etc., and can emit white light and narrow-band light alternately, or emit white light and ultraviolet light alternately, or emit white light and infrared light alternately, etc. By making the multi-spectral light source 100 emit a variety of different lights, the image sensor 200 can obtain image signals under different spectra. Then, the image sensor 200 sends the acquired first image signal (MIPI signal) to the signal conversion module 300, and the signal conversion module 300 copies the first image signal to form multiple second image signals. The number of second image signals corresponds to the number of different modes of light. There are several modes of light, and several second image signals are copied. Then, the signal conversion module 300 sends these second image signals to the main control module 400 for image processing and splicing.

[0051] In this example, the multi-spectral light source 100 is selected to emit white light and narrow-band light alternately, so that the image sensor 200 obtains image signals in the white light mode and the narrow-band light mode. Then, the image sensor 200 sends the obtained first image signal to the signal conversion module 300. In this example, the signal conversion module 300 uses the bridge chip LT9211, which divides the MIPI signal into two second image signals (i.e., MIPI1 and MIPI2) through the bridge chip LT9211 and sends these two signals to the main control module 400. Figure 1As shown, in some embodiments of the present invention, the signal conversion module 300 is connected to the main control module 400 via I2C, and the main control module 400 configures the signal channel, signal format, signal transmission rate and image resolution of the signal conversion module 300 via I2C, such as configuring the MIPI input channel, signal format RAW10, MIPI rate, image resolution, and two MIPI output channels of the signal conversion module 300. Through the above configuration, the signal conversion module 300 can generate two identical MIPI signals to the main control module 400 for processing.

[0052] In this example, the main control module 400 uses FPGA, and the main control module 400 is equipped with multiple image processing units, and the number of image processing units corresponds to the number of second image signals. When the multi-spectral light source 100 emits white light and narrow-band light alternately, the main control module 400 includes two image processing units (i.e., ISP1 and ISP2), forming two ISP channels (image processing channels), ISP1 is used to receive MIPI1, and ISP2 is used to receive MIPI2. Among them, ISP1 adopts the ISP parameters of the white light mode, and ISP2 adopts the ISP parameters of the narrow-band light mode. The spectrum of white light is as follows: Figure 2 As shown, the spectrum of narrowband light (special light) is Figure 3 shown; from Figure 3 It can be seen that the narrow-band light of the embodiment of the present invention includes UV light (peak wavelength 415 nm, half-width 20 nm) + green light (peak wavelength 550 nm, half-width 80 nm).

[0053] After the two image processing units receive the two second image signals, they need to perform image processing on the second image signals, such as Figure 4 As shown, image processing includes: dead point correction, black level compensation, lens shading correction, AWB gain control, Bayer noise reduction, color space conversion, color correction, gamma correction, demosaicing, brightness / chrominance noise filtering, brightness noise reduction, bilateral filtering, non-local mean denoising, edge enhancement, false color suppression, saturation control and contrast control. It should be noted that due to the great difference between the narrow-band light spectrum and the white light spectrum, the image processing of ISP1 is to obtain the imaging of the white light spectrum, and the image processing of ISP2 is to obtain the imaging of the narrow-band light spectrum. The image processing processes of the two are the same, but the parameters of the processes are different in the image adjustment process. The white light spectrum is an RGB image, and the image is in color, while the narrow-band light spectrum is not an RGB image. In order to facilitate the human eye to see the image clearly, it needs to be color corrected and some gain compensation is performed during the image processing process, so the parameters in the image processing process need to be adjusted and debugged, that is, the parameters in color correction, gamma correction, brightness filtering, and brightness noise reduction will be different.

[0054] After performing image processing on the second image signal, each channel of the second image signal is sent to the corresponding video output unit. The video output unit is used to discard the odd or even frames of the second image signal to facilitate the subsequent image stitching of multiple second image signals. In the case where there are two channels of second image signals, there are also two corresponding video output units. MIPI1 after image processing by ISP1 is sent to one of the video output units, and the video output unit discards the even frames of MIPI1; MIPI2 after image processing by ISP2 is sent to the other video output unit, and the video output unit discards the odd frames of MIPI2. Specifically, Figure 5 As shown, when there are two video output units, each video output unit includes a FIFO, and the FIFOs of the two video output units are connected through a signal inverter. Among them, the enable pin (EN) of FIFO1 and the enable pin (EN) of FIFO2 are connected through a signal inverter. By controlling the enable of FIFO, the even frame or odd frame is periodically discarded. The enable pin of FIFO1 and the enable pin of FIFO2 are reversed, so when the data image of ISP1 can pass through FIFO1, the data of FIFO2 is cut off, that is, FIFO1 only allows odd frames to pass, and FIFO2 only allows even frames to pass.

[0055] The multiple video output units send the video signals obtained after discarding the odd frames or even frames to the image splicing unit, which splices the multiple video signals to achieve the simultaneous output of multiple images with different spectra. Figure 1 As shown, when the display module 500 includes a display, and the multi-spectral light source 100 emits white light and narrow-band light alternately, the white light image and the narrow-band light image can be simultaneously presented on the display, so that the doctor can compare the two spectrum images in real time, which is beneficial for the doctor to perform examination. The display module 500 can also include multiple displays, and one spectrum image is displayed on each display.

[0056] Specifically, Figure 5 As shown, in this example, the image stitching unit includes a video processing subsystem (Video Processing Subsystem), a video data transmission module (VDMA) and a video stitching module (VdieoMixer) connected in sequence. Among them, the video processing subsystem can reduce or enlarge the image according to business needs. Through the video data transmission module and the video stitching module, the two images can be stitched together and sent to the back-end display channel (HDMI_TX), and then sent to the display module 500 through the output port (DVI), so that the display module 500 can realize the simultaneous real-time display of white light images and narrow-band light images.

[0057] It should be noted that if Figure 5 As shown, in this example, when the main control module 400 controls the multi-spectrum light source 100 to switch between white light and narrow-band light, the multi-spectrum light source 100 switches the spectrum output by the light source once every frame of image (16.6ms), thereby realizing turbulent switching of the spectrum and providing light source support for obtaining two spectral images.

[0058] According to the dual-spectrum imaging control system of the embodiment of the present invention, the multi-spectrum light source 100 is controlled to alternately switch to emit a variety of different modes of light, so that the image sensor 200 can obtain image signals under different spectra; then the image signal of one image sensor is copied into multiple copies through the signal conversion module 300, and a multi-channel image processing unit is built through the main control module 400 to process the multi-channel image signals, and after removing the even frames and odd frames of the multi-channel image signals respectively, the multiple images are spliced, so that multiple different spectral images can be presented on the display module 500 at the same time, and multiple endoscopic images of different spectra can be presented in real time, which is convenient for doctors to compare and observe to find lesions, and is helpful for the diagnosis and treatment of lesions.

[0059] On the other hand, based on the above dual-spectrum imaging control system, the embodiment of the present invention also proposes a dual-spectrum imaging control method, such as Figure 7 As shown, the method comprises the following steps:

[0060] Step S100: the main control module 400 controls the multi-spectrum light source 100 to emit light in a plurality of different modes alternately;

[0061] It should be noted that the multi-spectrum light source 100 can emit a variety of different lights, such as white light, narrow-band light, infrared light, ultraviolet light, etc., and can emit white light and narrow-band light alternately, or emit white light and ultraviolet light alternately, or emit white light and infrared light alternately, etc. In this example, the multi-spectrum light source 100 is selected to emit white light and narrow-band light alternately, such as Figure 6 As shown, when the main control module 400 controls the multi-spectrum light source 100 to switch between white light and narrow-band light, the multi-spectrum light source 100 switches the spectrum output by the light source once every frame of image (16.6ms), thereby realizing turbulent switching of the spectrum and providing light source support for obtaining two spectral images.

[0062] Step S200: the image sensor 200 acquires a first image signal illuminated by the multi-spectral light source 100, and sends the first image signal to the signal conversion module 300;

[0063] Taking the multi-spectral light source 100 emitting white light and narrow-band light alternately as an example, the image sensor 200 acquires image signals in white light mode and narrow-band light mode, and then the image sensor 200 sends the acquired image signals (MIPI signals) to the signal conversion module 300.

[0064] Step S300: the signal conversion module 300 copies the first image signal into a plurality of second image signals;

[0065] It should be noted that the number of second image signals corresponds to the number of light modes. The multi-spectral light source 100 emits several different modes of light, which corresponds to several second image signals. Taking the multi-spectral light source 100 emitting white light and narrow-band light alternately as an example, the signal conversion module 300 copies the first image signal into two second image signals. In this example, the signal conversion module 300 uses a bridge chip LT9211, which divides the MIPI signal into two second image signals (i.e., MIPI1 and MIPI2) through the bridge chip LT9211, and sends these two signals to the main control module 400.

[0066] Step S400: each image processing unit performs image processing on the corresponding second image signal;

[0067] The number of image processing units corresponds to the number of second image signals. When the multi-spectral light source 100 emits white light and narrow-band light alternately, the main control module 400 includes two image processing units (i.e., ISP1 and ISP2) to form two ISP channels (image processing channels). ISP1 is used to receive MIPI1, and ISP2 is used to receive MIPI2. ISP1 uses ISP parameters of white light mode, and ISP2 uses ISP parameters of narrow-band light mode. The spectrum of white light is as follows: Figure 2 As shown, the spectrum of narrowband light (special light) is Figure 3 shown; from Figure 3 It can be seen that the narrow-band light of the embodiment of the present invention includes UV light (peak wavelength 415nm, half-height width 20nm) + green light (peak wavelength 550nm, half-height width 80nm). After the two image processing units receive the two second image signals, they need to perform image processing on the second image signals, such as Figure 4As shown, image processing includes: dead point correction, black level compensation, lens shading correction, AWB gain control, Bayer noise reduction, color space conversion, color correction, gamma correction, demosaicing, brightness / chrominance noise filtering, brightness noise reduction, bilateral filtering, non-local mean denoising, edge enhancement, false color suppression, saturation control and contrast control. It should be noted that due to the great difference between the narrow-band light spectrum and the white light spectrum, the image processing of ISP1 is to obtain the imaging of the white light spectrum, and the image processing of ISP2 is to obtain the imaging of the narrow-band light spectrum. The image processing processes of the two are the same, but the parameters of the processes are different in the image adjustment process. The white light spectrum is an RGB image, and the image is in color, while the narrow-band light spectrum is not an RGB image. In order to facilitate the human eye to see the image clearly, it needs to be color corrected and some gain compensation is performed during the image processing process, so the parameters in the image processing process need to be adjusted and debugged, that is, the parameters in color correction, gamma correction, brightness filtering, and brightness noise reduction will be different.

[0068] Step S500: each video output unit discards even frames or odd frames of the corresponding second image signal after image processing to obtain a corresponding video signal;

[0069] After performing image processing on the second image signal, each channel of the second image signal is sent to the corresponding video output unit. The video output unit is used to discard the odd or even frames of the second image signal to facilitate the subsequent image stitching of multiple second image signals. In the case where there are two channels of second image signals, there are also two corresponding video output units. MIPI1 after image processing by ISP1 is sent to one of the video output units, and the video output unit discards the even frames of MIPI1; MIPI2 after image processing by ISP2 is sent to the other video output unit, and the video output unit discards the odd frames of MIPI2. Specifically, Figure 5 As shown, when there are two video output units, each video output unit includes a FIFO, and the FIFOs of the two video output units are connected through a signal inverter. Among them, the enable pin (EN) of FIFO1 and the enable pin (EN) of FIFO2 are connected through a signal inverter. By controlling the enable of FIFO, the even frame or odd frame is periodically discarded. The enable pin of FIFO1 and the enable pin of FIFO2 are reversed, so when the data image of ISP1 can pass through FIFO1, the data of FIFO2 is cut off, that is, FIFO1 only allows odd frames to pass, and FIFO2 only allows even frames to pass.

[0070] Step S600: the image splicing unit splices the video signals corresponding to the plurality of second image signals to obtain a spliced ​​signal;

[0071] Specifically, Figure 5 As shown, in this example, the image stitching unit includes a video processing subsystem (Video Processing Subsystem), a video data transmission module (VDMA) and a video stitching module (VdieoMixer) connected in sequence. Among them, the video processing subsystem can reduce or enlarge the image according to business needs. Through the video data transmission module and the video stitching module, the two images can be stitched together and sent to the back-end display channel (HDMI_TX), and then sent to the display module 500 through the output port (DVI), so that the display module 500 can realize the simultaneous real-time display of white light images and narrow-band light images.

[0072] Step S700: the output port outputs the spliced ​​signal to the display module 500 for display.

[0073] According to the dual-spectrum imaging control method of the embodiment of the present invention, the multi-spectrum light source 100 is controlled to alternately switch to emit a variety of different modes of light, so that the image sensor 200 can obtain image signals under different spectra; then the image signal of one image sensor is copied into multiple copies through the signal conversion module 300, and a multi-channel image processing unit is built through the main control module 400 to process the multi-channel image signals, and after removing the even frames and odd frames of the multi-channel image signals respectively, the multiple images are spliced, so as to realize the simultaneous presentation of multiple different spectral images on the display module 500, and the real-time presentation of multiple different spectral endoscopic images, which is convenient for doctors to compare and observe to find lesions, and is helpful for the diagnosis and treatment of lesions.

[0074] On the other hand, according to an endoscopic device of an embodiment of the present invention, the endoscopic device includes the dual-spectrum imaging control system described in the above embodiment. The endoscopic device includes a light source module, a lens module, a signal processing module, a main control module and a display module. The light source module can alternately switch to emit a plurality of different modes of light. The image sensor on the lens module can obtain image signals under different spectra (light of different modes). After that, the signal processing module (signal conversion module) copies the image signal of one image sensor into multiple copies, and a multi-channel image processing unit is built through the main control module to process the multi-channel image signals. After removing the even frames and odd frames of the multi-channel image signals, the multiple images are spliced, so as to realize that multiple different spectral images are simultaneously presented on the display module, and multiple different spectral endoscopic images are presented in real time, which is convenient for doctors to compare and observe to find lesions, and is helpful for the diagnosis and treatment of lesions.

[0075] On the other hand, an embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the above-mentioned dual-spectrum imaging control method is implemented.

[0076] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and are implemented to be located in one place, or may also be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0077] Although specific embodiments are described herein, those of ordinary skill in the art will recognize that many other modifications or alternative embodiments are also within the scope of the present disclosure. For example, any of the functions and / or processing capabilities described in conjunction with a particular device or component may be performed by any other device or component. In addition, although various exemplary implementations and architectures have been described according to embodiments of the present disclosure, those of ordinary skill in the art will recognize that many other modifications to the exemplary implementations and architectures described herein are also within the scope of the present disclosure.

[0078] Some aspects of the present disclosure are described above with reference to the block diagram and flow chart of the system, method, system and / or computer program product according to the exemplary embodiments.It should be understood that the combination of one or more blocks in the block diagram and the flow chart and the blocks in the block diagram and the flow chart can be realized by executing computer executable program instructions respectively.Equally, according to some embodiments, some blocks in the block diagram and the flow chart may not need to be executed in the order shown, or may not need to be executed in full.In addition, additional components and / or operations beyond those components and / or operations shown in the blocks in the block diagram and the flow chart may be present in certain embodiments.

[0079] Therefore, the blocks in the block diagrams and flow charts support the combination of devices for performing the specified functions, the combination of elements or steps for performing the specified functions, and the program instruction devices for performing the specified functions. It should also be understood that each block in the block diagrams and flow charts and the combination of blocks in the block diagrams and flow charts can be implemented by a special-purpose hardware computer system or a combination of special-purpose hardware and computer instructions that performs a specific function, element or step.

[0080] The program modules, applications to be installed, etc. described herein may include one or more software components, including, for example, software objects, methods, data structures, etc. Each such software component may include computer executable instructions that, in response to being executed, cause at least a portion of the functions described herein (e.g., one or more operations of the illustrative methods described herein) to be performed.

[0081] Software components can be encoded with any of various programming languages. An exemplary programming language can be a low-level programming language, such as an assembly language associated with a specific hardware architecture and / or an operating system platform. The software component comprising assembly language instructions may need to be converted to executable machine code by an assembler before being executed by hardware architecture and / or platform. Another exemplary programming language can be a higher-level programming language, which can be transplanted across multiple architectures. The software component comprising a higher-level programming language may need to be converted to an intermediate representation by an interpreter or a compiler before execution. Other examples of programming languages ​​include but are not limited to macro languages, shells or command languages, job control languages, scripting languages, database queries or search languages ​​or report writing languages. In one or more exemplary embodiments, the software component comprising the instruction of one of the above-mentioned programming language examples can be directly executed by an operating system or other software components, without first being converted into another form.

[0082] Software components may be stored as files or other data storage structures. Software components of similar type or related functions may be stored together, such as in a specific directory, folder, or library. Software components may be static (e.g., preset or fixed) or dynamic (e.g., created or modified at execution time).

[0083] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in the relevant technical field without departing from the purpose of the present invention.

Claims

1. A dual-spectrum imaging control system, characterized in that: include: Multi-spectral light sources, which can emit a variety of different modes of light; An image sensor, used for acquiring a first image signal under the illumination of the multi-spectral light source; A signal conversion module, used for copying the first image signal into a plurality of second image signals; the plurality of second image signals correspond to a plurality of different modes of light; A main control module, the main control module is used to control the multi-spectral light source to alternately emit a plurality of different modes of light; the main control module includes a plurality of image processing units, a plurality of video output units, an image stitching unit and an output port, the plurality of image processing units and the plurality of video output units correspond to the plurality of second image signals one by one, each of the image processing units is used to perform image processing on the corresponding second image signal, and each of the video output units is used to discard even frames or odd frames of the corresponding second image signal after image processing to obtain a corresponding video signal; the image stitching unit is used to stitch the video signals corresponding to the plurality of second image signals to obtain a stitching signal; A display module is used to obtain the spliced ​​signal through the output port and display the spliced ​​image.

2. The dual-spectrum imaging control system according to claim 1, characterized in that: The multi-spectrum light source is capable of emitting light in two modes.

3. The dual-spectrum imaging control system according to claim 2, characterized in that: There are two image processing units, one of which adopts ISP parameters of light in one mode, and the other of which adopts ISP parameters of light in another mode.

4. The dual-spectrum imaging control system according to claim 2, characterized in that: There are two video output units, each of which includes a FIFO, and the FIFOs of the two video output units are connected via a signal inverter.

5. The dual-spectrum imaging control system according to claim 1, characterized in that: The image splicing unit comprises a video processing subsystem, a video data transmission module and a video splicing module which are connected in sequence.

6. A dual-spectrum imaging control method, characterized in that: Applied to the dual-spectrum imaging control system according to any one of claims 1 to 5, the method comprises: The main control module controls the multi-spectrum light source to emit light in multiple different modes alternately; The image sensor acquires a first image signal under the illumination of the multi-spectral light source, and sends the first image signal to a signal conversion module; The signal conversion module copies the first image signal into a plurality of second image signals; the plurality of second image signals correspond to a plurality of different modes of light; Each image processing unit performs image processing on the corresponding second image signal; Each video output unit discards even-numbered frames or odd-numbered frames of the corresponding second image signal after image processing to obtain a corresponding video signal; The image splicing unit splices the video signals corresponding to the plurality of second image signals to obtain a spliced ​​signal; The output port outputs the spliced ​​signal to the display module for display.

7. The dual-spectrum imaging control method according to claim 6, characterized in that: The step of the main control module controlling the multi-spectrum light source to alternately emit a plurality of different modes of light comprises: The main control module controls the multi-spectral light source to switch and emit two modes of light every other frame of image.

8. The dual-spectrum imaging control method according to claim 6, characterized in that: The step of each image processing unit performing image processing on the corresponding second image signal comprises: The image processing unit performs at least one of dead point correction, black level compensation, lens shading correction, AWB gain control, Bayer noise reduction, color space conversion, color correction, gamma correction, demosaicing, brightness / chrominance noise filtering, brightness noise reduction, bilateral filtering, non-local mean denoising, edge enhancement, false color suppression, saturation control, and contrast control on the corresponding second image signal.

9. A storage medium, characterized in that: The storage medium stores computer executable instructions, and the computer executable instructions are used to enable a computer to execute the dual-spectrum imaging control method according to any one of claims 6 to 8.

10. An endoscope device, characterized in that: The endoscope device comprises a dual-spectrum imaging control system as described in any one of claims 1 to 5.

Citation Information

Cited By

  • Image processing method and device, electronic equipment and storage medium

    CN120013836A