Image acquisition and processing system, method and storage medium

Abstract

An image acquisition and processing system, method and storage medium, the system includes: an image acquisition module, a bridge processing module and an image processing module; the image acquisition module includes an image sensor; the image sensor has an SPI interface, which is used to acquire a first image of a region to be acquired, and send a serial image signal to the bridge processing module through the SPI interface; the bridge processing module has an SPI and a DVP interface, which is used to acquire and process the serial image signal to obtain a parallel image signal, and send the parallel image signal to the image processing module through the DVP interface; the image processing module has a DVP interface, which is used to control the image sensor to work, acquire parallel image signals, process the first image, and adjust at least one of the resolution, color, brightness, sharpness and contrast of the first image to improve the image quality of the first image. The system cost of the application is low, and the image quality and the size of the image acquisition module meet the application requirements of ultra-thin endoscopes.

Description

Technical Field

[0001] The present invention relates to the technical field of endoscope image acquisition, and in particular to an image acquisition and processing system, method and storage medium. Background Art

[0002] Medical endoscope is a commonly used medical device. It mainly enters the human organs through the natural channels of the human body or the small incision formed by surgery, such as the trachea, bronchi, stomach, duodenum of the lungs, and the renal pelvis of the urinary system, so as to assist doctors in making more accurate diagnosis and treatment of patients. Due to the large range of sizes of the cavities of human organs, some small cavities can only be entered by ultra-fine endoscopes, such as bronchi, kidneys, gallbladders, etc. The channels leading to these organs are very small, requiring the corresponding endoscope diameter to meet the ultra-fine standard.

[0003] Electronic endoscopes are new endoscopes with image sensors and image processors as their core. The diameter of electronic endoscopes is largely limited by the size of image sensors. Currently, ultra-small image sensors based on CMOS are mainly controlled by OmniVision Technologies, which basically forms a monopoly and are expensive. Their image sensors also need to be used in conjunction with private AD conversion chips. The cost of medical endoscopes designed based on these image sensors has always been high, which has brought huge economic pressure to ordinary patients. Summary of the invention

[0004] The main technical problem solved by the present invention is that the existing CMOS-based electronic endoscope is expensive.

[0005] According to the first aspect, an embodiment provides an image acquisition and processing system, including: an image acquisition module, a bridge processing module and an image processing module;

[0006] The image acquisition module includes an image sensor; the image sensor has an SPI interface, which is used to acquire a first image of the area to be acquired, and send a serial image signal to the bridge processing module through the SPI interface; the serial image signal is a digital signal, and the serial image signal includes first data corresponding to the first image;

[0007] The bridge processing module has an SPI interface and a DVP interface, which are used to obtain and process the serial image signal to obtain a parallel image signal, and send the parallel image signal to the image processing module through the DVP interface; the parallel image signal is a digital signal, and the parallel image signal includes the first data corresponding to the first image;

[0008] The image processing module has a DVP interface for controlling the operation of the image sensor, acquiring parallel image signals, processing the first image, and adjusting at least one of the resolution, color, brightness, sharpness, and contrast of the first image to improve the image quality of the first image.

[0009] According to the second aspect, an embodiment provides an image acquisition and processing method, including:

[0010] Capturing a first image of the area to be captured, obtaining a serial image signal corresponding to the first image, the serial image signal being output through an SPI interface of the image sensor, the serial image signal being a digital signal, and including first data corresponding to the first image;

[0011] Acquire and process the serial image signal to obtain a parallel image signal, and output the parallel image signal through the DVP interface; the parallel image signal is a digital signal, and the parallel image signal includes first data corresponding to the first image;

[0012] Acquire parallel image signals, process the first image, and adjust at least one of the resolution, color, brightness, sharpness, and contrast of the first image to improve the image quality of the first image.

[0013] According to the third aspect, an embodiment provides a computer-readable storage medium, on which a program is stored, and the program can be executed by a processor to implement the measurement method described in the second aspect.

[0014] According to the image acquisition and processing system, method and storage medium of the above-mentioned embodiments, by adopting an image sensor with an SPI interface, the small size requirement in endoscope applications can be met. The serial signal can be converted into a parallel signal through a bridge processing module, so that the image processing module can process the image information collected by the image sensor in real time, and perform image processing to improve the image quality. Ultimately, high-quality image acquisition is achieved with a low-cost image processor and a bridge processing module without technical limitations, thereby meeting the application requirements of the endoscope. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of the structure of an image acquisition system used in an existing ultra-thin endoscope;

[0016] Figure 2 A schematic diagram of the structure of an image acquisition and processing system provided in one embodiment of the present application (I);

[0017] Figure 3 A schematic diagram of the structure of an image acquisition module provided in one embodiment of the present application;

[0018] Figure 4A schematic diagram of the structure of an image acquisition and processing system provided in one embodiment of the present application (II);

[0019] Figure 5 A flowchart of an image acquisition and processing method provided in one embodiment of the present application (I);

[0020] Figure 6 Flowchart (II) of an image acquisition and processing method provided for one embodiment of the present application;

[0021] Figure 7 A flowchart for processing a first image is provided in accordance with an embodiment of the present application.

[0022] Figure numerals: 10 - image acquisition module, 100 - image sensor; 11 - housing; 111 - first mounting slot; 112 - first window; 20 - bridge processing module; 30 - image processing module. DETAILED DESCRIPTION

[0023] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0024] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0025] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0026] At present, there are two main implementation schemes in the field of ultra-fine endoscopes: one is a fiber-optic medical endoscope designed based on the total reflection principle of optical fiber. Each optical fiber can be regarded as a pixel point. When a bunch of orderly arranged optical fibers are imaged, a complete image of the internal tissue can be formed. Therefore, the main factor affecting the diameter of this fiber-optic endoscope is the single-filament diameter of each optical fiber. At present, the gap in manufacturing technology at home and abroad is not obvious. Japan's Olympus, Fuji, Pentax and other international endoscope giants all produce this type of fiber-optic ultra-fine endoscope and can meet most clinical needs. However, this fiber-optic endoscope is difficult to assemble in actual use, and the optical fiber is easy to wear and break, resulting in a high price for fiber-optic endoscopes. At the same time, the optical performance and indicators designed in a simulation environment are difficult to actually achieve, resulting in a large difference between the design and the finished product of the fiber-optic endoscope. Based on these shortcomings, the development of fiber-optic endoscopes is limited.

[0027] The other is the electronic endoscope, which is a new type of endoscope with image sensors and image processors as its core. The diameter of the electronic endoscope is largely limited by the size of the image sensor. Image sensors are divided into two camps, based on CCD and CMOS technology.

[0028] In the field of CCD-based image sensors, Japan's Olympus and Fuji have mastered advanced technology. They generally do not sell ultra-small CCD image sensors to the outside world. Even if they allow customization, the price is expensive, and the technology is completely confidential. Therefore, except for Japan, the only technical route in the field of ultra-small image sensors in the global market is CMOS. At present, ultra-small image sensors based on CMOS are mainly controlled by OmniVision Technologies (OV), which has basically formed a technology monopoly and is expensive. Its image sensors also need to be used in conjunction with private AD conversion chips. Medical endoscopes designed based on this image sensor are costly.

[0029] like Figure 1 As shown in (A) and (B), for the ultra-thin medical endoscope market, the following technical solutions are currently commonly used on the market. Among them, the image sensor uses OmniVision Technologies' OV6946 / OV6948 / OV6930 series. The size of these image sensors is less than or equal to 1 / 18 inch, which meets the small diameter requirements (less than or equal to 1 / 15 inch) in the field of ultra-thin endoscopes. Because these image sensors are limited to outputting private analog video signals, the company's private AD conversion chip (such as the OV426 series) is required to perform analog-to-digital conversion to obtain an open digital DVP signal. For the digital DVP signal, it can be as follows Figure 1 As shown in (A), an FPGA chip is used to execute the ISP algorithm for image processing, or as Figure 1As shown in (B), the ISP chip built into the ASIC chip is used for image processing. The specific processing method requires the user to program the image processing method himself.

[0030] Use Figure 1 The technical solutions shown in (A) and (B) have the following disadvantages:

[0031] a. Image quality: Since the video signal is an analog video signal when it is output from the image sensor, and the analog signal is easily interfered by external signals to varying degrees both inside the sensor and during the transmission process, it will produce ripples and noise, thus affecting the final image quality.

[0032] b. Cost: The video signals of this type of image sensor are private protocols and cannot be decoded using general decoding chips on the market. They need to be bound to the OV manufacturer's own decoding chips for collection and decoding, which further increases the risk of high prices due to monopoly. In the field of ultra-small sensors for the medical endoscope market, OmniVision's image sensors are almost in a monopoly position, resulting in high prices. At the same time, they need to work with the image sensor's matching decoding chips, resulting in extremely high prices for decoding chips. Therefore, the cost of products designed with this solution will be abnormally high.

[0033] like Figure 1 As shown in (C), considering the cost issue, an image sensor that outputs CVBS universal video protocol signals (such as OV6922 / OV6920) can be used. The advantage of this solution is that the image sensor output signal is the standard protocol CVBS, and any chip on the market that supports this video interface can collect and decode it. However, the main disadvantages of this solution are as follows:

[0034] 1. A new size problem is introduced. The size of this type of image sensor is too large, and the actual package size of the chip is larger than 1 / 15 inch, which cannot fully meet the diameter requirements of ultra-thin medical endoscopes.

[0035] 2. The video signal is an analog signal and still has the signal interference problem mentioned in Solution 1, which has a significant impact on image quality.

[0036] 3. The price of the image sensor chip is still more than a few US dollars. If it is used in a disposable endoscope scenario, the cost is still relatively high.

[0037] In the embodiment of the present application, by adopting an open non-proprietary image acquisition module 10 with an SPI interface and cooperating with a bridge processing module 20, the technical barriers of existing CMOS sensors in the field of endoscopes are overcome, a low-latency image acquisition module is realized, and the clinical image quality requirements can be met while solving the problem of high costs. It is particularly suitable for use in disposable endoscope scenarios. This is because of the special nature of some detection projects, and the endoscope lens is not convenient for recycling and needs to be used once.

[0038] like Figure 2 As shown, an embodiment of the present application provides an image acquisition and processing system (referred to as the system for short), which may include: an image acquisition module 10, a bridge processing module 20 and an image processing module 30.

[0039] The image acquisition module 10 may include an image sensor 100; the image sensor 100 has an SPI interface, the image sensor 100 is used to acquire a first image of the area to be acquired, and send a serial image signal to the bridge processing module 20 through the SPI interface; the serial image signal is a digital signal, and the serial image signal may include first data corresponding to the first image.

[0040] The bridge processing module 20 has an SPI interface and a DVP interface, and is used to obtain and process the serial image signal through the SPI interface to obtain a parallel image signal, and send the parallel image signal to the image processing module 30 through the DVP interface; the parallel image signal is a digital signal, and the parallel image signal may include the first data corresponding to the first image. In this embodiment, the first data corresponding to the first image can be understood as the data of a picture format file, which may not carry other clock or communication information, as the minimum data for restoring the first image, and of course may also carry other information, such as the signal of the image sensor 100, the shooting time and other information.

[0041] like Figure 4 As shown, in one embodiment, the serial image signal may further include a pixel clock signal, synchronization code data, a data packet ID and first data.

[0042] The bridge processing module 20 is also used to send a main clock signal to the image sensor 100, and the pixel clock signal is obtained by the image sensor 100 according to the frequency division of the main clock signal (MCLK); the bridge processing module 20 is also used to obtain the pixel clock signal, synchronization code data and data packet ID, obtain the line synchronization signal and the frequency synchronization signal according to the synchronization code data and the data packet ID, and send the pixel clock signal (PCLK), the line synchronization signal (HSYNC), the frequency synchronization signal (VSYNC) and the first data (data output by D0 to D7) to the image processing module 30.

[0043] The image processing module 30 has a DVP interface for controlling the operation of the image sensor 100, acquiring parallel image signals, processing the first image, and adjusting at least one of the resolution, color, brightness, sharpness, and contrast of the first image to improve the image quality of the first image.

[0044] In one embodiment, the maximum outer diameter of the image acquisition module 10 is less than or equal to 1 / 15 inch, the acquisition frame rate is greater than or equal to 30 frames, and the resolution is greater than or equal to 320*240 pixels; and / or, the bridge processing module 20 may include a programmable bridge chip; and / or, the image processing module 30 may include an image processing chip. For example, Figure 3 As shown, the image acquisition module 10 may include an image sensor 100 and a housing 11. The housing 11 is used to adapt to the lens of the endoscope to be used so that the image acquisition module 10 can be installed on the lens of the endoscope. The housing 11 may also have a first mounting groove 111 and a first window 112. The image sensor 100 is mounted on the housing 11 through the first mounting groove 111. The first window 112 limits the photosensitive angle range of the image sensor 100. The first window 112 can also be used to install optical elements such as lenses.

[0045] In one embodiment, the image sensor 100 can adopt any open non-private image sensor with SPI interface on the market, for example, CMOS sensors such as BYD's BF30A2 are adopted, which can meet the requirements of resolution and size. The resolution of this type of sensor is about 80,000 pixels (original resolution is 320*240), which is less than 160,000 pixels of OV's mainstream product ov6946. Although the resolution is half of ov6946, it is actually indirectly advantageous in size, which can just meet the application in the small-size medical endoscope market. At the same time, the output signal is the RAW digital signal of spi, and the original signal is better, which is convenient to optimize the image quality by using image processing algorithm, so that it is close to the image quality of ov6946. It needs to be explained here that although the pixel resolution of ov6946 sensor is higher, it first outputs analog signal, and then undergoes digital-to-analog conversion. The analog signal is very susceptible to different degrees of interference from external signals whether it is inside the sensor or during transmission, thereby generating ripples and noise, and the original image quality is reduced. Therefore, the image sensor 100 with relatively low resolution used in the present application outputs a digital signal, is less affected by external interference, and has high original image quality. Therefore, after image processing, the image quality is further improved and can be close to the image quality of ov6946.

[0046] In one embodiment, the bridge processing module 20 can be implemented by any open non-proprietary bridge chip on the market. For example, the H1C02 series of Jingwei Qili can be used. This chip is a low-power, low-cost programmable device that integrates a highly flexible FPGA and an enhanced 8051 MCU. It has a small package and low cost. It is based on the SRAM FPGA architecture - 1536 4-input lookup tables, 1024 DFF registers, and a performance of up to 200MHz. The performance is fully capable of real-time acquisition of 320*240 resolution video. Using this type of bridge chip and the above-mentioned image sensor 100 only costs about 4 yuan, which is much cheaper than Figure 1 The technical solutions (A) and (B) reduce the cost to less than 1% of the original (private analog-to-digital conversion chip and private analog signal image sensor). The image sensor 100 with a lower cost can meet the needs of disposable endoscope scenarios.

[0047] In one embodiment, the image processing module 30 can be implemented using an available FPGA chip or ASIC chip, such as the SSC336 of StarCent Technology. The ASIC chip is an image processing chip that can control the image sensor 100 through the I2C interface when processing image data, mainly including the initialization, reset, and real-time configuration of the exposure and gain-related registers of the image sensor 100.

[0048] In summary, on the first hand, the image sensor 100 with an SPI interface is used to meet the size requirements and the acquisition frame rate requirements, and the cost of the image acquisition module 10 is also greatly reduced. On the second hand, image processing is performed through the image processing module 30 to improve the final image quality and meet clinical requirements. On the third hand, the bridge processing module 20 is used to process the data of the image sensor 100 in real time without the need for caching and then processing. On the fourth hand, both the image sensor 100 and the bridge processing module 20 can be implemented with open and non-proprietary hardware, which greatly reduces the high cost problem caused by technology monopoly.

[0049] Further explanation, for the third aspect, if the image processing module 30 is used to directly receive the serial image signal output by the image sensor 100, the image processing module 30 needs to cache the serial image signal before processing, which has poor real-time performance. The present application cleverly uses the bridge processing module 20 to process the serial image signal in real time, without temporary storage, and sends it to the image processing module 30 in a parallel transmission mode, so that the image processing module 30 can process the first image in real time.

[0050] That is to say, the image acquisition and processing system provided by the present application can achieve high-quality, high-real-time image acquisition at a low cost and without technical restrictions, so as to meet the use requirements of ultra-fine diameter medical endoscopes.

[0051] The following is an explanation of the specific process of the image acquisition and processing method of the image acquisition and processing system. Figure 5 and Figure 6 As shown, the image acquisition and processing method (referred to as the method) may include the following steps:

[0052] Step 1: Capture a first image of the area to be captured, and obtain a serial image signal corresponding to the first image. The serial image signal is output through the SPI interface of the image sensor 100. The serial image signal is a digital signal, and the serial image signal may include first data corresponding to the first image.

[0053] For example, Figure 4 and Figure 6 As shown, the bridge processing module 20 sends a main clock signal to the image sensor 100, and the image processing module 30 initializes the image sensor 100 through the I2C port, controls the image sensor 100 to work, and acquires the first image of the area to be captured.

[0054] Step 2: Figure 6 As shown, a serial image signal is acquired and processed to obtain a parallel image signal, which is output through a DVP interface; the parallel image signal is a digital signal, and the parallel image signal may include first data corresponding to a first image.

[0055] In one embodiment, the serial image signal may further include a pixel clock signal, synchronization code data, and a data packet ID; the image sensor 100 outputs the serial image signal through the MOSI channel, and the serial image signal may include a pixel clock signal (PLCK), synchronization code data (Sync code), a data packet ID (Packet ID), and first data (data output from D0 to D7). The serial clock frequency (SCLK) is determined according to the sampling frequency and the number of sampling bits of the image sensor 100.

[0056] like Figure 6 As shown, before step 1, the method may further include:

[0057] Step 0: Send a master clock signal to the image sensor 100, and the pixel clock signal is obtained by the image sensor 100 by dividing the master clock signal.

[0058] like Figure 6 As shown, the above step 2 may include:

[0059] Step 210: Process the serial image signal to obtain a pixel clock signal, synchronization code data, data packet ID and first data. By reading the frame header and frame tail data and line header and line tail data in the serial image signal, a synchronization signal and a frequency synchronization signal can be obtained.

[0060] Step 220: Obtain a line synchronization signal and a frequency synchronization signal according to the synchronization code data and the data packet ID. The parallel image signal may include a pixel clock signal, a line synchronization signal, a frequency synchronization signal and first data.

[0061] In one embodiment, the above step 220 may include:

[0062] Step 221, determine whether the synchronization code data and the frame header data packet in the data packet ID are the same as the preset frame start synchronization header. If so, determine that the frequency synchronization signal is valid and output the frequency synchronization signal. In other words, when the frame header data meets the preset frame start synchronization header, it proves that the image transmission module is normally collecting images at this time and generates the system preset frame synchronization signal. If not, continue to collect serial image data until the matching frame header data is parsed.

[0063] Step 222, determine whether the row header data packet in the synchronization code data and the data packet ID is the same as the preset row start synchronization header. If so, determine that the row synchronization signal is valid and output the row synchronization signal. Similarly, the row header data conforms to the preset row start synchronization header, proving that the row frequencies are synchronized, and the first data can be acquired and transmitted. If not, continue to collect serial image data until the matching frame header data and row header data are parsed.

[0064] Step 223, determine whether the synchronization code data and the end-of-line data packet in the data packet ID are the same as the preset end-of-line synchronization header. If not, output the first data through the DVP interface; if so, determine that the frequency synchronization signal is invalid, and stop outputting the first data. Before the end of the line is detected, it means that the data corresponding to the current line of the first image of the current frame has not been completed. Collection and transmission, when the end-of-line data is detected, it means that the data corresponding to the current line of the first image of the current frame has been completed. Transmission, the bridge processing module 20 can collect data for the next line. .

[0065] Step 224: Determine whether the total number of rows of the synchronization code data and the frame tail data packet in the data packet ID is the same as the preset number of rows. If so, determine that the frequency synchronization signal is invalid and complete the acquisition of a frame of serial image signal. When all rows are acquired, it indicates that the first image of the current frame has been acquired, and the next frame acquisition can be performed or the image acquisition is completed.

[0066] Step 225, determine whether the serial image signals of all frames have been collected. If so, stop collecting the serial image signals; if not, re-collect a new frame of serial image signals. For example, the image processing module 30 controls the image sensor 100 to work according to a preset collection time, or, under a control instruction triggered by a user, controls the image sensor 100 to work or stop working.

[0067] Step 3: Figure 7 As shown, a parallel image signal is acquired, a first image is processed, and at least one of the resolution, color, brightness, sharpness and contrast of the first image is adjusted to improve the image quality of the first image.

[0068] In one embodiment, in the above step 3, processing the first image may include:

[0069] Step 310: Perform 3D noise reduction and white balance processing on the first image. The original format of the first image in the serial image signal output by the image sensor 100 is the RAW format. The RAW image is an unprocessed raw data image, which can provide a very large post-processing space. 3D digital noise reduction processing can reduce the noise interference of weak signal images. White balance processing can restore the color effect of the image, and the white balance can be adjusted automatically or manually.

[0070] Step 320: Perform a demosaicing process on the first image in RAW format to obtain a first image in RGB format. The demosaicing process can restore missing pixel values ​​from the raw image to form a standard RGB image.

[0071] Step 330: Perform format conversion on the first image in RGB format to obtain a first image in YUV format.

[0072] In one embodiment, before step 310, step 3 may further include:

[0073] Step 300: Perform black level correction on the first image. The image sensor 100 itself has dark current, which results in a certain output voltage even when there is no light. Therefore, this voltage value needs to be subtracted during the image processing, otherwise it will affect the subsequent white balance modules of the image.

[0074] In one embodiment, in step 310, before performing 3D noise reduction and white balance processing on the first image, step 3 may further include:

[0075] Step 311, perform bad pixel correction processing on the first image. If there are bad pixels in the image, the ISP will affect the surrounding pixel values ​​when performing interpolation and filtering processing. Therefore, the bad pixels need to be corrected before interpolation and filtering. If there are many bad pixels or many dynamic bad pixels in the image, false colors will appear on the edge of the image. This phenomenon not only affects the clarity of the image, but also affects the color of the edge;

[0076] In one embodiment, after step 310, step 3 may further include:

[0077] Step 312: Perform lens shading correction on the first image. The uneven image brightness caused by the vignetting phenomenon will affect the accuracy of subsequent processing. Therefore, the digital signal output from the image sensor 100 must first be processed by lens correction to eliminate the effect of vignetting on the first image.

[0078] In one embodiment, after step 320, step 3 may further include:

[0079] Step 321: Perform color correction on the first image in RGB format. When the image sensor 100 obtains a digital image, the acquired image is often very different from the original image due to the influence of ambient lighting or human factors. Color correction can reduce this difference to a certain extent.

[0080] In one embodiment, after step 330, step 3 may further include:

[0081] Step 340: Perform color processing and brightness processing on the first image in YUV format. During the image transmission process, due to environmental interference or device limitations, the image received in the end often contains a lot of noise, and the image becomes very blurred. Therefore, it is necessary to use filtering, denoising and other methods to perform color processing and brightness processing to improve image quality.

[0082] The color processing of the first image in the YUV format may include at least one of color scaling, color filtering and color denoising. The brightness processing of the first image in the YUV format may include at least one of brightness denoising, sharpening and brightness scaling.

[0083] In step 3, a variety of image processing steps are involved. For each step, those skilled in the art can use the corresponding disclosed algorithm, or further design an algorithm. The algorithm involved in the specific steps is not described in detail in this application. Even if the existing disclosed algorithm is used, after the image is processed in step 3, the image quality of the first image can be improved, so that the image clarity is improved, the noise is reduced, and the color is better, so as to meet the application requirements of the endoscope.

[0084] In one embodiment, after step 3, the method may further include:

[0085] Step 4: resample the processed first image according to a preset output resolution, wherein the resampling includes color scaling and brightness scaling to change the resolution of the current first image.

[0086] The design of the bridge chip (bridge processing module 20) greatly facilitates the selection of the ISP image processing chip 9 (image processing module 30). There are many chips to choose from, which well guarantees the principle of low cost. Here, the SSC336 chip is used as an example to illustrate that the main function of the ISP image processing chip is to collect the DVP video signal data output by the bridge chip, and then perform a series of denoising, white balance, color correction, gamma correction, sharpening, etc. through the image processing algorithm in the following steps, so that the image quality can fully meet the clinical needs of medical endoscopes.

[0087] The present application reduces the cost of the imaging module to about 1% of the original general solution by designing an image module with an SPI interface (including an image sensor 100 and a housing 11, etc.), fully realizing the low-cost requirement while ensuring the diameter requirement of the ultra-thin medical endoscope. At the same time, due to the digital video signal designed as an SPI interface, external interference can be avoided to a greater extent, ensuring the cleanliness of the image.

[0088] By designing a bridge chip to complete the conversion of SPI non-standard video signals to standard video signals DVP, it is possible to use the common ISP chips on the market to optimize the image quality, which can ensure the excellent image quality to the greatest extent.

[0089] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0090] This document is described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope of this document. For example, various operating steps and components for performing the operating steps may be implemented in different ways (e.g., one or more steps may be deleted, modified, or incorporated into other steps) depending on the specific application or considering any number of cost functions associated with the operation of the system.

[0091] Although the principles of this invention have been shown in various embodiments, many modifications of structures, arrangements, proportions, elements, materials and components particularly suitable for specific environments and operational requirements can be used without departing from the principles and scope of this invention. The above modifications and other changes or amendments will be included in the scope of this invention.

[0092] The foregoing specific description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of the present disclosure. Therefore, the consideration of the present disclosure will be illustrative rather than restrictive, and all these modifications will be included in its scope. Similarly, the advantages, other advantages and solutions to the problems of various embodiments have been described above. However, the benefits, advantages, solutions to the problems and any elements that can produce these, or make them more clear, should not be interpreted as critical, necessary or necessary. The term "include" and any other variants used in this article are all non-exclusive inclusions, so that the process, method, article or device including the list of elements not only includes these elements, but also includes other elements that are not explicitly listed or do not belong to the process, system, method, article or device. In addition, the term "coupled" and any other variants used in this article refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections and / or any other connections.

[0093] Those skilled in the art will appreciate that many changes may be made to the details of the above-described embodiments without departing from the basic principles of the invention. Therefore, the scope of the present invention should be determined solely by the claims.

Claims

1. An image acquisition and processing system, characterized in that: include: Image acquisition module, bridge processing module and image processing module; The image acquisition module includes an image sensor; the image sensor has an SPI interface, which is used to acquire a first image of the area to be acquired, and send a serial image signal to the bridge processing module through the SPI interface; the serial image signal is a digital signal, and the serial image signal includes first data corresponding to the first image; The bridge processing module has an SPI interface and a DVP interface, which are used to obtain and process the serial image signal to obtain a parallel image signal, and send the parallel image signal to the image processing module through the DVP interface; the parallel image signal is a digital signal, and the parallel image signal includes first data corresponding to the first image; The image processing module has a DVP interface, which is used to control the operation of the image sensor, obtain the parallel image signal, process the first image, and adjust at least one of the resolution, color, brightness, sharpness and contrast of the first image to improve the image quality of the first image; The serial image signal also includes a pixel clock signal, synchronization code data, a data packet ID and the first data; The bridge processing module is further used to send a main clock signal to the image sensor, and the pixel clock signal is obtained by the image sensor according to the frequency division of the main clock signal; the bridge processing module is further used to obtain the pixel clock signal, the synchronization code data and the data packet ID, obtain the line synchronization signal and the frequency synchronization signal according to the synchronization code data and the data packet ID, and send the pixel clock signal, the line synchronization signal, the frequency synchronization signal and the first data to the image processing module; Wherein, obtaining a line synchronization signal and a frequency synchronization signal according to the synchronization code data and the data packet ID includes: Determine whether the synchronization code data and the frame header data packet in the data packet ID are the same as the preset frame start synchronization header, and if so, determine that the frequency synchronization signal is valid and output the frequency synchronization signal; Determine whether the row header data packet in the synchronization code data and the data packet ID is the same as the preset row start synchronization header, and if so, determine that the row synchronization signal is valid and output the row synchronization signal; Determine whether the synchronization code data and the end-of-line data packet in the data packet ID are the same as the preset end-of-line synchronization header, and if not, output the first data through the DVP interface; if so, determine that the frequency synchronization signal is invalid, and stop outputting the first data; Determine whether the total number of rows of the synchronization code data and the frame tail data packet in the data packet ID is the same as the preset number of rows. If so, determine that the frequency synchronization signal is invalid, and complete the acquisition of one frame of the serial image signal; It is determined whether the serial image signals of all frames have been collected. If so, the collection of the serial image signals is stopped; if not, a new frame of the serial image signals is collected again.

2. The image acquisition and processing system according to claim 1, characterized in that: The maximum outer diameter of the image acquisition module is less than or equal to 1 / 15 inch; and / or, the bridge processing module includes a programmable bridge chip; and / or, the image processing module includes an image processing chip.

3. An image acquisition and processing method for the image acquisition and processing system according to claim 1, characterized in that: include: Capturing a first image of the area to be captured, obtaining a serial image signal corresponding to the first image, wherein the serial image signal is output through an SPI interface of the image sensor, the serial image signal is a digital signal, and the serial image signal includes first data corresponding to the first image; Acquire the serial image signal and process it to obtain a parallel image signal, and output the parallel image signal through the DVP interface; the parallel image signal is a digital signal, and the parallel image signal includes first data corresponding to the first image; Acquire the parallel image signal, process the first image, and adjust at least one of the resolution, color, brightness, sharpness, and contrast of the first image to improve the image quality of the first image; The serial image signal also includes a pixel clock signal, synchronization code data and a data packet ID; Before performing a first image acquisition on the area to be acquired, the method further includes: Sending a main clock signal to the image sensor, wherein the pixel clock signal is obtained by the image sensor according to the frequency division of the main clock signal; Acquiring and processing the serial image signal to obtain a parallel image signal includes: Processing the serial image signal to obtain the pixel clock signal, synchronization code data, data packet ID and first data; Obtaining a line synchronization signal and a frequency synchronization signal according to the synchronization code data and the data packet ID, wherein the parallel image signal includes the pixel clock signal, the line synchronization signal, the frequency synchronization signal and the first data; Wherein, obtaining a line synchronization signal and a frequency synchronization signal according to the synchronization code data and the data packet ID includes: Determine whether the synchronization code data and the frame header data packet in the data packet ID are the same as the preset frame start synchronization header, and if so, determine that the frequency synchronization signal is valid and output the frequency synchronization signal; Determine whether the row header data packet in the synchronization code data and the data packet ID is the same as the preset row start synchronization header, and if so, determine that the row synchronization signal is valid and output the row synchronization signal; Determine whether the synchronization code data and the end-of-line data packet in the data packet ID are the same as the preset end-of-line synchronization header, and if not, output the first data through the DVP interface; if so, determine that the frequency synchronization signal is invalid, and stop outputting the first data; Determine whether the total number of rows of the synchronization code data and the frame tail data packet in the data packet ID is the same as the preset number of rows. If so, determine that the frequency synchronization signal is invalid, and complete the acquisition of one frame of the serial image signal; It is determined whether the serial image signals of all frames have been collected. If so, the collection of the serial image signals is stopped; if not, a new frame of the serial image signals is collected again.

4. The image acquisition and processing method according to claim 3, characterized in that: Processing the first image includes: Performing 3D noise reduction and white balance processing on the first image; Performing demosaic processing on the first image in RAW format to obtain the first image in RGB format; The first image in RGB format is format converted to obtain the first image in YUV format.

5. The image acquisition and processing method according to claim 4, characterized in that: Before performing 3D noise reduction and white balance processing on the first image, processing the first image further includes: performing black level correction processing on the first image; And / or, before performing 3D noise reduction and white balance processing on the first image, processing the first image further includes: performing bad pixel correction processing on the first image; And / or, after performing 3D noise reduction and white balance processing on the first image, processing the first image further includes: performing lens shading correction processing on the first image; And / or, after performing de-mosaic processing on the first image in RAW format, processing the first image further includes: performing color correction processing on the first image in RGB format.

6. The image acquisition and processing method according to claim 5, characterized in that: After converting the format of the first image in RGB format, processing the first image further includes: Performing color processing and brightness processing on the first image in YUV format; The color processing of the first image in the YUV format includes at least one of color scaling, color filtering and color denoising; The brightness processing of the first image in the YUV format includes at least one of brightness noise reduction, sharpening and brightness scaling.

7. A computer-readable storage medium, characterized in that: The medium stores a program, which can be executed by a processor to implement the image acquisition and processing method as described in any one of claims 3 to 6.

Images