An endoscope image tuning system
Through the endoscopic image tuning system, combined with signal processing and image processing technology, signal burrs are removed, noise reduction and demixing, resolution enhancement and jitter compensation are solved, and the problem of low endoscopic image imaging quality is realized and high-definition image display is achieved , provides more accurate conditions for medical judgment.
Patent Information
- Application Number
- CN202210339650.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-01
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-04-01
AI Technical Summary
The installation space of the endoscope lens is limited, resulting in low image imaging quality. The image clarity of simple sharpening processing in the prior art is limited and easily causes image distortion, affecting medical judgment.
The endoscopic image tuning system is adopted, including video image acquisition module, video image analysis module, video re-recording conversion module and video data storage module. The signal burrs are removed through filtering algorithms, noise reduction and demixing, resolution enhancement, and refresh rate increase and parameter adjustment, combined with gyroscope sensor for jitter compensation, use oleophobic film to reduce lens mucus, and achieve high-definition image display.
It significantly improves the clarity and imaging quality of the endoscopic image, provides more accurate medical judgment conditions, and optimizes the video image quality in multiple aspects to ensure the clarity and stability of the image.
Smart Images

Figure CN114827384B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of endoscopes, and in particular to an endoscope image tuning system. Background Art
[0002] An endoscope is a tube equipped with a light. It can enter the human body through natural orifices or through a small incision made during surgery. The original endoscopes were made of hard tubes, which have been gradually improved since then, but they are still not widely used. In the 1950s, endoscopes were made of soft tubes, so they could bend easily at corners in the human body. In 1965, Harold Hopkins installed a cylindrical lens on the endoscope to make the field of view clearer. Today's endoscopes usually have two fiberglass tubes, light enters the body through one of them, and the doctor observes through the other tube or through a camera. Some endoscopes even have micro-integrated circuit sensors that feed the observed information back to the computer.
[0003] At present, due to the limited installation space of the endoscope lens, the space occupied by the bottom of the lens CMOS is limited. Therefore, the imaging quality of the endoscope image has always been low, affecting the doctor's judgment of the patient's condition. In the prior art, a simple sharpening process of the image is generally adopted. This process increases the image clarity to a limited extent and is also prone to cause image distortion. Therefore, the present invention proposes an endoscope image tuning system to solve the problems existing in the prior art. Summary of the invention
[0004] In view of the above problems, the present invention proposes an endoscopic image tuning system, which optimizes the video image quality from multiple aspects, ensures the clarity of the image, and facilitates medical judgment.
[0005] To achieve the purpose of the present invention, the present invention is implemented through the following technical solutions: an endoscope image tuning system, including a video image acquisition module, a video image analysis module, a video re-recording conversion module and a video data storage module, the video image acquisition module is used to acquire video images taken by an endoscope and convert them into digital video images, and the video image acquisition module has a filtering algorithm to remove signal glitches during the acquisition process, the video image analysis module is used to convert digital video images into analog video images, and analyze the quality of the analog video images;
[0006] The video re-recording conversion module includes a video noise reduction and demodulation module, a resolution processing module, a refresh rate processing module and a parameter adjustment module. The video re-recording conversion module is used to perform noise reduction and demodulation, resolution enhancement, refresh rate improvement and parameter adjustment on analog video images. The video data storage module is used to encode, compress, cache and display the processed video data and store it locally.
[0007] A further improvement is that the video image acquisition module includes a conversion chip and a filtering algorithm module. The conversion chip simulates RGB and HDMI input interfaces to receive video images captured by an input endoscope and converts and outputs an 8-bit digital video image data stream.
[0008] A further improvement is that the filtering algorithm module identifies the starting point and the ending point of the burr during the signal acquisition process by differential threshold division, and uses the previous and next data to replace the data in the burr.
[0009] A further improvement is that the video image analysis module includes a decompression chip, an image quality analysis module and a trigger module. The decompression chip is used to decompress the digital video image acquired and converted by the video image acquisition module into an analog video image and simulate RGB and HDMI output interfaces.
[0010] Further improvement lies in: the image quality module is used to analyze the parameter baseline values of the analog video image after conversion by the decompression chip, specifically including image resolution, image refresh rate, image noise, image white balance, image brightness, image red, green and blue saturation; the trigger module determines whether the above analysis data meets the settings according to the system preset; if not, the image re-recording conversion module is started to process the video image.
[0011] A further improvement is that the video denoising and unmixing module uses a VCA algorithm to extract endmember spectra containing noise in the video image, performs SSA algorithm denoising on the extracted endmember spectra, extracts the endmember spectrum Y using an endmember extraction algorithm, calculates its trajectory matrix X, and then performs singular value decomposition, grouping and reconstruction to obtain a denoised and unmixed video image.
[0012] Further improvements are as follows: the resolution processing module adopts the super-resolution algorithm SRCNN, based on the convolutional neural network, takes the original video image as input, uses the upsampling algorithm to increase the resolution to the set value, and then passes through 3 layers of convolution operations of 9*9*128, 3*3*64, and 5*5 respectively to obtain the super-resolution output and enhance the video image resolution; the refresh rate processing module adopts the RIFT real-time video interpolation algorithm to interpolate the video image and adjust the refresh rate.
[0013] A further improvement is that the parameter adjustment module provides white balance adjustment, brightness adjustment, and red, green, and blue saturation adjustment functions based on a human-computer interactive display, and is used to perform secondary processing on the video after being cached and displayed by the video data storage module, and directly display it.
[0014] Further improvements are: the video data storage module includes a video codec module and a local storage module, the video codec module adopts H265 codec, encodes and compresses the video image processed by the video re-recording conversion module, and passes the encoded video stream to the local storage module, the local storage module includes a cache module and a storage, the video codec module sends the encoded video stream to the cache module, realizes high-definition image display through high-definition video DAC, and synchronously stores the data stream marked time in the storage.
[0015] Further improvements are: the endoscope includes a CCD camera, a mobile bracket, a gyro sensor and a processing chip, the CCD camera is installed on the mobile bracket, the gyro sensor detects the direction, speed and movement of the CCD camera shake, the processing chip processes the detected signal, calculates the CCD movement amount to offset the shake, controls the movement of the CCD camera on the mobile bracket, and the lens of the CCD camera is coated with an oleophobic film.
[0016] The beneficial effects of the present invention are:
[0017] 1. The present invention optimizes images by combining signal processing and image processing. In signal processing, a filtering algorithm module is used to remove signal glitches in the acquisition process, thereby optimizing the quality of video images. In image processing, analog video images are subjected to noise reduction and unmixing, resolution enhancement, refresh rate improvement, and manual parameter adjustment, thereby optimizing video image quality from multiple aspects, ensuring image clarity, and providing convenience for medical judgment.
[0018] 2. During the process of endoscopic shooting, the present invention detects the jitter of the CCD camera through a gyro sensor, cooperates with the processing chip analysis, and controls the movement of the CCD camera on the mobile bracket to perform jitter compensation, thereby preventing shaking and improving the quality of image shooting. In addition, the effect of the oleophobic film reduces the adhesion of mucus on the CCD camera lens, making the shooting clearer. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is a schematic diagram of the system composition of the present invention;
[0020] Figure 2 It is a schematic diagram of an endoscope of the present invention. DETAILED DESCRIPTION
[0021] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with examples. The examples are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.
[0022] Embodiment 1
[0023] according to Figure 1As shown, this embodiment proposes an endoscope image tuning system, including a video image acquisition module, a video image analysis module, a video re-recording conversion module and a video data storage module, wherein the video image acquisition module is used to acquire video images taken by the endoscope and convert them into digital video images, and the video image acquisition module is provided with a filtering algorithm to remove signal glitches during the acquisition process, and the video image analysis module is used to convert digital video images into analog video images and analyze the quality of the analog video images;
[0024] The video re-recording conversion module includes a video noise reduction and demodulation module, a resolution processing module, a refresh rate processing module and a parameter adjustment module. The video re-recording conversion module is used to perform noise reduction and demodulation, resolution enhancement, refresh rate improvement and parameter adjustment on analog video images. The video data storage module is used to encode, compress, cache and display the processed video data and store it locally.
[0025] The present invention optimizes images by combining signal processing and image processing. In signal processing, a filtering algorithm module is used to remove signal glitches in the acquisition process, thereby ensuring that there are no glitches in the process of the video image acquisition module acquiring the video image signal taken by the endoscope, thereby optimizing the quality of the video image. In image processing, the analog video image is subjected to noise reduction and unmixing, resolution enhancement, refresh rate improvement and manual parameter adjustment, thereby optimizing the video image quality from multiple aspects, ensuring the clarity of the image, and providing convenience for medical judgment.
[0026] Embodiment 2
[0027] according to Figure 1 As shown, this embodiment proposes an endoscope image tuning system, the video image acquisition module includes a conversion chip and a filtering algorithm module, the conversion chip simulates RGB, HDMI input interface, is used to receive the video image taken by the input endoscope, and converts and outputs 8-bit digital video image data stream. When used, it is used to convert data for fast compression acquisition.
[0028] The filtering algorithm module identifies the starting point and the ending point of the burr during the signal acquisition process through differential threshold division, and uses the previous and next data to replace the data in the burr.
[0029] The present invention adopts a filtering algorithm module to identify the starting point and the ending point of the burr in the signal acquisition process through differential threshold division, and uses the previous and next data to replace the data in the burr, so as to ensure that there are no burrs in the process of the video image acquisition module acquiring the video image signal taken by the endoscope, thereby optimizing the quality of the video image.
[0030] The video image analysis module includes a decompression chip, an image quality analysis module and a trigger module. The decompression chip is used to decompress the digital video image acquired and converted by the video image acquisition module into an analog video image and simulate RGB and HDMI output interfaces. When in use, it is used to decompress the data and restore the quality of the video image.
[0031] The image quality module is used to analyze the parameter reference values of the analog video image converted by the decompression chip, including image resolution, image refresh rate, image noise, image white balance, image brightness, and image red, green, and blue saturation. The trigger module determines whether the above analysis data meets the setting according to the system preset. If not, the image re-recording conversion module is started to process the video image. If it meets the setting, the trigger module automatically skips the image re-recording conversion module and directly transmits the composite standard image video data to the video data storage module.
[0032] The video denoising and unmixing module uses the VCA algorithm to extract the endmember spectrum containing noise in the video image, performs SSA algorithm denoising on the extracted endmember spectrum, extracts the endmember spectrum Y using the endmember extraction algorithm, calculates its trajectory matrix X, and then performs singular value decomposition, grouping and reconstruction to obtain the denoised and unmixed video image.
[0033] The resolution processing module adopts the super-resolution algorithm SRCNN, which is based on a convolutional neural network and takes the original video image as input. The upsampling algorithm is used to increase the resolution to a set value, and then three layers of convolution operations, 9*9*128, 3*3*64, and 5*5, are performed to obtain a super-resolution output and enhance the video image resolution. The refresh rate processing module adopts the RIFT real-time video interpolation algorithm to interpolate the video image and adjust the refresh rate.
[0034] The present invention extracts endmember spectra containing noise in video images through the VCA algorithm, performs SSA algorithm denoising on the extracted endmember spectra, uses the super-resolution algorithm SRCNN based on the convolutional neural network, takes the original video image as input, uses the upsampling algorithm to increase the resolution to a set value, and enhances the image resolution through convolution operation. The video image is interpolated through the RIFT real-time video interpolation algorithm to increase the refresh rate. In summary, the video image quality is optimized from multiple aspects, the image clarity is guaranteed, and convenience is provided for medical judgment.
[0035] The VCA algorithm is used to extract the noisy endmember spectrum. Based on the convex geometry theory, the endmembers are extracted under the assumption that there are pure pixels in the data. The convex cone is used to model the data. The projection on the hyperplane is a monomer with the endmember as the vertex. The data is projected onto the selected hyperplane. VCA projects all the pixels on the image into random directions, and takes the pixel with the largest projection as the first endmember. The remaining endmembers are extracted by iteratively projecting the data into a direction orthogonal to the subspace formed by the extracted endmembers. The pixels corresponding to the limit projection are taken as new endmembers. The extracted endmember spectrum is denoised by the SSA algorithm, and the endmember spectrum Y is extracted using the endmember extraction algorithm. Its trajectory matrix X is calculated. The specific process is: the one-dimensional endmember spectrum information Y T =(y1,…y T ) is converted into its trajectory matrix, and then the trajectory matrix X of the spectral information is calculated according to the window length L. The trajectory matrix X is of L×K order, where K=N-L+1. The calculation formula of the trajectory matrix X is shown in formula (1):
[0036]
[0037] Among them, X i =(x i ,x i+1 ,…,x i+L-1 ); the element x at (i,j) of the trajectory matrix X ij =x i+j-1 ; All elements on the anti-diagonal are equal;
[0038] The specific process of singular value decomposition is:
[0039] Define the matrix C x =XX T ,X T is the transposed matrix of X, and then calculates the matrix C x The eigenvalue λ i and the eigenvector U i , according to the order of eigenvalues from large to small, the eigenvalues are λ1≥…≥λ L ≥0, the corresponding eigenvectors are U1,…,U L , and then set d = L * ,L * =min{L,K}, U i With V i are the left and right eigenvectors of the trajectory matrix, U is called the time empirical orthogonal function, V is called the time principal component, and the trajectory matrix X can be synthesized by elementary matrices. The synthesis formula is shown in formula (2):
[0040] X=X1+…+X d (2)
[0041] The elementary matrix is in is the singular spectrum of X, where the maximum eigenvalue corresponds to the maximum eigenvector, representing the spectrum of the object, and the eigenvector corresponding to the smaller eigenvalue represents the noise;
[0042] The specific process of grouping is as follows:
[0043] The elementary matrix X i The subscripts {1, 2, ... d} are divided into p disjoint subsets I1, I2, ... I p , suppose I={i1,i2,…i m}, then the composite matrix X I =X i1 +X i2 +…+X im , calculate the set I = I1, I2, ..., I p For each composite matrix, the decomposition of formula (2) is expressed by formula (3):
[0044] X=X I1 +X I2 +…+X Ip (3)
[0045] Among them, select the set I1,…I p The process of grouping is called grouping;
[0046] The specific reconstruction process is:
[0047] Each X obtained by grouping formula (3) Ij Convert to new spectral data of length N, let Y be a L×K matrix, whose elements are y ij , where 1≤i≤L,1≤j≤K,L * =min{L,K},K * = max{L,K} and N = L+K-1, and transform the matrix Y into y1,…,y according to the diagonal average formula N The spectral information of the diagonal average formula is shown in formula (4):
[0048]
[0049] For the resultant matrix X Ik , a reconstruction sequence can be generated by formula (4) Then the initial spectral information Y T =(y1,…y T ) is decomposed into a reconstruction sequence with a sum of p, expressed as The denoised endmember spectrum is used to estimate the abundance according to the least squares method. The abundance estimation expression is shown in formula (5):
[0050]
[0051] Where E represents the endmember spectrum after denoising, x i is the original image.
[0052] The parameter adjustment module is based on a human-computer interactive display and provides white balance adjustment, brightness adjustment, and red, green, and blue saturation adjustment functions. The parameter adjustment module is used to perform secondary processing on the video after the video data storage module caches and displays it directly. During the image output and display process, the white balance, brightness, red, green, and blue saturation can also be adjusted by human-computer interaction, which is convenient for medical personnel to adjust according to their judgment habits.
[0053] The video data storage module includes a video codec module and a local storage module. The video codec module adopts H265 codec to encode and compress the video image processed by the video re-recording conversion module, and transmits the encoded video stream to the local storage module. The local storage module includes a cache module and a storage. The video codec module sends the encoded video stream to the cache module, realizes high-definition image display through high-definition video DAC, and synchronously stores the data stream marked time in the storage.
[0054] Embodiment 3
[0055] according to Figure 2 As shown, this embodiment proposes an endoscope image tuning system, the endoscope includes a CCD camera, a mobile bracket, a gyro sensor and a processing chip, the CCD camera is installed on the mobile bracket, the gyro sensor detects the direction, speed and movement of the CCD camera shake, the processing chip processes the detected signal, calculates the CCD movement amount to offset the shake, and controls the CCD camera to move on the mobile bracket, and the lens of the CCD camera is coated with an oleophobic film. During the process of endoscopic shooting, the present invention detects the CCD camera shake through the gyro sensor, cooperates with the processing chip to analyze, and controls the CCD camera to move left and right and up and down on the mobile bracket, so as to perform shake compensation, thereby preventing shaking and improving the quality of image shooting. In addition, the effect of the oleophobic film reduces the adhesion of mucus on the CCD camera lens, and the shooting is clearer.
[0056] The present invention optimizes the image by combining signal processing and image processing. In signal processing, a filtering algorithm module is used to identify the starting point and the ending point of the burr in the signal acquisition process through differential threshold division, and the data before and after are used to replace the data in the burr, so as to ensure that there is no burr in the process of the video image acquisition module acquiring the video image signal taken by the endoscope, and optimize the quality of the video image; in image processing, the end member spectrum containing noise in the video image is extracted by the VCA algorithm, and the extracted end member spectrum is denoised by the SSA algorithm. Through the super-resolution algorithm SRCNN, based on the convolutional neural network, the original video image is used as input, and the resolution is increased to a set value by the upsampling algorithm. After convolution operation, the resolution of the image is enhanced, and the video image is interpolated by the RIFT real-time video interpolation algorithm to improve the refresh rate. In summary, the video image quality is optimized from many aspects, the clarity of the image is guaranteed, and convenience is provided for medical judgment. In addition, in the process of image output and display, the present invention can also adjust the white balance, brightness, red, green, and blue saturation by human-computer interaction, so that medical personnel can adjust according to judgment habits. At the same time, during the process of endoscopic shooting, the present invention detects the jitter of the CCD camera through the gyro sensor, cooperates with the processing chip analysis, and controls the movement of the CCD camera on the mobile bracket to perform jitter compensation, thereby preventing shaking and improving the quality of image shooting. In addition, the effect of the oleophobic film reduces the adhesion of mucus on the CCD camera lens, making the shooting clearer.
[0057] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. An endoscope image tuning system, comprising a video image acquisition module, a video image analysis module, a video re-recording conversion module and a video data storage module, characterized in that: The video image acquisition module is used to acquire video images taken by the endoscope and convert them into digital video images, and the video image acquisition module has a filtering algorithm to remove signal glitches during the acquisition process. The video image analysis module is used to convert digital video images into analog video images and analyze the quality of the analog video images; The video re-recording conversion module includes a video noise reduction and unmixing module, a resolution processing module, a refresh rate processing module and a parameter adjustment module. The video re-recording conversion module is used to perform noise reduction and unmixing, resolution enhancement, refresh rate improvement and parameter adjustment on analog video images. The video data storage module is used to encode and compress the processed video data, cache display and local storage; The video image acquisition module includes a conversion chip and a filtering algorithm module. The conversion chip simulates RGB and HDMI input interfaces to receive video images captured by an input endoscope and converts and outputs 8-bit digital video image data streams. The filtering algorithm module identifies the starting point and the ending point of the burr during the signal acquisition process by dividing the differential threshold, and uses the previous and next data to replace the data in the burr; The video image analysis module includes a decompression chip, an image quality analysis module and a trigger module. The decompression chip is used to decompress the digital video image acquired and converted by the video image acquisition module into an analog video image and simulate RGB and HDMI output interfaces; The image quality module is used to analyze the parameter benchmark values of the analog video image converted by the decompression chip, specifically including image resolution, image refresh rate, image noise, image white balance, image brightness, and image red, green, and blue saturation. The trigger module determines whether the above analysis data meets the settings according to the system preset. If not, the image re-recording conversion module is started to process the video image.
2. The endoscope image tuning system according to claim 1, characterized in that: The video denoising and unmixing module uses the VCA algorithm to extract the endmember spectrum containing noise in the video image, performs SSA algorithm denoising on the extracted endmember spectrum, extracts the endmember spectrum Y using the endmember extraction algorithm, calculates its trajectory matrix X, and then performs singular value decomposition, grouping and reconstruction to obtain the denoised and unmixed video image.
3. The endoscopic image tuning system according to claim 2, characterized in that: The resolution processing module adopts the super-resolution algorithm SRCNN, which is based on a convolutional neural network and takes the original video image as input. The upsampling algorithm is used to increase the resolution to a set value, and then three layers of convolution operations, 9*9*128, 3*3*64, and 5*5, are performed to obtain a super-resolution output and enhance the video image resolution. The refresh rate processing module adopts the RIFT real-time video interpolation algorithm to interpolate the video image and adjust the refresh rate.
4. The endoscopic image tuning system according to claim 3, characterized in that: The parameter adjustment module is based on a human-computer interactive display and provides white balance adjustment, brightness adjustment, and red, green, and blue saturation adjustment functions. The parameter adjustment module is used to perform secondary processing on the video cached and displayed by the video data storage module and directly display it.
5. The endoscopic image tuning system according to claim 4, characterized in that: The video data storage module includes a video codec module and a local storage module. The video codec module adopts H265 codec to encode and compress the video image processed by the video re-recording conversion module, and transmits the encoded video stream to the local storage module. The local storage module includes a cache module and a storage. The video codec module sends the encoded video stream to the cache module, realizes high-definition image display through high-definition video DAC, and synchronously stores the data stream marked time in the storage.
6. An endoscopic image tuning system according to any one of claims 1 to 5, characterized in that: The endoscope includes a CCD camera, a mobile bracket, a gyro sensor and a processing chip. The CCD camera is installed on the mobile bracket. The gyro sensor detects the direction, speed and movement of the CCD camera shake. The processing chip processes the detected signal, calculates the CCD movement amount to offset the shake, and controls the movement of the CCD camera on the mobile bracket. The lens of the CCD camera is coated with an oleophobic film.
