A medical endoscope image optimization system and method

Abstract

The application provides a medical endoscope image optimization system and method, and relates to the technical field of endoscopes.The system comprises a snapshot storage module, a large database and an analysis module.The analysis module comprises a texture structure analysis module, a comparison module, an AI image expansion module, a data set arrangement module and a packing output module.The snapshot storage module is used to connect an endoscope device, collect snapshot image data and store the data.The large database is used to store internal organ images required by the endoscope and corresponding organ texture structure vector data.The application analyzes the texture and structure vector data of the snapshot images, compares the data with the data in the large database, determines the internal organ to which each snapshot image belongs, simultaneously expands the snapshot images through AI image expansion technology, determines the position sequence of a plurality of snapshot images representing the corresponding internal organ, and classifies and outputs the images to a display device, so as to facilitate medical staff in determining the organ and position of the images and improving efficiency.

Description

Technical Field

[0001] This invention relates to the field of endoscopy technology, and more particularly to a medical endoscopy image optimization system and method. Background Technology

[0002] A medical endoscope is a medical device consisting of a flexible part, a light source, and a set of lenses. The endoscope is inserted into the human body through natural orifices or through a small incision made during surgery. When in use, the endoscope is guided into the organ to be examined, and changes in the relevant parts can be directly observed. The quality of the image directly affects the effectiveness of the endoscope and also marks the level of development of endoscopic technology.

[0003] During endoscopic examinations, doctors insert an endoscope into the patient's body. The endoscope lens transmits images of the patient's internal organs. Doctors then capture necessary images based on these images and analyze them to arrive at a conclusion. However, analyzing these images, especially when examining multiple organs, is cumbersome due to the nature of the capture process. It requires recall or image analysis to determine which organ within the patient's body the captured image corresponds to, and its location within that organ, for a comprehensive analysis. This is time-consuming and labor-intensive. Therefore, this invention proposes a medical endoscopic image optimization system and method to address the problems existing in the prior art. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a medical endoscope image optimization system and method. This system and method facilitate medical personnel in identifying the organ and location of an image, thereby improving diagnostic and treatment efficiency.

[0005] To achieve the objectives of this invention, the invention is implemented through the following technical solution: a medical endoscope image optimization system, comprising an image capture and storage module, a large database, and an analysis module, wherein the analysis module includes a texture construction analysis module, a comparison module, an AI image expansion module, a dataset arrangement module, and a packaged output module;

[0006] The image capture and storage module is used to connect to the endoscope device, capture and store the captured image data. The large database is used to store the images of internal organs to be captured by the endoscope and their corresponding texture vector data. The texture analysis module is used to analyze the texture and structure vector data of the images stored in the image capture and storage module. The comparison module is used to compare the texture and structure vector data analyzed by the texture analysis module with the data in the large database to determine the internal organ to which each captured image belongs. The AI ​​image expansion module uses the large database as the analysis basis to perform AI image expansion on the captured images and determine the positional order of several captured images representing the corresponding internal organs. The dataset arrangement module is used to classify and set the captured images according to the internal organs they represent, and sort them in the set according to the position represented by each captured image. The packaging and output module is used to output each set to the display device.

[0007] Further improvements are made in that: the capture and storage module includes a data transmission module, a storage unit, and a time calibration module. The data transmission module is used to connect to the endoscope device, with analog RGB and HDMI input interfaces, to receive video and capture images captured by the endoscope, and to convert and output an 8-bit digital image data stream. The storage unit is used to store the 8-bit digital image data stream and to store the capture images in the original image format.

[0008] A further improvement is that the time calibration module is built into the storage device, used to timestamp the captured images and videos in the storage device, and to provide a function for searching by time.

[0009] Further improvements include: the large database includes a NoSQL database and a search engine. The NoSQL database connects to the data websites of hospitals and relevant departments, collecting images of internal organs required for endoscopy and their corresponding texture vector data. The NoSQL database is MongoDB, which, as a non-relational database, reflects the relationships between data and is used for data interaction. The NoSQL database has a hash encryption module to prevent data theft. The search engine uses the ElasticSearch search server to build a distributed, multi-user full-text search engine and utilizes ELK technology to achieve human-computer interactive online retrieval.

[0010] Further improvements are made in that: the texture construction analysis module includes an image processing module and a vector data extraction module. The image processing module is used to process the captured image, and to analyze and optimize the image resolution, image refresh rate, image noise, image white balance, image brightness, and image red, green and blue saturation. The vector data extraction module is used to extract the texture features and construction features of the captured image, restore the corresponding 3D model, and output feature vector data.

[0011] Further improvements are made in that: the comparison module is connected to the texture construction analysis module and the big data, and a convolutional neural network is used to compare and learn the feature vector data analyzed by the texture construction analysis module with the texture construction vector data in the big data. Each image is compared at least three times, and the comparison result with the highest weight is taken to determine the internal organs represented by each captured image and mark them.

[0012] Further improvements are made in the following: The AI ​​image expansion module includes an algorithm module and a sorting module. The algorithm module is based on an image super-resolution algorithm using deep learning and neural networks. It trains a neural network model based on images of internal organs in a large database. This model is used to extract local images to predict a high-resolution complete image. Then, the captured image to be expanded is input into the trained neural network model. Through the inference process of the neural network, a high-resolution complete image is generated. The sorting module compares the complete images generated from several captured images representing the corresponding internal organs, selects the complete image with the highest probability, determines the corresponding position of each captured image in the internal organ, and sorts the captured images into the complete image according to their positions.

[0013] Further improvements are made in the following ways: The dataset arrangement module includes a main set module and a subset module. The main set module is connected to the comparison module, which classifies all captured images according to their corresponding internal organs and sets them into multiple main sets. The subset module is connected to the AI ​​image expansion module, which sets the sorted images of several captured images in the complete image within each main set into a subset. The packaging output module is used to package the main set and subset into an HDMI signal and output it to the display device.

[0014] A method for optimizing medical endoscopic images includes the following steps:

[0015] S1: The endoscope lens is placed into the area of ​​the patient to be examined, and video feedback is transmitted. During this process, the doctor operates the endoscope to capture images.

[0016] S2: The image capture and storage module collects captured image data and timestamps it for storage;

[0017] S3: The texture construction analysis module optimizes the captured images and analyzes the texture and construction vector data of the images;

[0018] S4: The comparison module compares the analyzed texture and construction vector data with data in the large database to determine the internal organs to which each captured image belongs;

[0019] S5: The AI ​​image expansion module performs AI image expansion on several captured images of the corresponding internal organs, generates a complete image, performs overlapping comparison, determines the final complete image, and thus obtains the corresponding position of each captured image in the internal organ.

[0020] S6: The dataset sorting module classifies all captured images into a main set according to their corresponding internal organs. Within the main set, it sorts several captured images into subsets according to their positions, and then outputs them to the display device to assist doctors in treatment.

[0021] A further improvement is that, in S2, when a doctor needs the original image for data tracing, the data is retrieved from the capture and storage module according to time.

[0022] The beneficial effects of this invention are as follows:

[0023] 1. This invention analyzes the texture and structural vector data of captured images and compares them with data in a large database to determine the internal organs to which each captured image belongs. At the same time, it uses AI image expansion technology to expand the captured images to determine the positional order of several captured images representing the corresponding internal organs. These images are then classified and collected and output to a display device to help medical personnel determine the organs and locations to which the images belong, thereby improving diagnostic efficiency.

[0024] 2. When comparing the texture and structural vector data of captured images with data in a large database, this invention uses a convolutional neural network for comparison learning. Each image is compared at least three times, and the comparison result with the highest weight is selected to improve the accuracy of organ analysis. During AI image expansion, an image super-resolution algorithm based on deep learning and neural networks is used to build and train a neural network model. The captured image to be expanded is input into the model, and a high-resolution complete image is generated through inference. The complete images generated from several captured images are overlapped and compared, and the complete image with the highest probability is selected to improve the accuracy of location analysis.

[0025] 3. When doctors need the original image for data tracing, the present invention allows them to retrieve data by time in the capture and storage module, providing diverse functions. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the system of the present invention;

[0027] Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0028] To enhance understanding of the present invention, the present invention will be further described in detail below with reference to embodiments. These embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0029] Example 1

[0030] according to Figure 1As shown, this embodiment proposes a medical endoscope image optimization system, including an image capture and storage module, a large database, and an analysis module. The analysis module includes a texture construction analysis module, a comparison module, an AI image expansion module, a dataset arrangement module, and a packaged output module.

[0031] The image capture and storage module is used to connect to the endoscope device, capture and store the captured image data. The large database is used to store the images of internal organs to be captured by the endoscope and their corresponding texture vector data. The texture analysis module is used to analyze the texture and structure vector data of the images stored in the image capture and storage module. The comparison module is used to compare the texture and structure vector data analyzed by the texture analysis module with the data in the large database to determine the internal organ to which each captured image belongs. The AI ​​image expansion module uses the large database as the analysis basis to perform AI image expansion on the captured images and determine the positional order of several captured images representing the corresponding internal organs. The dataset arrangement module is used to classify and set the captured images according to the internal organs they represent, and sort them in the set according to the position represented by each captured image. The packaging and output module is used to output each set to the display device. When in use, the texture and structural vector data of the captured images are analyzed and compared with data in a large database to determine the internal organs to which each captured image belongs. At the same time, AI image expansion technology is used to expand the captured images to determine the positional order of several captured images representing the corresponding internal organs. These images are then classified and collected and output to the display device to help medical personnel determine the organs and locations to which the images belong, thereby improving diagnostic efficiency.

[0032] The image capture and storage module includes a data transmission module, a storage unit, and a time calibration module. The data transmission module connects to the endoscope device, uses analog RGB and HDMI input interfaces, receives video and captured images from the endoscope, and converts them into an 8-bit digital image data stream. The storage unit stores the 8-bit digital image data stream and stores the captured images in their original format, facilitating rapid and compressed acquisition.

[0033] The time stamping module is built into the storage unit and is used to timestamp the captured images and videos in the storage unit, and provides a time-based retrieval function. When doctors need the original images for data tracing, they can retrieve data by time from the capture storage module, offering diverse functionality.

[0034] The large database includes a NoSQL database and a search engine. The NoSQL database connects to data websites of hospitals and relevant departments, collecting images of internal organs taken for endoscopy and their corresponding texture vector data. The NoSQL database is MongoDB, a non-relational database that reflects the relationships between data and is used for data interaction. The NoSQL database has a hash encryption module to prevent data theft and improve security. The search engine uses the ElasticSearch search server to build a distributed, multi-user full-text search engine, utilizing ELK technology to achieve interactive online retrieval. When data in the NoSQL database is insufficient, data can be retrieved online through the search engine.

[0035] The texture construction analysis module includes an image processing module and a vector data extraction module. The image processing module is used to process the captured image, analyze and optimize the image resolution, image refresh rate, image noise, image white balance, image brightness, and image red, green and blue saturation. The vector data extraction module is used to extract the texture features and construction features of the captured image, restore the corresponding 3D model, and output feature vector data.

[0036] Specifically, the image processing module employs the VCA algorithm to extract noisy endmember spectra from the image, performs SSA algorithm denoising on the extracted endmember spectra, extracts the endmember spectrum Y using the endmember extraction algorithm, calculates its trajectory matrix, and then performs singular value decomposition, grouping, and reconstruction to obtain the denoised and demixed image. The super-resolution algorithm SRCNN, based on a convolutional neural network, takes the original video image as input, uses an upsampling algorithm to upscale the resolution to a set value, and then performs three layers of convolution operations (9*9*128, 3*3*64, and 5*5) to obtain the super-resolution output, thus enhancing the image resolution.

[0037] The vector data extraction module works as follows: ENVI software's filtering function is used to filter the image along its overall direction to extract texture. The extracted texture results are then placed in ArcGIS for density analysis to determine the image's texture density. Next, the second derivative of the image's spectrum is calculated. A second derivative calculation algorithm is written in ENVI IDL to determine the texture feature value of each image. Then, the image is stretched and vectorized into points, lines, and surfaces to determine its shape. Finally, the above data is used to reconstruct the corresponding 3D model, outputting it as feature vector data.

[0038] The comparison module connects to the texture construction analysis module and a large database. It uses a convolutional neural network to compare the feature vector data analyzed by the texture construction analysis module with the texture construction vector data in the large database. Each image is compared at least three times, and the comparison result with the highest weight is taken to determine the internal organs represented by each captured image and label them. The DGL neural network is integrated with mainstream deep learning frameworks, enabling free conversion from traditional tensor operations to graph operations. DGL provides a message-passing-based programming model to complete graph computations, can handle various types of graph data, and supports comparing feature data on different types of nodes and edges. Users can easily create graphs and graph neural networks, and perform training and comparative evaluation.

[0039] Example 2

[0040] according to Figure 1 As shown, this embodiment proposes a medical endoscope image optimization system, including an image capture and storage module, a large database, and an analysis module. The analysis module includes a texture construction analysis module, a comparison module, an AI image expansion module, a dataset arrangement module, and a packaged output module.

[0041] The image capture and storage module is used to connect to the endoscope device, capture and store the captured image data. The large database is used to store the images of internal organs to be captured by the endoscope and their corresponding texture vector data. The texture analysis module is used to analyze the texture and structure vector data of the images stored in the image capture and storage module. The comparison module is used to compare the texture and structure vector data analyzed by the texture analysis module with the data in the large database to determine the internal organ to which each captured image belongs. The AI ​​image expansion module uses the large database as the analysis basis to perform AI image expansion on the captured images and determine the positional order of several captured images representing the corresponding internal organs. The dataset arrangement module is used to classify and set the captured images according to the internal organs they represent, and sort them in the set according to the position represented by each captured image. The packaging and output module is used to output each set to the display device. When in use, the texture and structural vector data of the captured images are analyzed and compared with data in a large database to determine the internal organs to which each captured image belongs. At the same time, AI image expansion technology is used to expand the captured images to determine the positional order of several captured images representing the corresponding internal organs. These images are then classified and collected and output to the display device to help medical personnel determine the organs and locations to which the images belong, thereby improving diagnostic efficiency.

[0042] The comparison module connects to the texture construction analysis module and a large database. It uses a convolutional neural network to compare the feature vector data analyzed by the texture construction analysis module with the texture construction vector data in the large database. Each image is compared at least three times, and the comparison result with the highest weight is taken to determine the internal organs represented by each captured image and label them. The DGL neural network is integrated with mainstream deep learning frameworks, enabling free conversion from traditional tensor operations to graph operations. DGL provides a message-passing-based programming model to complete graph computations, can handle various types of graph data, and supports comparing feature data on different types of nodes and edges. Users can easily create graphs and graph neural networks, and perform training and comparative evaluation.

[0043] The AI ​​image expansion module includes an algorithm module and a sorting module. The algorithm module is based on deep learning and neural network image super-resolution algorithms. It trains a neural network model based on images of internal organs in a large database. This model is used to extract local images to predict high-resolution complete images. Then, the captured images to be expanded are input into the trained neural network model. Through the inference process of the neural network, a high-resolution complete image is generated. The sorting module compares the complete images generated from several captured images representing the corresponding internal organs, selects the complete image with the highest probability, determines the corresponding position of each captured image in the internal organ, and sorts the captured images into the complete image according to their positions. This study employs a PyTorch Geometic neural network, a deep learning library built on PyTorch, to model and train unstructured data. High-performance training is achieved using a dedicated CUDA kernel. Based on the PyTorch graph neural network framework, it supports various graph neural network models, including Graph Convolutional Networks (GCN), Graph Attention Networks (GAT), and a variant of Graph Convolutional Networks (GraphSAGE). It also provides a large number of graph datasets and preprocessors. AI image augmentation is used in medical imaging to improve image quality and accuracy. AI image augmentation leverages the feature extraction capabilities of neural networks to retain more details in the original image, making the magnified image more realistic and clear. The neural network model for AI image augmentation can automatically learn the structural and textural features of images through training data, thus adapting to different types of image magnification tasks.

[0044] The dataset arrangement module includes a main set module and a subset module. The main set module connects to the comparison module, classifying all captured images according to their corresponding internal organs and creating multiple main sets. The subset module connects to the AI ​​image expansion module, sorting several captured images within each main set into a subset based on their order within the complete image. The packaging and output module packages the main sets and subsets into an HDMI signal for output to the display device. Sending data in sets ensures greater completeness and faster processing.

[0045] Example 3

[0046] according to Figure 2 As shown in the figure, this embodiment proposes a method for optimizing medical endoscope images, including the following steps:

[0047] The endoscope lens is placed into the area of ​​the patient to be examined, and video feedback is transmitted. During this process, the doctor operates the endoscope to capture images.

[0048] The image capture and storage module collects captured image data and timestamps it for storage; when doctors need the original image for data tracing, the data can be retrieved from the image capture and storage module according to time.

[0049] The texture construction analysis module optimizes the captured images and analyzes the texture and construction vector data of the images;

[0050] The comparison module compares the analyzed texture and structural vector data with data in a large database to determine the internal organs to which each captured image belongs.

[0051] The AI ​​image expansion module expands several captured images of the corresponding internal organs using AI image expansion, generates a complete image, and performs overlapping comparison to determine the final complete image, thereby obtaining the corresponding position of each captured image in the internal organ.

[0052] The dataset sorting module categorizes all captured images into a main set according to their corresponding internal organs. Within the main set, several captured images are sorted into subsets based on their positions, and then output to the display device to assist doctors in treatment.

[0053] This invention analyzes the texture and structural vector data of captured images and compares them with data in a large database to determine the internal organs to which each captured image belongs. Simultaneously, it uses AI image expansion technology to expand the captured images, determining the positional order of several captured images representing the corresponding internal organs. This classification and set is then output to a display device, facilitating medical personnel in identifying the organs and locations of images and improving diagnostic efficiency. Furthermore, when comparing the texture and structural vector data of captured images with data in the large database, this invention employs a convolutional neural network for comparison learning. Each image is compared at least three times, and the comparison result with the highest weight is selected, improving the accuracy of organ analysis. During AI image expansion, an image super-resolution algorithm based on deep learning and neural networks is used to construct and train a neural network model. The captured image to be expanded is input into the model, which then infers and generates a high-resolution complete image. The complete images generated from several captured images are overlapped and compared, and the complete image with the highest probability is selected, improving the accuracy of location analysis. Additionally, when doctors need the original images for data tracing, this invention allows data to be retrieved by time in the captured image storage module, offering diverse functionality.

[0054] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A medical endoscope image optimization system, comprising an image capture and storage module, a large database, and an analysis module, characterized in that: The analysis module includes a texture construction analysis module, a comparison module, an AI image expansion module, a dataset arrangement module, and a packaging output module; The image capture and storage module is used to connect to the endoscope device, capture and store the captured image data. The large database is used to store the images of internal organs to be captured by the endoscope and their corresponding texture vector data. The texture analysis module is used to analyze the texture and structure vector data of the images stored in the image capture and storage module. The comparison module is used to compare the texture and structure vector data analyzed by the texture analysis module with the data in the large database to determine the internal organ to which each captured image belongs. The AI ​​image expansion module uses the large database as the basis for analysis and is used to perform AI image expansion on the captured images to determine the positional order of several captured images representing the corresponding internal organs. The dataset arrangement module is used to arrange the captured images according to the internal organs they represent. The system categorizes images into sets, sorts them according to the position represented by each captured image, and outputs these sets to a display device. The capture storage module includes a data transmission module, a storage unit, and a time calibration module. The data transmission module connects to the endoscope, uses analog RGB and HDMI input interfaces, receives video and captured images from the endoscope, and converts them into 8-bit digital image data streams. The storage unit stores the 8-bit digital image data streams and stores the captured images in their original form. The time calibration module, built into the storage unit, timestamps the captured images and videos in the storage unit and provides time-based retrieval functionality. The large database includes NoSQL databases. The system includes a database and a search engine. The NoSQL database connects to the data websites of hospitals and relevant departments, collecting images of internal organs taken by endoscopy and their corresponding texture vector data. The NoSQL database is MongoDB, a non-relational database that reflects the relationships between data and is used for data interaction. The NoSQL database has a hash encryption module to prevent data theft. The search engine uses the ElasticSearch search server to build a distributed, multi-user full-text search engine and utilizes ELK technology to achieve online retrieval with human-computer interaction.

2. The medical endoscope image optimization system according to claim 1, characterized in that: The texture construction analysis module includes an image processing module and a vector data extraction module. The image processing module is used to process the captured image, analyze and optimize the image resolution, image refresh rate, image noise, image white balance, image brightness, and image red, green and blue saturation. The vector data extraction module is used to extract the texture features and construction features of the captured image, restore the corresponding 3D model, and output feature vector data.

3. The medical endoscope image optimization system according to claim 2, characterized in that: The comparison module connects to the texture construction analysis module and the big data database. It uses a convolutional neural network to compare and learn the feature vector data analyzed by the texture construction analysis module with the texture construction vector data in the big data database. Each image is compared at least three times, and the comparison result with the highest weight is taken to determine the internal organs represented by each captured image and mark them.

4. The medical endoscope image optimization system according to claim 3, characterized in that: The AI ​​image expansion module includes an algorithm module and a sorting module. The algorithm module is based on deep learning and neural network image super-resolution algorithms. It trains a neural network model based on images of internal organs in a large database. This model is used to extract local images to predict high-resolution complete images. Then, the captured images to be expanded are input into the trained neural network model. Through the inference process of the neural network, a high-resolution complete image is generated. The sorting module compares the complete images generated from several captured images representing the corresponding internal organs, selects the complete image with the highest probability, determines the corresponding position of each captured image in the internal organ, and sorts the captured images into the complete image according to their positions.

5. The medical endoscope image optimization system according to claim 4, characterized in that: The dataset arrangement module includes a main set module and a subset module. The main set module is connected to the comparison module, which classifies all captured images according to their corresponding internal organs and sets them into multiple main sets. The subset module is connected to the AI ​​image expansion module, which sets the sorted images of several captured images in the complete image within each main set into a subset. The packaging output module is used to package the main set and subset into an HDMI signal and output it to the display device.

6. A method for optimizing medical endoscope images, using the medical endoscope image optimization system described in claim 5, characterized in that, Includes the following steps: S1: The endoscope lens is placed into the area of ​​the patient to be examined, and video feedback is transmitted. During this process, the doctor operates the endoscope to capture images. S2: The image capture and storage module collects captured image data and timestamps it for storage; S3: The texture construction analysis module optimizes the captured images and analyzes the texture and construction vector data of the images; S4: The comparison module compares the analyzed texture and construction vector data with data in the large database to determine the internal organs to which each captured image belongs; S5: The AI ​​image expansion module performs AI image expansion on several captured images of the corresponding internal organs, generates a complete image, performs overlapping comparison, determines the final complete image, and thus obtains the corresponding position of each captured image in the internal organ. S6: The dataset sorting module classifies all captured images into a main set according to their corresponding internal organs. Within the main set, it sorts several captured images into subsets according to their positions, and then outputs them to the display device to assist doctors in treatment.

7. The method for optimizing medical endoscopic images according to claim 6, characterized in that: In S2, when a doctor needs the original image for data tracing, the data is retrieved from the capture and storage module according to time.

Images