751 results about "Fundus image" patented technology

According to the Mama and diffusion model-based OCTA fundus image generation system, method, medium and device provided by the invention, the unique Mama block is designed to effectively predict the accurate diffusion noise of the OCTA image from the OCT potential noise, the artifact of the generated image is reduced, and the contrast of the generated OCTA image is improved. Secondly, the proposed bidirectional Bow scanning scheme effectively alleviates the mechanism challenge of original Mama scanning, enhances the spatial continuity of the generated OCTA image, and improves the vascular extension of the generated image. Besides, the understanding ability of the OCT image is improved by combining a Mama block and a diffusion model through designed condition prompt, and the authenticity of the generated fundus OCTA image is further improved. According to experimental verification on two public data sets of OCTA500-3M and OCTA500-6M and one private data set, the SSIMs of 87.5% and 88.1% are achieved respectively, it is indicated that the fundus OCT image information can be fully utilized, the high-quality fundus OCTA image can be effectively generated, and new possibility is provided for low-cost screening of retinal diseases.

The invention provides a construction and training method and device for an eye fundus image multi-task model, and belongs to the field of image processing, and the method comprises the steps: S1, collecting and sorting a public eye fundus image data set, constructing an image text pair according to a real label, and carrying out the two-stage training of a multi-modal large language model, the multi-mode large language model comprises a visual encoder, a visual projector and a large language model; s2, inputting the image data # imgabs0 # into a visual encoder in the trained multi-modal large language model to obtain an enhanced visual feature # imgabs1 #, and extracting a visual feature # imgabs3 # from the # imgabs2 # through a visual projector; and S3, embedding the text input # imgabs4 # to obtain a text feature # imgabs5 #, splicing the text feature # imgabs5 # with the visual feature # imgabs6 #, and inputting the spliced text feature # imgabs5 # and the visual feature # imgabs6 # into a large language model to generate a prediction text A. According to the method, a wide range of fundus image data is collected for training, multilevel lesion features in the fundus image are fully utilized, and the performance of the model for executing a fundus disease auxiliary diagnosis task can be effectively improved.

The invention discloses an unsupervised deep learning image splicing method for portable fundus color photo equipment. The method comprises the following steps: constructing a multi-scale feature extraction module of an image registration network; the feature matching module is used for constructing an image registration network; the multi-scale feature extraction module and the feature matching module are sequentially connected to form the constructed image registration network; constructing a shallow feature extractor; constructing a feature reconstruction module; constructing a joint generator; sequentially connecting the shallow feature extractor, the feature reconstruction module and the seam generator to form a fusion network; constructing a fusion network; connecting the image registration network and the image fusion network to form an image splicing model; collecting data; preprocessing the data; obtaining a trained image registration network; obtaining a trained image fusion network; forming a fundus image splicing model by using the trained image registration network and the trained image fusion network; and splicing the images by using the eye fundus image splicing model. According to the method, a high-quality fundus spliced image can be efficiently generated.

The invention relates to the technical field of ophthalmologic image processing, and discloses a fundus blood vessel segmentation method based on multi-modal data. The method comprises the following steps: firstly, synchronously acquiring an optical coherence tomography image, a color fundus photographic image and a fluorescent angiography image to form multi-modal fundus data; preprocessing the eye fundus image data set to generate a standardized multi-mode eye fundus image data set; fusing heterogeneous features based on the data set, and constructing a multi-dimensional fundus feature space; using the space to train a deep learning segmentation network model, and generating an initial fundus blood vessel segmentation mask; carrying out confidence evaluation analysis on the initial mask to obtain a blood vessel segmentation result confidence distribution map; dividing blood vessel segmentation quality grades according to the distribution diagram, and defining a judgment rule; and performing interactive correction on the low-confidence region based on a rule to generate a final optimized fundus blood vessel segmentation result. And the final result is transmitted to a visual terminal for three-dimensional topology reconstruction. According to the method, the advantages of multi-modal data are integrated, and a more comprehensive fundus blood vessel segmentation result is formed.

The invention discloses a cross-modal eye fundus image generation method and system based on a generative adversarial network, relates to the technical field of medical image processing, and constructs an eye fundus focus perception and edge consistency generative adversarial network by taking a cyclic consistency generative adversarial network as a baseline. The core of the method is that a lesion perception mixed attention module is embedded in a bottleneck layer of a generator so as to strengthen the extraction capability of fine features of a lesion area; an edge information extraction module is designed, and key edge features are accurately extracted in combination with Roberts edge detection, wavelet transform and non-local mean denoising; and a joint loss function containing edge consistency loss is constructed, and the semantic consistency of a focus structure during cross-modal generation is ensured by minimizing the feature difference between the source image and the generated image. According to the method, the problems of disordered content, inconsistent structure and unstable training of the generated image in the prior art are effectively solved, and the simulation degree and clinical availability of the generated image are remarkably improved.

The invention discloses a heterogeneous double-flow fusion method and system for diabetic retinopathy grading. The method comprises the following steps: obtaining an output result of diabetic retinopathy grading by utilizing a heterogeneous double-flow architecture; processing an input fundus image into images with different resolutions; extracting global context features from the low-resolution image by using a lightweight visual Transform model distilled by composite knowledge, and extracting local focus features from the high-resolution image by using a convolutional neural network model; performing interactive fusion on the global context features and the local focus features of the double-branch architecture through a symmetric bidirectional cross attention fusion module to obtain enhanced fusion feature representation; and finally, inputting the fusion features into a classifier, and outputting a severity grading result of the lesion. The method aims at improving the accuracy and robustness of hierarchical diagnosis through deep analysis of global information and local details, and can be applied to the medical fields of clinical computer-aided diagnosis, eye image analysis and the like.

The invention discloses a multi-modal data glaucoma classification method and system, a terminal and a medium, and the method comprises the steps: obtaining a color fundus image and an OCT image in different image modes, and carrying out the advanced feature extraction of the color fundus image and the OCT image based on a first feature extraction module and a second feature extraction module, the mode specificity characteristics of the two modes are obtained; fusing the mode specific features of the two modes based on a shallow feature fusion module to obtain shallow fusion features; inputting the mode specific features of the two modes and the shallow fusion features into a deep feature fusion module for fusion to obtain final fusion features; and based on a multi-layer perceptron classifier, performing classification prediction on the final fusion feature to obtain a classification result, the classification result including no glaucoma, early glaucoma or mid-late glaucoma. According to the method, through two-stage feature extraction, two-stage feature fusion and the classifier, accurate glaucoma classification can be obtained.

The invention belongs to the field of artificial intelligence and fundus image enhancement, and discloses a fundus image enhancement method and system based on machine learning, electronic equipment and a storage medium, and the method comprises the steps: obtaining an original fundus spectral image, and carrying out the preprocessing of the original fundus spectral image, and obtaining a preprocessed spectral image; constructing a backbone network based on a residual network, and extracting multi-scale features of the preprocessed spectral image in combination with cavity convolution; introducing a channel-space-spectrum multi-attention module into the backbone network, and performing multi-attention fusion on the multi-scale features to obtain an enhanced feature map; and performing adversarial training on the backbone network by using the generative adversarial network and the enhanced feature map, and performing image enhancement on the collected fundus spectral image by using the trained network to obtain an enhanced fundus spectral image. According to the method, more-dimensional image support is provided for medical diagnosis, the quality of the fundus image can be effectively improved, the diagnosis accuracy of a doctor on fundus lesions is improved, and the method has important clinical application value.

An ophthalmic imaging system provides an automatic focus mechanism based on the difference of consecutive scan lines. The system also provides of user selection of a focus point within a fundus image. A neural network automatically identifies the optic nerve head in an FA or ICGA image, which may be used to determine fixation angle. The system also provides additional scan tables for multiple imaging modalities to accommodate photophobia patients and multi-spectrum imaging options.

The present application provides a medical image recognition method. The method includes obtaining a fundus image. A feature extraction model is invoked to extract features from the fundus image to obtain a fundus feature map. Once the fundus feature map is input into an attention mechanism model to obtain an attention-weighted feature map, a classification model is invoked to classify the attention-weighted feature map to obtain a target disease label of the fundus image. The present application also provides a method for training a medical image recognition model and an electronic device.

The invention provides a cross-modal fundus image registration method and device, equipment and a medium, and the method comprises the steps: converting a to-be-matched multi-modal image into a blood vessel image through a wide-area blood vessel segmentation network, the multi-modal image being two fundus images of different modals obtained by the same fundus of the same patient at different times and in different imaging modes; performing feature matching on the vascular image of the to-be-matched multi-modal image through the key point detection model to obtain a matching set; and realizing coordinate transformation of the to-be-matched multi-modal image by using a double-fitting strategy based on the matching set, and completing image registration. Therefore, according to the registration strategy, the best registration performance at present is achieved, the registration performance of other existing models is greatly improved through the cutting alignment strategy and the double-fitting strategy, and the cross-modal fundus image registration method can be widely applied to cross-modal fundus image registration under the large-view-difference condition.

The invention discloses an automatic retinal vessel segmentation method fusing large-kernel multi-scale cavity convolution and an attention mechanism, which is suitable for accurate extraction of a fine-grained vessel structure in a fundus image. According to the method, an improved deep neural network (LKD-UNet) is constructed, a collaborative receptive field module is embedded in each layer of an encoder, and large kernel convolution and multi-scale cavity convolution are combined to enlarge the receptive field and enhance the texture analysis capability; an efficient Transform module is introduced into a bottleneck layer, a cross-regional semantic relationship is modeled through a multi-head self-attention mechanism, and the topological consistency of microvessels is improved; the decoder adopts a shallow symmetric structure and a kernel scale scheduling strategy, and realizes high-resolution reduction in combination with jump connection. In addition, a four-category pixel-level error visualization mechanism is introduced and is used for assisting segmentation error diagnosis and result interpretation. Experimental results show that the method is superior to an existing mainstream model in multiple public data sets, has remarkable advantages in the aspects of segmentation precision, structural coherence and deployment efficiency, and is suitable for remote fundus screening and blood vessel extraction tasks in embedded medical equipment.

A state prediction method and apparatus, a computer device, a storage medium, and a computer program product. The method comprises: obtaining data for detection, wherein the data for detection comprises a fundus image and covariate data, and the covariate data is variable data for assisting in identifying a target lesion; on the basis of a preset feature extraction model, performing feature extraction on the fundus image and the covariate data respectively to obtain an image feature vector, a first weight corresponding to the image feature vector, a covariate feature vector, and a second weight corresponding to the covariate feature vector; and processing the image feature vector and the covariate feature vector on the basis of the first weight, the second weight, and a preset hybrid model to obtain a target state prediction result, wherein the preset hybrid model is constructed on the basis of the image feature vector and the covariate feature vector that separately satisfy a maximum likelihood function condition and under the conditions where the data for detection has censored data or does not have censored data.

The invention provides an eye fundus image intelligent enhancement method and system based on multi-modal image fusion. The method comprises the following steps of obtaining a color eye fundus photography CFP image and an optical coherence tomography OCT image and performing preprocessing; establishing an image space coordinate mapping relation for the preprocessed color fundus photography CFP image and the preprocessed optical coherence tomography OCT image, and performing feature space alignment; the first branch network structure and the second branch network structure of the multi-layer Transform feature fusion module are used for carrying out feature extraction and enhancement on the features of the processed color fundus photography CFP image and the features of the processed optical coherence tomography OCT image respectively, and the features of the color fundus photography CFP image and the features of the processed optical coherence tomography OCT image are fused; multi-modal fusion image features are obtained; according to the multi-modal fusion image features, a deep learning model is adopted to reconstruct an eye fundus image; and constructing an image enhancement model to obtain an enhanced fundus image.

The invention discloses a retina thickness prediction method and system based on a multi-modal image, and the method and system achieve the effective estimation of the retina thickness under a low-cost condition through feature alignment and fusion modeling, and improve the basic screening and follow-up visit capability. According to the invention, through fusion of the multi-mode retina image data, the structure and function information of the optic nerve can be more comprehensively obtained, and the prediction accuracy of the thickness of the retina nerve fiber layer (RNFL) is improved. The OCT high-resolution hierarchical structure and the wide-view texture features of the eye fundus image are combined, so that anatomy and pathological states of optic nerves can be truly restored. The method can be used as an auxiliary method for early screening and early warning of optic neurodegenerative diseases such as glaucoma, and provides support for low-cost and high-efficiency primary screening and clinical auxiliary decision making.

The invention relates to the technical field of intelligent medical treatment, and discloses a cataract surgery navigation control method based on multi-modal fusion, which comprises the following steps: carrying out lightweight preprocessing on an eye fundus image, ultrasonic probe data, a corneal topographic map and intraocular pressure parameters through an edge computing architecture, and constructing an efficient feature extraction network by adopting a knowledge distillation technology; self-adaptive feature fusion is realized based on an uncertainty quantification mechanism, real-time structure recognition is provided by utilizing an augmented reality technology, an intelligent navigation strategy is generated by pre-training a reinforcement learning model, an instrument control system with multilayer safety guarantee is established, and continuous optimization of the system is realized by adopting an online learning mechanism. According to the invention, the precision and safety of the operation are effectively improved, and an effective technical scheme is provided for intelligent navigation control of the cataract operation.

A color checker device is disclosed. The color checker device includes a substrate configured to be inserted into a model eye and to conform to a surface curvature of the model eye when inserted therein. A color test target is located on the substrate. The color test target comprises a plurality of color sections configured for generating a color characterization profile of a fundus imaging camera configured to capture a color image of a human retina. A method of generating a color characterization profile using the color checker device and a fundus imaging system having the color characterization profile stored thereon are also disclosed.

The disclosure describes a reading system and a reading method for fundus images, the reading system comprising: an input module for receiving fundus images; a screening module for outputting screening results based on the fundus images, the screening results at least including quality control judgment results and lesion judgment results; a first classification module for classifying the fundus images into screening qualified images and first images to be quality controlled based on the quality control judgment results, and taking at least one image from the first images to be quality controlled and the screening qualified images as an image to be quality controlled; and a quality control module for outputting quality control results based on the image to be quality controlled. According to the disclosure, the screening accuracy of the reading system can be improved.

The invention provides an image lesion attribute prediction model training method, a prediction method and a related device, and the method comprises the steps: carrying out the visual coding of a fundus image sample through a visual encoder, and obtaining a pseudo-image visual feature; performing text coding on each lesion attribute sub-category description text to obtain lesion attribute sub-category features; performing matching according to the visual features of the pseudo image and the lesion attribute sub-category features to obtain a pseudo matching probability; performing visual coding on the fundus image sample through the initial image lesion attribute prediction model to obtain initial image visual features; performing matching according to the initial image visual features and the lesion attribute sub-category features to obtain a predicted matching probability; carrying out loss calculation according to the pseudo matching probability and the predicted matching probability to obtain a target matching loss function; and updating the initial model based on the target matching loss function to obtain an image lesion attribute prediction model. In conclusion, the lesion analysis accuracy of the fundus image can be improved.

The invention provides a fundus image enhancement method based on adaptive fuzzy evaluation and frequency domain enhancement, and the method comprises the following steps: S1, obtaining cataract fundus image samples of different fuzzy levels, and constructing a training data set; s2, constructing an image adaptive enhancement model based on fuzzy evaluation, wherein the image adaptive enhancement model comprises a fuzzy evaluation module, a consistency keeping module and a frequency adaptive enhancement module; and S3, performing model training on the constructed image adaptive enhancement model by using the training data set to obtain a trained image adaptive enhancement model which is used for adaptive enhancement of the cataract eye fundus image. According to the method, the structural definition and color fidelity of the cataract blurred fundus image can be effectively improved, and the diagnosis availability and clinical value of the image are improved.

The invention discloses a multi-mode cardiovascular disease detection method based on comprehensive view analysis, and relates to the problems that in the automatic diagnosis technology of cardiovascular diseases (CVD), CVD progress analysis is difficult, and diagnosis and prevention are difficult to achieve as early as possible. Although the current CVD automatic diagnosis technology based on deep learning achieves a certain progress, most of the technologies are limited to analysis of single-mode data, and the progress of CVD cannot be comprehensively predicted. In order to solve the problem, the invention provides a multi-mode cardiovascular disease detection method based on comprehensive view analysis. Experiments show that the method uses the blood vessel segmentation image to assist the retina fundus image to perform cross-view learning, and enhances the feature perception of the model on the retina blood vessel; two multi-modal data of a retina fundus image and a non-invasive clinical index are used as input, so that the deep understanding of the model on a CVD event is enhanced; a multi-order belief interactive fusion method is used for carrying out fusion and complementation on multi-modal features, and the accuracy and reliability of detection are improved. The method is applied to CVD detection.

The invention discloses a premature infant retinopathy recognition system based on multi-modal data, and relates to the technical field of computers, artificial intelligence and image processing, and the system comprises a data set construction module which obtains an eye fundus image, carries out the preprocessing of the eye fundus image, and constructs a data set based on the eye fundus image obtained through the preprocessing and a structured prompt project; the primary training module is used for inputting the data set into a preset LLaVA-v1.5 model and training the LLaVA-v1.5 model based on a LoRA mechanism to obtain a primary training model; the secondary training module is used for constructing a knowledge enhanced thinking chain and training the primary training model based on the knowledge enhanced thinking chain to obtain a recognition model; and the recognition module inputs the fundus image to be recognized into the recognition model to obtain a recognition result. According to the novel intelligent diagnosis system, retina image analysis and medical knowledge reasoning are integrated, and high-precision and interpretable ROP screening is achieved.

The invention relates to the technical field of eye fundus image processing, and particularly discloses an eye fundus image multi-label classification model and method, and the method comprises the steps: extracting the features of eye fundus images from RGB and gray eye fundus images through two branch structures based on TransNeXt; the multi-scale space-channel attention mechanisms are respectively embedded in the two branch structures, modeling is carried out after each feature extraction stage of each branch structure, and multi-scale modeling is carried out on a feature map of each stage; and the feature interaction module is used for realizing communication between the two branches, integrates a large-selectivity module to detect local details and global context information of modeling, performs feature extraction through standard convolution and expansion convolution by using a double-path architecture, and performs splicing, dimension reduction, aggregation and compression processes in sequence after feature extraction, so as to obtain the communication between the two branches. And a final output result is generated, so that the problems confronted by fundus image multi-label classification at present are solved.

The invention discloses a method and system for constructing a fundus anomaly recognition model and a medium, and the method comprises the steps: collecting a normal fundus color image, constructing a training set and a verification set according to the normal fundus color image, and training an AutoEncoder model, thereby obtaining a basic pre-training model with the capability of reconstructing the normal fundus image; constructing a Unet model, freezing an encoder in the basic pre-training model, and fusing the Unet model with the frozen basic pre-training model to obtain a fused model; generating a mask matrix by using Berlin noise, and carrying out pixel replacement on the plurality of normal fundus color images based on the mask matrix to obtain a plurality of abnormal color images with abnormal areas; and performing data training on the fusion model by using the abnormal color photo image and the normal fundus color photo image to obtain a fundus anomaly recognition model. The fundus anomaly recognition model constructed by the invention can effectively segment and recognize the abnormal region in the fundus image, thereby greatly reducing the workload of doctors and improving the diagnosis efficiency and accuracy.

The invention provides a multi-view eye fundus image splicing method suitable for multiple modalities, and the method comprises the steps: obtaining a plurality of to-be-spliced images of different modalities, carrying out the corresponding preprocessing according to the imaging characteristics of the modalities to which the images belong, and obtaining a preprocessed image group; selecting one of the preprocessed image groups as a reference image, and performing image registration on the other images and the reference image to obtain corresponding registration image groups; on the basis of a preset registration precision threshold value and the proportion of overlapping areas between the images, screening out the images meeting the requirements from the registration image group; and performing multi-scale fusion processing according to the spatial relationship between the screened images to obtain a spliced eye fundus image. According to the multi-view eye fundus image splicing method provided by the invention, through preprocessing, registration, screening and multi-scale fusion, images of different modes can be efficiently integrated, the splicing precision and the image quality are improved, and the method is suitable for accurate eye fundus analysis and medical diagnosis.

The invention discloses an eye disease diagnosis system and method based on a lesion space structure relation, and belongs to the technical field of intelligent eye disease diagnos.The eye disease diagnosis system comprises a lesion significance enhancement module and a lesion visual distribution information fusion module; the method comprises the following steps of: firstly, obtaining the characteristics of an eye lesion pixel, further refining the saliency characteristics to obtain the characteristics of the eye lesion pixel, constructing an eye lesion graph, and finally obtaining a global distribution mode of a lesion area through pooling operation to complete the diagnosis of the eye disease. According to the eye disease diagnosis system and method provided by the invention, effective information can be provided for identification and diagnosis of different types of eye diseases, an accurate retinal disease diagnosis result can be obtained, dispersed pathological features in the fundus image and a global distribution mode of the dispersed pathological features can be comprehensively understood, the calculation speed of a network is accelerated, and the calculation cost is reduced.

The invention discloses a continuous vision-language comparison pre-training method for eye fundus image analysis, which adopts a representative image-text pair replay strategy to enable a model to review knowledge of a previous stage in a continuous learning process, and combines a non-diagonal information distillation technology to effectively maintain the consistency of image and text representation so as to improve the learning efficiency. Therefore, the problem of disastrous forgetting is relieved. Compared with a traditional single-mode or static pre-training method, the method can achieve the incremental fusion of the representation features of different modes, and forms a unified multi-mode basic model. In a dynamic data environment, the method does not need to obtain data of all modalities at the beginning of training, and can effectively adapt to continuously arriving image text pairs of different modalities through a continuous pre-training mode. Experimental results show that the performance of the method is superior to that of an existing method in downstream tests of fundus images of various modes, the forgetting rate can be effectively reduced in the continuous learning process, and the plasticity and generalization of the model are improved.

The invention discloses a focus recognition system and method for a diabetic retinopathy image, and relates to the technical field of medical image processing, and the system comprises a fundus image collection module, an image feature processing module, a focus feature extraction module, a network model training module and a focus model recognition module. By integrating the fundus image collection module, the image feature processing module, the lesion feature extraction module, the network model training module and the lesion model recognition module, full-process automation from original fundus image acquisition to lesion intelligent recognition is realized, and the efficiency of diabetic retinopathy lesion recognition is remarkably improved; in addition, errors possibly caused by manual intervention are reduced, the accuracy and consistency of recognition results are ensured, specifically, the fundus image collection module can automatically obtain images from ophthalmology equipment or a hospital image storage system, format unification and index construction are carried out, and great convenience is provided for follow-up processing.

The invention relates to the technical field of computer vision, in particular to a multi-modal intelligent glaucoma recognition method and system based on transfer learning, and the method comprises the following steps: collecting fundus images of a glaucoma patient in multiple stages based on medical image data, analyzing the texture and shape changes in the images, and obtaining a multi-modal image; and simulating changes of the pathological features in a plurality of development stages, and generating a dynamic pathological feature sequence. According to the method, retina changes of a glaucoma patient in multiple disease development stages are simulated, visual feature information of pathological features in multiple development stages is captured, visual features of fundus images of the patient are extracted and compared, the disease development stages of the patient are evaluated, and the feature recognition process is optimized in combination with transfer learning. The method improves the generalization ability of the model and the diagnosis accuracy, achieves the accurate evaluation of the visual impairment through combining the measurement and prediction of the intraocular pressure data, and helps medical personnel to judge the retina state of a glaucoma patient.

The invention relates to the technical field of fundus image quality evaluation, and particularly discloses a fundus image quality evaluation network and evaluation method combining global and local features, and the network comprises a feature extraction module which comprises a global feature extraction branch and a local feature extraction branch which are independently operated, applying an RGB image enhanced by CLAHE to a G channel, and inputting a local feature extraction branch; the feature fusion module is used for fusing the global features and the local features output by the feature extraction module by adopting a cross attention mechanism and a plurality of full connection layers, and respectively providing independent supervision signals for a global feature extraction branch and a local feature extraction branch through a three-way weighting loss strategy; according to the method, key features in the fundus image are effectively highlighted, deep fusion of global and local features is realized, richer feature interaction information is captured, and the method has excellent performance in quality evaluation of the fundus image.