132 results about "Image fusion" patented technology

The invention relates to the field of image analysis, in particular to a temporal bone disease classification method and system based on a multi-modal medical image fusion technology. The method comprises the following steps: acquiring a multi-modal image of a patient, performing adaptive distortion correction, and generating a standardized image set; performing layer-by-layer anatomical structure semantic segmentation and multi-modal image fusion on the standardized image set to construct an image fusion framework; according to the image fusion framework, performing intelligent recognition on the fine structure of the temporal bone, and constructing a personalized temporal bone anatomical structure chart; performing tissue function state analysis and digital pathology dynamic simulation based on the personalized temporal bone anatomical structure chart, and constructing a digital pathology model; and performing intelligent pathological feature classification based on the digital pathological model to obtain an intelligent classification report. According to the method, rapid, efficient and accurate temporal bone disease classification is realized.

The invention relates to the technical field of image analysis, in particular to a temporal bone and gallbladder sebaceous tumor detection method and system based on multi-modal medical image fusion. The method comprises the following steps: acquiring a multi-modal temporal bone and gallbladder sebaceous tumor image of a patient, carrying out multi-modal image time sequence space alignment, carrying out feature vector change rate fitting, and constructing a multi-time-point image feature vector map; performing multi-modal image voxel decomposition on the multi-time-point image feature vector map, and performing three-dimensional morphological analysis to generate a geometric morphological parameter set; carrying out potential pathological hierarchical structure mining based on the geometric morphology parameter set, carrying out semantic label labeling, and constructing a multi-modal lesion semantic labeling image; and carrying out multi-time-point image change analysis on the multi-modal lesion semantic marking image, and carrying out parallel lesion evolution state prediction so as to generate a cholangioma evolution state prediction map. According to the method, accurate positioning, effective segmentation and visualized display of the temporal bone and gallbladder sebaceous tumor focus are realized.

The invention relates to the field of new energy, and discloses a photovoltaic system efficiency evaluation method and system based on unmanned aerial vehicle multi-source image fusion, and the method comprises the steps: collecting a visible light orthoimage, thermal imaging data and environment parameters of a photovoltaic system, and obtaining a registration result through employing an improved feature point matching algorithm; based on a registration result, combining spatial features of a visible light image and temperature features of thermal imaging, realizing accurate segmentation of the photovoltaic panel through a deep learning model, and identifying the type, the arrangement mode and the installation angle of the photovoltaic panel at the same time; establishing a mathematical model of the relation between the photovoltaic panel temperature distribution and the power generation efficiency, distinguishing the normal working temperature difference and the fault hot spot, and analyzing and determining the efficiency attenuation degree and the fault type of the photovoltaic system through a heat distribution mode. The method is accurate in image registration, good in data fusion effect, accurate in temperature anomaly detection and comprehensive in efficiency evaluation, and provides more efficient and accurate technical support for management and maintenance of the distributed photovoltaic system.

The invention relates to a multi-modal image matching method and system based on learning features and epipolar geometric constraints. The method comprises the following steps: carrying out edge enhancement processing on an input image through wavelet transform; extracting a multi-scale dense feature map based on the transformed convolutional neural network, and generating a feature descriptor with rotation and scale invariance in combination with principal direction normalization; adopting an FLANN algorithm and dynamic distance constraint to realize preliminary feature matching; and introducing a basic matrix construction and epipolar geometric consistency verification mechanism, and eliminating mismatching point pairs in combination with an RANSAC affine constraint model. According to the method, image enhancement, deep learning and geometric verification strategies are fused, the problems of radiation nonlinearity and geometric distortion caused by imaging mechanism differences among multi-modal images are effectively solved, the matching precision and robustness are improved, and the method is suitable for remote sensing application scenes such as optical-SAR registration, multi-source image fusion and earth surface change detection.

The invention relates to the field of medical image processing and pathological diagnosis, aims to solve the problems that brain glioma multi-modal image fusion precision is low, heterogeneity feature and pathological association is weak, and visualization readability is poor, and provides a visualization method and device. The method comprises the steps of multi-modal image preprocessing and registration, regional adaptive multi-modal fusion, pathology association type heterogeneity feature quantification, multi-dimensional pathology visualization, and result verification and optimization. Images are rigidly registered and aligned through mutual information, features are enhanced by adopting a regional differentiation attention fusion algorithm, features are quantified through an improved LSSVM, pathological gold standards are associated, and a three-dimensional visualization model is constructed to visually present heterogeneity information. The device comprises a memory and a processor and can execute the method. According to the method, the heterogeneity evaluation accuracy and clinical readability are improved, treatment scheme optimization is assisted, and the method is suitable for accurate diagnosis and treatment scenes of brain glioma.

The invention belongs to the technical field of remote sensing, and discloses a wolfberry planting distribution high-precision remote sensing monitoring method based on multi-temporal fusion and spectral feature optimization. According to the method, a multi-temporal and multi-spectral satellite image is used as a data source, and preprocessing and multi-temporal image fusion are firstly carried out; the method comprises the following core steps: establishing a time sequence characteristic curve according to a unique phenological period (such as bare soil characteristics in a dormancy period and high vegetation coverage in a rapid growth period) of wolfberry; in a spectral domain, screening out a characteristic spectrum dimension combination with the highest discrimination degree between the wolfberry and other crops through a characteristic wave band optimization algorithm (such as vegetation index difference degree and red edge characteristics); and in combination with an object-oriented classification or deep learning classification model, constructing a space-time coupling classifier, and performing high-precision extraction and distribution mapping on the Chinese wolfberry planting region. The method can effectively solve the problem of confusion classification of Chinese wolfberry and similar ground features (such as other shrubs and orchards), and realizes rapid and accurate monitoring of Chinese wolfberry planting area and spatial distribution.

The invention discloses a construction safety management method based on excavator visual angle image fusion, and belongs to the technical field of construction early warning, and the method specifically comprises the steps: obtaining the position and working condition of excavation equipment, and setting a dynamic alarm fence; triggering an equipment vision device to scan a personnel target in real time, identify a personnel bounding box and extract a personnel identity identification code; synchronously calling original coordinates of the corresponding ultra-wideband tag, capturing a posture visual frame, and generating a posture feature vector through skeleton joint point topology analysis; inputting the attitude vector into an attitude-error compensation model, and outputting a positioning offset predicted value to correct an original coordinate; according to the method, real-time compensation is carried out through dynamic accurate matching of the construction dangerous area and personnel positioning errors, misjudgment is avoided to the greatest extent, the safety risk active pre-judgment and alarm accuracy is improved, and the safety risk active pre-judgment and alarm accuracy is improved. And intelligent safety management of a construction site is realized.

The invention provides a vehicle-mounted instrument driving record panoramic image fusion three-dimensional projection method and system, and relates to the technical field of vehicle-mounted display, and the method comprises the steps: obtaining original panoramic image data collected by cameras around a vehicle and real-time driving state information of the vehicle; distortion correction and multi-view splicing processing are carried out on the original image to generate a spliced panoramic image; determining the type of a driving scene according to the real-time driving state information, setting corresponding three-dimensional viewpoint parameters including viewpoint height, viewpoint distance and pitch angle, performing three-dimensional space coordinate transformation on the spliced panoramic image based on the parameters, and generating a three-dimensional aerial view image adaptive to the current driving scene; an obstacle target is extracted, the boundary of an image area of the obstacle target is recognized, height stretching rendering is carried out at the corresponding position of the three-dimensional aerial view image, and a three-dimensional panoramic projection image containing an obstacle three-dimensional identifier is generated; and finally adaptively outputting to a vehicle-mounted instrument panel display screen for real-time display. The visual angle can be dynamically adjusted according to different driving scenes, the stereoscopic perception of the obstacle is enhanced, and the driving safety is improved.

The invention relates to the technical field of medical image diagnosis, and discloses a tooth and fracture line recognition treatment method based on the combination of an AI technology and a CBCT image, and the method comprises the steps: obtaining and preprocessing the CBCT image, and carrying out the multi-scale analysis and recognition of a tooth structure, a microcrack and a fracture line through a first AI model. And the second AI model combines the identification result and the patient characteristics, and generates a personalized treatment scheme through multi-objective optimization. Clinical feedback is used for continuously iteratively optimizing double models, and the diagnosis and treatment precision and effect are improved. The system comprises an image data acquisition unit, an image data preprocessing unit, a tooth and fracture line identification unit, a personalized treatment scheme generation unit and a feedback and optimization unit. Through AI and CBCT image fusion, accurate identification of teeth and fracture lines is realized, a personalized treatment scheme is recommended in combination with individual features of a patient and a multi-objective optimization algorithm, rapid response is realized, a closed-loop feedback mechanism continuous optimization model is established, and diagnosis and treatment precision, efficiency and individualization level are remarkably improved.

The invention discloses a focus intelligent detection and diagnosis system based on multi-modal medical image fusion, and relates to the technical field of medical image processing. A molybdenum target image, an ultrasonic image and a magnetic resonance image of the mammary gland of the patient are acquired through the data acquisition module; a data registration module is used for carrying out partition affine and local refinement registration on the multi-modal image by taking the molybdenum target image as a geometric anchor point and combining parameters such as compression force; extracting registered fusion feature data through a data fusion module; generating a candidate focus set by a focus detection module; finally, the risk assessment module calculates a comprehensive risk score through weighting, correction and consistency verification processes based on a plurality of interpretable feature indexes. And the output module generates a visual diagnosis result. According to the invention, intelligent detection and risk assessment of rechecking of breast lesions are realized, and the diagnosis accuracy and clinical reliability are effectively improved.

The invention belongs to the field of medical image processing, and relates to a three-dimensional reconstruction method and system based on multi-modal medical image fusion, and the method comprises the steps: obtaining an original multi-modal image sequence; the original multi-modal image sequence comprises a CT image and an MRI image; performing spatial alignment on the original multi-modal image sequence to obtain a spatial alignment image pair; the space alignment image pair comprises an aligned CT image and a corresponding aligned MRI image after alignment; performing multi-scale layered cognitive feature extraction on the spatial alignment image pair to obtain a corrected feature map; performing time-space frequency domain collaborative attention fusion on the corrected characteristic spectrum to obtain fusion characteristics; performing topological constraint three-dimensional diffusion reconstruction on the fusion features to obtain a medical image three-dimensional model; the three-dimensional reconstruction precision of multi-modal fusion is improved, and more reliable three-dimensional model support is provided for clinical diagnosis and treatment.

The invention discloses a multi-modal image fusion assisted precise tumor needle biopsy guide system, and aims to solve the problem that the traditional needle biopsy guide technology is insufficient in precision. The system innovatively adopts a multi-scale feature fusion algorithm and an adaptive weight fusion strategy, deeply fuses ultrasonic, CT, MRI and other multi-modal images, and comprehensively improves the information amount of the images. Through real-time force feedback and path adjustment, in combination with VR / AR guide interaction, intelligent puncture guide is realized, and safe and accurate operation is guaranteed. Tumor features are extracted by using a deep learning algorithm, accurate tumor recognition and dynamic image analysis are realized, and the tumor growth trend is predicted. According to the system, the precision and success rate of needle biopsy are remarkably improved, the pain of a patient and the medical cost are reduced, powerful support is provided for personalized precise treatment of tumors, and the system has extremely high clinical application value.

The invention discloses an AI foot shoe tree model construction system based on image reconstruction and parameterization, which comprises a multi-modal medical image fusion subsystem, a dynamic mechanical parameter adaptive subsystem, an AI feature enhancement subsystem, an intelligent parameterization driving subsystem and a real-time comfort simulation subsystem, the subsystems cooperatively work through data streams and interaction relations, and high-precision feature extraction of the extreme foot shape, individual specificity comfort simulation and rapid construction of a shoe tree model are achieved. By means of the multi-mode medical image fusion technology, low-dose CT images and optical scanning data are organically combined, the three-dimensional foot model containing skeleton and soft tissue layering is constructed, the problem that the robustness of extreme foot shape feature extraction is insufficient is solved, and the model construction precision is improved.

The invention relates to the technical field of photogrammetry, and discloses a photogrammetry space reconstruction method based on multi-source image fusion, which comprises the following steps: extracting an interior orientation element and an initial exterior orientation element, and establishing an image beam intersection constraint intensity model in an object space to calculate a constraint weight; monitoring a weight space change rate to identify a heterogeneous image connection area and extracting a high-weight image bundle as a geometric anchor point; a differential regularization item is applied to geometric anchor points to construct a stable geometric skeleton, a weight damping factor is utilized to restrain a low-weight image beam to execute conformal mapping correction, the image beam constraint strength is represented, the condition number of an adjustment method equation matrix is improved, numerical oscillation caused by difference of different-source sensors is restrained, geometric step of a fusion interface is eliminated, and the conformal mapping correction accuracy is improved. And geometric closing and seamless stitching of a multi-source image under a unified framework are realized.

The invention relates to the technical field of orthopedic surgery, and discloses an orthopedic surgery real-time navigation system based on multi-modal image fusion and artificial intelligence. The multi-modal image fusion module is used for generating a three-dimensional skeleton and soft tissue fusion model for surgical navigation; the artificial intelligence analysis module is used for determining a fracture line, a screw implantation angle and a cutting track; the intraoperative tracking module is used for generating a real-time correction instruction; the verification updating module is used for carrying out rationality verification on the real-time correction instruction; if the inspection is not passed, an alarm mechanism is triggered, instantaneous motion state parameters of the surgical instrument are collected, the instantaneous motion state parameters are compared with the cutting track, and the correction instruction is updated according to the comparison result; and the execution module is used for driving the surgical instrument to execute the surgical operation according to the updating correction instruction. According to the invention, manual operation errors can be effectively reduced.

The invention provides a multi-modal image fusion system and method for neurosurgery, and the method comprises the steps: carrying out the rigid transformation of a multi-modal magnetic resonance image and an ultrasonic image, and obtaining a preoperative magnetic resonance image and an intraoperative ultrasonic image after preliminary alignment; inputting the preoperative magnetic resonance image and the intraoperative ultrasonic image into a pre-trained lightweight deformation registration network for deformation modeling to obtain a voxel-level dense deformation field, and determining a confidence map of deformation field region registration based on the dense deformation field; according to the confidence map, multi-scale feature fusion based on confidence weighting is carried out on the deformed preoperative multi-modal labeling information and the corresponding features of the intra-operative ultrasonic image, and a three-dimensional anatomical structure chart for real-time navigation in the neurosurgery operation is obtained. Based on the scheme, brain tissue drift can be dynamically compensated in real time in a neurosurgery operation, and multi-modal image fusion is guided based on a compensation result.

The invention discloses a nasopharyngeal carcinoma necrosis prediction method and device based on feature fusion and a medium, and the method comprises the steps: obtaining multi-modal medical data of a target patient at a plurality of preset time points, and extracting medical features in different modals for each time point, so as to form a medical time sequence feature matrix in different modals; inputting the medical time sequence feature matrixes in the different modes into a gated twin-tower Transform network, and performing feature fusion on the medical time sequence feature matrixes in the different modes through a preset inter-channel-time step cross attention mechanism to obtain key image fusion features and nasopharyngeal carcinoma necrosis risk prediction results; and if the nasopharyngeal carcinoma necrosis risk prediction result is greater than a preset risk threshold, inputting the key image fusion features and the original three-dimensional image data into a 3D nnU-Net network to generate a nasopharyngeal carcinoma necrosis prediction map of the target patient. The prediction accuracy of nasopharyngeal carcinoma necrosis can be improved.

The invention discloses an AI auxiliary radiotherapy dose intelligent optimization device, which comprises a multi-modal image fusion module, a dose distribution prediction module, a dynamic adaptive optimization module and a prognosis model integration module, and is characterized in that the multi-modal image fusion module is used for aligning anatomical features of different images, eliminating respiratory motion artifacts and predicting the dose distribution of the different images; the dose distribution prediction module is used for rapidly predicting radiotherapy dose distribution based on anatomical features and historical data, the dynamic adaptive optimization module is used for monitoring anatomical changes in real time and dynamically adjusting a dose plan, and the prognosis model integration module is used for quantifying correlation between dose distribution and radioactive injury risks. High-precision alignment and respiratory motion artifact elimination of an anatomical structure are achieved through the multi-modal image fusion module, three-dimensional dose calculation is rapidly and accurately conducted through the dose distribution prediction module, organ displacement and deformation are effectively coped with through the dynamic self-adaptive optimization module through a real-time image monitoring and reinforcement learning algorithm, and the accuracy of the three-dimensional dose calculation is improved. And the prognosis model integration module constructs an individualized risk prediction model.

The invention provides a sea target identification method and device based on AIS and SAR image fusion, and the method comprises the steps: obtaining the AIS data of a sea target through an AIS system, obtaining the SAR image of the sea target through an SAR satellite, and carrying out the preprocessing of the data; after time calibration and space calibration are carried out on the preprocessed data, the fusion weight of the AIS data and the SAR image in Kalman filtering is determined according to the confidence score of the AIS data and the SAR image; determining a dynamic measurement noise covariance matrix according to the fusion weight, calculating a Kalman gain according to the dynamic measurement noise covariance matrix and an error covariance matrix, and updating the state of the marine target and the error covariance matrix by using the Kalman gain; and inputting the SAR image and the final state output by the Kalman filtering into the deep learning model to obtain a sea target recognition result output by the deep learning model. According to the invention, the precision, robustness and real-time performance of sea target identification are improved.

The invention discloses a multi-modal information collaborative breast cancer auxiliary identification method. The method comprises the following steps: S1, acquiring a breast ultrasonic image, a breast X-ray image and clinical text data; s2, carrying out pretreatment operation; s3, segmenting the breast ultrasonic image to obtain a breast ultrasonic segmentation mask graph; s4, performing feature extraction operation on the breast ultrasonic image and the breast ultrasonic segmentation mask graph; s5, acquiring enhanced ultrasonic joint features according to feature extraction results of the breast ultrasonic image and the breast ultrasonic segmentation mask graph; s6, performing feature extraction on the mammary gland X-ray image to obtain X-ray image features; the image fusion feature is fused with the enhanced ultrasonic joint feature to obtain an image fusion feature; s7, performing feature extraction on the clinical text data to obtain clinical text features; the multi-modal features are fused with the image fusion features to obtain multi-modal features; and S8, inputting the multi-modal features into an MLP network model for cancer identification.

The invention belongs to the field of laparoscopes, and particularly relates to an integrated multi-modal image fusion laparoscope system which comprises a laparoscope body, a multi-modal image acquisition module, an image fusion processing module, a display control module and an energy supply module. The laparoscope body is integrally designed, and a plurality of image acquisition probes are arranged at the front end of the laparoscope body and used for acquiring image information in different modes; corresponding optical conduction and signal transmission lines are integrated in the lens body, so that acquired image information can be stably and efficiently transmitted to a rear-end module; the multi-mode image acquisition module comprises a white light imaging unit, a fluorescence imaging unit, an ultrasonic imaging unit and a photoacoustic imaging unit which are respectively used for acquiring image data of different modes. The system is reasonable in design, provides comprehensive operation area information for doctors by integrating various image modalities and realizing efficient fusion, improves the operation precision and operation efficiency, and is high in expandability.

The invention discloses an orthopedic surgery navigation positioning method and system based on three-dimensional registration, and belongs to the technical field of orthopedic surgery navigation. According to the method, through preoperative multi-modal image fusion reconstruction, intraoperative dynamic point cloud collection and an improved mixed registration algorithm, precise alignment of a preoperative model and an intraoperative actual skeleton is achieved, and by combining real-time error monitoring and dynamic compensation, precise positioning parameters of a surgical instrument are output to guide surgical operation; the system corresponds to the method and comprises an image acquisition module, a three-dimensional reconstruction module, a registration calculation module, a navigation guiding module, an error monitoring module and a control module, all the modules work cooperatively, and it is ensured that the navigation positioning precision is smaller than or equal to 0.3 mm, the registration time is smaller than or equal to 3.5 s, and the dynamic error compensation response time in the operation is smaller than or equal to 50 ms. The method does not need to depend on a physical marker, does not need a doctor to manually intervene the registration process, is high in anti-interference capability, is suitable for various orthopedic surgery scenes, effectively reduces the surgical risk, and improves the surgical efficiency and the standardization level.

The invention relates to the technical field of radiotherapy, and discloses a multi-modal image fusion guided chest tumor adaptive radiotherapy system. The system comprises an asynchronous bimodal sensing module, a spectrum preprocessing module, a parameter inversion module, a boundary construction module and a biomechanical calculation module. According to the method, the feedback voxel-level velocity field is utilized to execute Lagrange reverse warping on the high-energy projection with physical time delay, and motion artifacts are eliminated; and dynamically updating the Young modulus of the lung tissue mechanical model based on the inverted real-time electron density, and solving a full-field three-dimensional displacement field in combination with double boundary conditions of body surface flexibility and internal skeleton rigidity so as to reconstruct a tumor target region and generate a beam control instruction. According to the method, through closed-loop interaction of image physics and biomechanics, the problems of dual-energy imaging motion artifact interference and insufficient lung tissue fixed parameter modeling precision are solved, and accurate self-adaptive tracking of a tumor target region is realized.

The invention relates to the field of agriculture, and discloses a method for evaluating sugarcane smut based on an unmanned aerial vehicle image, which comprises the following steps: data acquisition: planning a route by using an unmanned aerial vehicle carrying a multispectral camera, a hyperspectral camera and an RGB (Red, Green and Blue) camera, so that the unmanned aerial vehicle acquires image data of sugarcane; preprocessing: sequentially carrying out geometric correction, radiation correction and image fusion processing on the image data to obtain preprocessed data; feature fusion: constructing a fusion model including spectral feature extraction, visual feature extraction and spatial-temporal feature fusion according to the feature expression of the sugarcane smut; and model generalization: collecting multi-source images and disease investigation data according to the actual condition of the sugarcane. The method comprises the following steps: performing data acquisition through an unmanned aerial vehicle, obtaining multi-dimensional image data in a sugarcane growth critical period, performing preprocessing, constructing a fusion model including spectral feature extraction, visual feature extraction and spatial-temporal feature fusion, performing fine adjustment and optimization in combination with a transfer learning strategy, and performing real-time analysis on a disease detection result.

The invention relates to a virtual reality and reality image fusion navigation method and system for temporal bone surgery, and the method comprises the steps: obtaining medical image data of a temporal bone region, and carrying out the three-dimensional reconstruction, thereby obtaining a temporal bone dynamic virtual model; optical tracking data and instrument space pose data are obtained, dynamic registration mapping is conducted on the temporal bone dynamic virtual model, and a real-time space fusion relation is obtained; performing navigation view construction on the instrument space pose data and the real-time space fusion relationship to obtain an augmented reality navigation view; performing structural distance analysis on the augmented reality navigation view to obtain dynamic early warning parameters; and performing display regulation and control according to the augmented reality navigation view and the dynamic early warning parameters to obtain a real-time navigation strategy. The problem of traditional static image registration lag can be effectively solved, and it is ensured that the virtual model and the actual anatomical structure are always kept in high-precision matching.

The invention provides a multi-modal image fusion medical adhesive applicability evaluation system, and the system comprises a multi-modal image collection module which is used for collecting CT, MRI and ultrasonic image data; the wound surface geometric feature extraction module is used for extracting geometric features of the wound surface from the image data; the hemodynamic parameter extraction module is used for extracting hemodynamic parameters from the image data; the deep learning feature fusion module is used for fusing multi-modal features; and the applicable scene classification module is used for evaluating the applicability of the medical adhesive in embolism, spraying and smearing scenes according to the extracted features and outputting recommendation indexes. According to the technical scheme, the intelligent auxiliary decision-making system for automatically evaluating the application scene of the medical adhesive is realized.

The invention discloses a urinary surgery simulation navigation method and device based on image fusion and a medium, and relates to the technical field of surgery simulation. The method comprises the steps that a real-time video stream of a visual angle in a urinary cavity is obtained through an endoscope arranged in a physical urinary model, and miniature liquid absorption cotton is arranged in the endoscope; calling a virtual three-dimensional model corresponding to the physical urinary model, and registering the current image frame of the real-time video stream with the virtual three-dimensional model to establish a mapping relationship between the physical urinary model and the virtual three-dimensional model; in the real-time video stream, the tail end position and posture of the surgical instrument are detected and tracked in real time; based on the mapping relation, the tail end position of the surgical instrument is displayed on the real-time video stream in an overlapping mode, and a fusion navigation view is generated and output. By means of the method, the endoscope real-time video stream of the physical model and the virtual three-dimensional model are registered and fused, and space positioning and instrument control of an operator in a complex cavity are greatly assisted.

The invention belongs to the technical field of artificial intelligence and medical image processing, particularly relates to an intelligent auxiliary diagnosis system for radiographic images, and aims to solve the problems of missed diagnosis and misdiagnosis caused by overload of doctors, large subjective difference and low early focus recognition sensitivity in the prior art. The system integrates a multi-scale feature extraction engine, a cross-modal alignment network, a pathological semantic reasoning unit and a diagnosis consensus generator, and realizes multi-modal image fusion analysis, dynamic knowledge reasoning and man-machine collaborative decision. A thermodynamic diagram and a feature contribution degree map are output through an interpretability visualization module, and a structured report conforming to clinical specifications is generated by a report automation engine, so that the diagnosis accuracy and efficiency are improved, the report period is shortened, and credible deployment of AI auxiliary diagnosis in a complex medical scene is supported.

The invention relates to the technical field of medical image processing, and particularly discloses a dermatosis auxiliary diagnosis method and system based on multi-mode optical image fusion, and the method comprises the steps: carrying out the semantic segmentation of an obtained dermatoscope image of a target skin region, and recognizing at least one target feature region; based on the semantic category and position information of the target feature region, screening out at least one associated feature region from an obtained RCM image set of a target skin region; extracting a first feature vector in the target feature region, and extracting a second feature vector in the associated feature region; and fusing the first feature vector and the second feature vector, and performing disease classification on the fused feature vector based on a trained classification model to obtain a skin disease classification result of the target skin area. According to the method, the corresponding associated region can be screened out from the RCM image set according to the semantic segmentation result of the dermatoscope, and the efficiency and reliability of multi-modal data integration are remarkably improved.

The application provides a verification method for a surgical navigation robot system and a CT verification phantom, the CT verification phantom comprising at least one measuring module and a base, wherein each of the measuring modules is substantially cylindrical, has two substantially parallel measuring end faces, and has a certain height removed on both axial sides, thereby forming two planes on the side faces; the base is substantially plate-shaped, and the measuring modules are fixed to the base through fixing devices; the measuring modules, the base and the fixing devices are all made of high-molecular organic materials. The method and the CT verification phantom have high stability and simple structure, can be used for measuring the precision of image reconstruction, image registration, surgical navigation and tracking, image fusion and the like of the surgical navigation robot system in application, and can also verify the surgical navigation / tracking precision, early warning, surgical path planning and the like of the surgical navigation system.