Multifunctional multi-dimensional visual diagnosis recording system

Abstract

A multifunctional multi-dimensional visual diagnosis recording system integrates a 3D camera, a polarized light camera and an infrared thermal imaging camera, and realizes high-precision acquisition of a human body surface form, a subcutaneous structure and temperature distribution. According to the system, a high-precision human body structure model is constructed, a shooting part is accurately mapped to the model, and three-dimensional visual angle inspection and dynamic updating are supported. And in combination with a medical knowledge base and an algorithm, auxiliary diagnosis and evaluation are provided. The medical history recording module records the medical history of the patient in detail and correlates with the position information of the shooting part. The treatment response tracking function comprehensively manages treatment data, and supports visual display and deep comparison. And the image fusion module integrates the multi-mode image data to form comprehensive image representation to assist doctors in making decisions. The system is suitable for various cases such as skin ulcers and cancerous ulcers, and diagnosis accuracy and treatment efficiency are improved.

Description

Technical Field

[0001] This patent specification relates to the field of medical diagnostic technology, specifically a multifunctional multidimensional visual diagnosis recording system. This system integrates multiple advanced technologies such as a 3D camera acquisition module, a human body structure model mapping module, a polarized light camera acquisition module, an infrared thermal imaging camera acquisition module, and a data fusion module. It aims to provide medical staff with comprehensive and three-dimensional information on the patient's body surface and internal structure through high-precision, multidimensional image acquisition and processing, in order to assist doctors in making more accurate diagnoses and assessments. Background Technology

[0002] In the current field of medical diagnostics, the observation and analysis of human body surface morphology mainly relies on traditional two-dimensional imaging data, such as photographs and X-rays. However, these two-dimensional imaging data have significant limitations in expressing the three-dimensional morphology and subtle structural changes of the human body, making it difficult to meet the needs of high-precision, multi-dimensional diagnosis. Especially when dealing with complex cases, such as skin ulcers and tumors, doctors often need to comprehensively consider information from multiple aspects, including morphology, structure, and temperature distribution, and two-dimensional imaging data often cannot provide sufficiently comprehensive and accurate data support.

[0003] With the continuous development of medical imaging technology, new imaging technologies such as 3D imaging, polarized light imaging, and infrared thermal imaging are gradually being applied in the medical field, providing new perspectives and methods for observing and analyzing the morphology of the human body surface. However, the image data generated by these new imaging technologies are often independent and lack effective integration and fusion mechanisms, which means that doctors need to view and analyze multiple image data separately during the diagnostic process, increasing their workload and difficulty.

[0004] Furthermore, traditional human anatomy models are often built based on average parameters, lacking personalized details and failing to accurately reflect the patient's actual condition. During the diagnostic process, doctors need to manually match and compare imaging data with human anatomy models, which is not only inefficient but also prone to errors.

[0005] Therefore, the market urgently needs a multifunctional, multidimensional visual examination recording system that can integrate various novel imaging technologies, achieve deep fusion of image data, and provide high-precision personalized human anatomy models to assist doctors in making more efficient and accurate diagnoses and assessments. The multifunctional, multidimensional visual examination recording system of this invention is proposed based on this technological need. It aims to integrate multiple imaging technologies such as 3D cameras, polarized light cameras, and infrared thermal imaging cameras to construct high-precision personalized human anatomy models and achieve deep fusion and unified mapping of image data, providing doctors with more comprehensive and accurate diagnostic and assessment information. Summary of the Invention

[0006] A multi-functional, multi-dimensional visual examination and recording system integrates a 3D camera, a polarized light camera, and an infrared thermal imaging camera to achieve high-precision acquisition of human body surface morphology, subcutaneous structure, and temperature distribution. The system constructs a high-precision human structural model, accurately mapping the imaged area to the model, supporting three-dimensional viewing and dynamic updates. Combined with a medical knowledge base and algorithms, it provides auxiliary diagnosis and assessment. The medical history recording module records detailed patient medical history and links it to the location information of the imaged area. The treatment response tracking function comprehensively manages treatment data, supporting intuitive display and in-depth comparison. The image fusion module integrates multi-modal image data to form a comprehensive image representation, assisting doctors in decision-making. The system is applicable to various cases such as skin ulcers and cancerous ulcers, improving diagnostic accuracy and treatment efficiency.

[0007] A multi-functional, multi-dimensional visual examination recording system, comprising:

[0008] The 3D camera acquisition module is specifically designed to perform high-efficiency human body surface morphology, accurately capture subtle morphological changes, and provide detailed basic data for constructing high-precision human body stereoscopic images.

[0009] Human Body Structure Model Mapping Module: This module ensures that the captured human body parts are accurately mapped onto a high-precision human body structure model. Through this module, users can easily and comprehensively examine the specific condition of the captured parts from a three-dimensional perspective, achieving intuitive and three-dimensional observation and analysis.

[0010] Furthermore, a multifunctional multidimensional visual examination recording system is characterized in that the specific implementation method of the human body structure model mapping module includes the following steps:

[0011] a. Model building: First, a high-precision human body structure model is built, which includes detailed anatomical structures and physiological features to provide an accurate basis for subsequent mapping of the shooting parts;

[0012] b. Camera Location Recognition: Using image recognition technology or manual annotation, identify the camera location and match it with the corresponding position in the human body structure model;

[0013] c. Precise mapping: Precisely mapping the location information of the identified shooting parts in the image onto the human body structure model;

[0014] d. Dynamic updates: As filming progresses, the mapping information on the human body structure model is updated in real time to reflect the latest status of the filmed area;

[0015] e. 3D perspective viewing: Provides interactive operations such as rotation, zoom, and pan, allowing users to freely adjust the viewing angle and observation distance to gain a deeper understanding of the details and features of the shooting location;

[0016] f. Assisted diagnosis and assessment: Combining medical knowledge and algorithms, the system automatically analyzes the imaged area and provides diagnostic suggestions or assessment reports to assist doctors in making decisions.

[0017] The auxiliary diagnosis and assessment in the multifunctional multidimensional visual examination recording system mainly involves the following steps and

[0018] Method:

[0019] I. Construction of Medical Knowledge Base

[0020] Knowledge collection and organization:

[0021] Collect authoritative medical information, including textbooks, clinical guidelines, and the latest research papers, to ensure knowledge.

[0022] The authority and timeliness of the database;

[0023] Organizing a knowledge framework: Organizing medical knowledge according to dimensions such as human body systems, disease types, and symptom manifestations.

[0024] To form a structured knowledge base;

[0025] Feature extraction and annotation:

[0026] Disease Feature Extraction: Extracting typical features of various diseases from medical data, such as lesion morphology and color.

[0027] Color, position, etc.;

[0028] Feature annotation: These features are annotated on the human body structure model to form a disease feature library corresponding to the model;

[0029] II. Algorithm Development and Integration

[0030] Image processing and analysis algorithms:

[0031] Image preprocessing: Denoising, enhancement, and other processing are performed on the acquired image data to improve image quality;

[0032] Feature detection and extraction: Utilizing image processing techniques to detect features such as edges, corners, and textures in images.

[0033] The characteristics were compared with those in the medical knowledge base;

[0034] Machine learning and deep learning algorithms:

[0035] Training the model: Based on a large amount of labeled medical image data, train a machine learning or deep learning model.

[0036] This allows it to identify and classify different lesions;

[0037] Model optimization: Improve the accuracy and generalization ability of the model through continuous iteration and algorithm optimization;

[0038] III. Auxiliary Diagnosis and Assessment Process

[0039] Data input and preprocessing:

[0040] Image data input: 3D morphology, polarized light images, and infrared thermal images captured by the visual examination recorder.

[0041] Image data input system;

[0042] Data preprocessing: Preprocessing operations such as denoising, enhancement, and registration are performed on image data to ensure data integrity.

[0043] quality;

[0044] Feature matching and analysis:

[0045] Feature comparison: Comparing features in image data with features in a medical knowledge base to identify similar features.

[0046] Similar disease characteristics;

[0047] Lesion identification: Using trained machine learning or deep learning models to identify lesions in imaging data.

[0048] Variations include type, location, and severity.

[0049] Diagnostic recommendations and assessment report generation:

[0050] Comprehensive analysis: A comprehensive analysis is conducted by combining lesion identification results, patient medical history, clinical manifestations, and other information;

[0051] Diagnostic recommendations: Based on the analysis results, provide possible diagnostic recommendations or disease risk assessments;

[0052] Assessment Report Generation: Automatically generates assessment reports containing lesion descriptions, diagnostic suggestions, and treatment recommendations to assist doctors in making decisions.

[0053] Furthermore, the specific steps of the model construction method in the human body structure model docking function include:

[0054] a. Basic information gathering and initial modeling:

[0055] First, collect basic information about the patient, including height, weight, gender, age, etc. This information helps to provide basic parameters for subsequent modeling.

[0056] Based on this information, a preliminary human anatomy model is constructed. This model can be a general model based on average parameters, and will be adjusted and optimized based on the patient's specific data in the future.

[0057] b. 3D morphological data acquisition and integration:

[0058] The 3D camera in the visual examination recorder is used to record the human body surface morphology in a high-precision and efficient manner. This provides detailed human morphological data, including but not limited to the size, shape, and outline of various body parts;

[0059] By integrating these 3D morphological data with a preliminary human body structure model, and using data fusion techniques such as image registration and calibration, the 3D morphological data is accurately mapped onto the model, resulting in a more accurate and personalized patient human body model.

[0060] Furthermore, the identification of the shooting location in the human body structure model docking function is characterized by the following steps:

[0061] a. Multimedia Acquisition: First, multimedia data is acquired through the camera module of the visual examination recorder, including taking gross photographs, detailed photographs, and videos. The gross photographs and videos should include easily identifiable parts of the human body, such as joints (shoulder, elbow, wrist, hip, knee, ankle), head, face, buttocks, back, perineum, forearm, etc., for quick preliminary positioning on the human body structure model; the detailed photographs and videos are taken at high resolution and high precision for specific shooting areas to capture more detailed features and dynamic changes;

[0062] b. Feature extraction: In the acquired images and videos, representative features are extracted using image processing techniques. These features include, but are not limited to, edges, corners, textures, motion trajectories, and specific anatomical landmarks in the images and videos.

[0063] c. Model matching: The extracted features are matched with the corresponding features in the pre-built or acquired high-precision human body structure model, and the best matching position is determined by similarity measurement methods (such as Euclidean distance, cosine similarity, etc.).

[0064] d. Determining the shooting location: Based on the model matching results, determine the specific location of the shooting location on the human body structure model, and generate the corresponding location information, including coordinates, size, orientation, etc.

[0065] e. Assisted Recognition: To improve recognition accuracy, the system can also combine other auxiliary information, such as the patient's medical history, annotations at the time of shooting, timestamps in the video, or manual confirmation by medical staff. It first quickly locates the general image and video, and then precisely corrects and confirms the shooting location in detailed images and videos. By comprehensively utilizing multimedia data, accurate identification and positioning of the shooting location can be achieved.

[0066] Furthermore, a model matching method for identifying the imaged area in a multifunctional multidimensional visual examination recording system is characterized by comprising the following steps:

[0067] a. Model feature library construction:

[0068] Construct or acquire a high-precision human body structural model, which must include detailed anatomical structures and physiological characteristics;

[0069] In the human anatomy model, detailed annotations are made of feature points, contours, and specific anatomical landmarks for each part, especially detailed anatomical information for the target area, forming a comprehensive and refined model feature library; b. Feature extraction and comparison:

[0070] From the gross and detail photos captured by the visual examination recorder camera module, representative features are extracted using advanced image processing technology, including but not limited to edges, corners, textures, and specific anatomical landmarks.

[0071] The extracted image features are compared one by one with the features in the model feature library to ensure that each feature point or region is accurately matched.

[0072] c. Similarity calculation:

[0073] Similarity measurement methods, such as Euclidean distance and cosine similarity, are applied to calculate the degree of similarity between image features and model features, and a similarity score is obtained.

[0074] A comprehensive analysis of the similarity scores of all compared feature points is performed to determine the overall similarity level;

[0075] d. Determining the optimal matching position:

[0076] Based on the similarity calculation results, the position with the highest similarity score is selected as the best matching position of the shooting part on the human body structure model;

[0077] To verify the accuracy of the matching results, factors such as the spatial distribution of feature points, morphological consistency, and the rationality of anatomical structures are considered to ensure the precision of the matching position.

[0078] e. Auxiliary information fusion and correction:

[0079] The matching results were further verified and corrected by combining the patient's medical history, the annotation information at the time of shooting, and the timestamps in the video.

[0080] If necessary, invite medical personnel to manually confirm the location of the imaged area on the human anatomy model to ensure that the information is accurate.

[0081] Furthermore, the visual examination recorder is characterized by the addition of a medical history recording module, which comprehensively and meticulously records the patient's medical history information and closely correlates this information with the precise location of the imaging site on the human body structure model.

[0082] The medical history recording module includes:

[0083] a. A data structure design unit, configured to design a data structure for storing patient medical history information, wherein the data structure can flexibly contain various types of data such as text, date, numerical value, image and video to adapt to the recording needs of different parts and different medical histories;

[0084] b. User interface design unit, configured to design an intuitive and easy-to-use user interface, which should include a dedicated area for inputting and viewing medical history information from different sites, including input boxes, selection boxes, date pickers for inputting medical history information, and tables or lists for displaying and editing medical history information, while providing functions to facilitate medical staff to quickly select and record relevant medical history information based on the imaging site;

[0085] c. The medical history information entry and storage unit is configured as follows:

[0086] Medical staff are allowed to input detailed medical history information for different parts of the patient through the user interface, including but not limited to symptom descriptions, diagnosis results, treatment process, medication use, etc., and this information is accurately stored in the data structure, forming a one-to-one correspondence with the location information of the imaging site;

[0087] It supports the automatic import of patients' medical history data from electronic medical record systems or other medical information systems, and automatically classifies and integrates the data according to the shooting location, so as to achieve seamless data connection and efficient management;

[0088] Ensure that the entered medical history information is closely linked to the patient's identification and the location information of the imaging site, so that relevant information can be quickly retrieved and viewed according to the location or type of medical history later;

[0089] d. The medical record and location association unit is further configured as follows:

[0090] When recording medical history information, the system automatically associates the identified location information of the imaging site with the corresponding medical history information, enabling medical staff to intuitively compare and analyze the patient's medical history information with the imaging data of specific sites.

[0091] It provides the function of classifying, filtering and viewing medical history information according to the location or type of medical history, so that medical staff can more easily understand the patient's medical history and provide strong support for diagnosis and treatment;

[0092] e. The medical history information should cover the patient's comprehensive medical history, including but not limited to historical symptoms in different parts of the body, disease progression, relevant influencing factors, diagnostic assessment results, and treatment response, to ensure the completeness and accuracy of the record.

[0093] Furthermore, a medical history recording module including treatment response tracking function is characterized by being configured as follows:

[0094] a. A comprehensive unit for recording and managing treatment response data:

[0095] In the data structure of the medical history recording module, a field or sub-table is added specifically for recording treatment response data. The recorded data covers the treatment date, specific treatment plan, symptom improvement after treatment, possible adverse reactions, and related medical imaging data.

[0096] It provides an intuitive and easy-to-use user interface, supporting medical staff to enter treatment response information in various ways, such as text description, selection of preset response types, and recording of key time points;

[0097] It supports the automatic import of treatment response data from electronic medical record systems or other medical information systems, ensuring seamless data connection and efficient integration;

[0098] Ensure that the entered treatment response information is closely linked to the patient's identification and the location information of the imaging site to guarantee the integrity and traceability of the data;

[0099] b. Visual presentation and in-depth comparison of treatment response data:

[0100] The user interface simultaneously displays the patient's treatment response data, imaging data of the imaging site, and other relevant medical information;

[0101] It employs multiple display methods, such as split-screen display, overlay display, and linked display, to facilitate medical staff in comparing and analyzing changes in treatment response and imaging sites;

[0102] It provides a timeline function, enabling medical staff to dynamically view the patient's treatment response records in chronological order of treatment time, thereby clearly observing the changing trend of the treatment response and the dynamic evolution of the imaging data of the imaging site.

[0103] It integrates advanced comparative analysis tools such as image difference analysis and data statistical analysis to automatically calculate and analyze the differences before and after treatment and generate intuitive comparison reports;

[0104] c. Intelligent reminder and report generation unit:

[0105] The system automatically sends treatment response data recording reminders to medical staff based on preset rules or conditions, ensuring timely and accurate recording of treatment response data;

[0106] It provides powerful report generation capabilities, enabling medical staff to quickly generate detailed and accurate treatment response reports based on treatment response data and imaging data of the imaging sites;

[0107] It supports exporting reports to multiple formats such as PDF and Word for easy sharing and discussion.

[0108] Furthermore, a multi-functional, multi-dimensional visual examination recording system also includes a polarized light camera acquisition module, an infrared thermal imaging camera acquisition module, and a data fusion module;

[0109] The polarization camera acquisition module reveals and adjusts the microstructural features under the skin tissue by finely controlling the polarization state of polarized light, thus enhancing the visualization ability of subcutaneous tissue structures.

[0110] The infrared thermal imaging camera acquisition module is designed to capture and analyze the infrared radiation emitted from the human body surface, generate detailed and high-precision temperature distribution images, and then accurately locate areas of local temperature abnormalities or potential lesions.

[0111] The data fusion module is responsible for deeply and accurately integrating the data information collected by the 3D camera, polarized light camera, and infrared thermal imaging camera to construct a three-dimensional image of the human body and record its dynamic changes in real time.

[0112] The data fusion module is characterized in that it is responsible for deeply and accurately integrating the data information collected by the 3D camera, polarized light camera and infrared thermal imaging camera to construct a three-dimensional image of the human body and record its dynamic changes in real time; the data fusion module adopts advanced image registration and calibration technology, as well as a highly integrated multimodal feature fusion system.

[0113] Implementation methods and steps

[0114] Data Acquisition

[0115] 3D morphological data: Using the 3D camera in the visual examination recorder, the target parts of the human body are recorded in a high-precision and high-efficiency three-dimensional manner to capture subtle morphological changes.

[0116] Polarized light imaging: By using a polarized light camera, the polarization state of polarized light can be precisely controlled to obtain the fine structural features under the skin tissue.

[0117] Infrared thermal imaging: Using an infrared thermal imaging camera, the infrared radiation emitted from the human body surface is captured and analyzed to generate detailed and high-precision temperature distribution images.

[0118] b. Image registration and calibration

[0119] Feature extraction: Accurately extract representative feature points or feature regions, such as edges, corners, and textures, from images acquired by 3D cameras, polarized light cameras, and infrared thermal imaging cameras.

[0120] Feature matching: Using similarity measurement methods, we find and confirm matching feature pairs among various types of images to ensure that these feature pairs have a high degree of consistency in spatial location.

[0121] Transformation estimation: Using appropriate transformation models (such as rigid body transformation, affine transformation, etc.), the relative positions and orientations between various types of images are estimated, and the optimal transformation parameters are determined so that the images are accurately aligned in a unified spatial coordinate system.

[0122] Image overlay: Based on transformation parameters, various images are seamlessly overlaid to form a unified and high-quality 3D image. Interpolation and resampling techniques are used to handle differences in resolution and sampling rate between images, ensuring that the overlaid image has excellent coherence and clarity.

[0123] c. Multimodal feature fusion

[0124] Feature representation: Unifying the representation of image features acquired by various cameras and transforming them into comparable and fusion forms, such as vectors and matrices.

[0125] Feature fusion strategy: Select and apply appropriate feature fusion strategies, such as weighted average and max pooling, based on the nature and importance of the features to meet diagnostic needs.

[0126] Feature selection and optimization: Feature selection and optimization are carried out using techniques such as feature dimensionality reduction and feature selection algorithms to remove redundant and irrelevant features and improve the representativeness and discriminative power of the fused features.

[0127] d. Evaluation and optimization of fusion results

[0128] Evaluation methods: The fused results are rigorously evaluated and verified, using methods such as quantitative evaluation based on manually labeled samples or qualitative evaluation based on actual application scenarios.

[0129] Optimization process: Based on the evaluation results, the fusion strategy and optimization algorithm are continuously adjusted and optimized to ensure the accuracy and reliability of the fusion results.

[0130] e. Results Presentation and Application

[0131] Display methods: The merged integrated image is displayed in the user interface, providing multiple display methods such as split-screen display and overlay display.

[0132] Application Scenarios: Medical staff can use the human anatomy model docking function to comprehensively examine the fused image data from a 3D perspective, enabling unified viewing and comparison. The fused results are not only used for display but also assist doctors in diagnosis and evaluation, providing more comprehensive medical information support.

[0133] Furthermore, a multifunctional, multidimensional visual examination recording system is characterized by further optimizing the human body structural model using medical imaging data, specifically including the following steps:

[0134] a. Acquisition of imaging data: Obtain other medical imaging data of patients from the hospital's information system, including but not limited to CT, MRI, X-ray, B-ultrasound, etc.;

[0135] b. Image preprocessing: Preprocessing the acquired medical image data, including denoising and enhancement, to improve image quality and ensure that images of different modalities have uniform spatial and temporal resolution;

[0136] c. Image and model registration: Using advanced image registration technology, the preprocessed medical image data is accurately registered with the human body structure model that has integrated 3D morphological data. Through key steps such as feature extraction, feature matching, and transformation estimation, the image data is ensured to be accurately aligned with the human body structure model in a unified spatial coordinate system.

[0137] e. Model optimization and refinement: After integrating medical imaging data, the human body structure model is further optimized and refined. The shape, size, texture, etc. of the model are adjusted to ensure that the model is highly consistent with the actual situation of the patient. Detailed features of the model, such as blood vessels, nerves, and lesions, are added or adjusted to provide more comprehensive and accurate human body structure information.

[0138] f. Model Validation and Confirmation: The constructed and optimized human structural model is rigorously validated and confirmed. By comparing it with the patient's actual imaging data or through evaluation by medical experts, it is ensured that the model can accurately reflect the patient's actual condition and provide a reliable basis for subsequent auxiliary diagnosis and evaluation.

[0139] Furthermore, a multi-functional multi-dimensional visual examination recording system is further equipped with an image fusion module. This module is configured to deeply integrate the image data from the visual examination recording system with other medical image data and uniformly map them onto the created human body structure model to form a comprehensive image representation, which is convenient for medical staff to view and compare at the same time. At the same time, the data of all images are directly displayed on the fusion model, providing doctors with a more comprehensive and accurate basis for diagnosis and evaluation.

[0140] The implementation method of the image fusion module includes the following steps:

[0141] I. Image Data Acquisition and Preprocessing

[0142] Image data acquisition:

[0143] Obtain high-quality body surface images from the visual examination recording system, including but not limited to 3D morphology, polarized light images and infrared thermal imaging;

[0144] Obtain other medical imaging data of the patient from the hospital's information system, such as CT and MRI scans;

[0145] Image preprocessing:

[0146] All acquired image data are preprocessed, including denoising and enhancement, to improve image quality;

[0147] Unified spatial and temporal resolution processing is performed on images of different modalities to ensure the accuracy of subsequent fusion;

[0148] II. Image and Model Registration

[0149] Using advanced image registration technology, various image data are accurately registered with the created human body structure model;

[0150] Through key steps such as feature extraction, feature matching, and transformation estimation, we ensure that the image data is accurately aligned with the human body structure model in a unified spatial coordinate system.

[0151] The registered image data is mapped to the corresponding positions on the human body structure model to form a preliminary fused image;

[0152] III. Image Deep Fusion and Data Integration

[0153] Based on the registration results, a highly integrated fusion strategy is adopted to deeply fuse image data from different modalities;

[0154] During the fusion process, the feature information of various images is fully integrated and analyzed, including but not limited to morphological structure, density distribution, temperature distribution, etc.

[0155] The fused image data is closely integrated with the human body structure model to ensure that all image data can be directly viewed on the model;

[0156] IV. Evaluation and Optimization of Fusion Results

[0157] The fused results are rigorously evaluated and verified, using methods such as quantitative evaluation based on manually labeled samples or qualitative evaluation based on actual application scenarios.

[0158] Based on the evaluation results, the fusion strategy and optimization algorithm are continuously adjusted and optimized to ensure the accuracy and reliability of the fusion results;

[0159] V. Display and Application of Fusion Results

[0160] The integrated image is displayed in the user interface, offering multiple display methods such as split-screen display, overlay display, and linked display.

[0161] It allows medical staff to adjust parameters such as image transparency and contrast as needed for better observation and analysis;

[0162] Through the human body structure model docking function, medical staff can comprehensively examine the fused image data from a three-dimensional perspective, enabling unified viewing and comparison.

[0163] The fusion results are not only used for display, but also to assist doctors in diagnosis and evaluation, providing more comprehensive medical information support.

[0164] Furthermore, the specific methods and steps for feature extraction and comparison in the model matching method of a multifunctional multidimensional visual examination recording system are as follows:

[0165] 1. Image Feature Extraction

[0166] Feature extraction from macroscopic images:

[0167] Use the camera module of the visual examination recorder to take photos that include the patient's gross body parts (such as easily identifiable parts like joints and head);

[0168] Image processing techniques (such as edge detection and corner detection) are used to extract representative features from photos, such as joint contours and facial contours.

[0169] Feature extraction from detailed photos:

[0170] High-resolution, high-precision detailed photographs are taken of specific areas (such as skin ulcers, lesions, etc.).

[0171] Extract detailed features, such as the edge morphology of the ulcer, changes in the texture of the surrounding skin, and specific anatomical landmarks (such as blood vessel distribution and skin folds);

[0172] 2. Preparation of Model Feature Library

[0173] Model feature annotation:

[0174] In a high-precision human body structure model, the feature points, contours, and specific anatomical landmarks of each part are marked in detail in advance.

[0175] For the target area being photographed, special attention should be paid to and its anatomical information should be annotated in detail to form a comprehensive and detailed model feature library;

[0176] 3. Feature comparison process

[0177] Compare one by one:

[0178] Each feature point extracted from the overall and detailed photos will be compared one by one with the corresponding features in the model feature library;

[0179] Ensure that every feature point or region (such as edges, corners, textures, etc.) can find a corresponding match in the model feature library;

[0180] Similarity metric:

[0181] Similarity measurement methods (such as Euclidean distance, cosine similarity, etc.) are applied to calculate the degree of similarity between image features and model features;

[0182] For edge features, the spatial distance between edge points can be calculated; for texture features, texture similarity algorithms can be used for comparison.

[0183] Comprehensive analysis:

[0184] A comprehensive analysis of the similarity scores of all compared feature points is conducted, taking into account factors such as the number, distribution, morphological consistency, and rationality of the anatomical structure of the feature points.

[0185] Determine the overall similarity level and assess the accuracy and reliability of feature matching;

[0186] 4. Matching Validation and Adjustment

[0187] Spatial consistency verification:

[0188] Check whether the spatial distribution of the matching feature points is consistent with the actual anatomical structure, and ensure that the relative positional relationship of the feature points is correct;

[0189] Morphological consistency verification:

[0190] Verify whether the shape of the matched feature points matches the shape in the model feature library to ensure that there are no shape abnormalities caused by deformation or mismatch.

[0191] Auxiliary information correction:

[0192] The matching results were further verified and corrected by combining the patient's medical history, the annotation information at the time of shooting, and the timestamps in the video.

[0193] Manual confirmation:

[0194] If necessary, invite medical personnel to manually confirm the location of the imaged area on the human anatomy model to ensure that the information is accurate. Detailed Implementation

[0195] Specific implementation method of model construction in cases of skin ulcers on the lower leg

[0196] Using a patient with skin ulcers on their lower leg as an example, this paper details how to use a visual examination recorder to construct a human body structural model.

[0197] I. Basic Information Collection and Initial Modeling

[0198] Patient information collection: Name: Zhang San, Age: 45, Gender: Male, Height: 175cm, Weight: 70kg, Other relevant information: No history of diabetes, but has been standing for long periods of time at work, and has had skin ulcers on his lower legs for a month.

[0199] Initial modeling:

[0200] Based on Zhang San's basic information (height, weight, gender, age), a general human body model that is closest to the model is selected from the system database as the initial model. This model is built on a large amount of human body data and includes basic anatomical structures and physiological characteristics.

[0201] The initial model was adjusted appropriately to better reflect Zhang San's body proportions and shape, but at this point the model was still relatively generic and lacked personalized details.

[0202] II. Acquisition and Integration of 3D Morphological Data

[0203] 3D morphological data acquisition:

[0204] The 3D camera in the visual examination recorder was used to record Zhang San's lower leg area in a high-precision and efficient three-dimensional manner. Ensure that the ambient lighting is moderate and the patient remains still during the recording to obtain clear 3D morphological data.

[0205] During the filming process, the camera captured the detailed morphology of the skin ulcer on the lower leg, including the size, depth, edge shape, and condition of the surrounding skin.

[0206] Data integration:

[0207] The acquired 3D morphological data is imported into the initial human body structure model. Image registration and calibration techniques are used to ensure precise alignment between the 3D morphological data and the corresponding positions in the model.

[0208] Through data fusion algorithms, details of 3D morphological data (such as the shape and outline of the calf and the specific morphology of the ulcer) are accurately mapped onto the model. This step makes the model more accurate and personalized, and can truly reflect the actual situation of Zhang San's calf skin ulcer.

[0209] Model optimization:

[0210] After integrating the 3D morphological data, the model was further optimized. The shape, size, and texture of the model were adjusted to ensure that the model closely matched Zhang San's actual appearance.

[0211] Special attention is paid to skin ulcers on the lower legs, with fine adjustments made to details such as the shape and depth of the ulcers to provide more accurate diagnostic and assessment data.

[0212] Example of patient case for imaging site recognition: Identification of imaging site for skin ulcers on the lower leg.

[0213] The following are the specific steps for identifying the imaging site in cases of skin ulcers on the lower leg:

[0214] Patient basic information: Patient name: Zhang, Gender: Male, Age: 45, Height: 175cm

[0215] Weight: 70kg. Case description: Chief complaint: Skin ulcer on the lateral side of the left calf. Ulcer description: Approximately 3cm × 4cm in size, accompanied by redness, swelling, pain, and slight exudation;

[0216] Specific implementation steps for image location recognition

[0217] 1. Multimedia Acquisition

[0218] Overall photo shooting:

[0219] Use the camera module of the visual examination recorder to take a gross photograph including the patient's lower leg area;

[0220] Ensure that the photos include easily identifiable parts, such as joints like the knees and ankles, so that they can be initially located on the human anatomy model;

[0221] Detailed photo shooting:

[0222] High-resolution, high-precision detailed photographs were taken of the patient's calf skin ulcers.

[0223] Capture subtle morphological changes in ulcers, such as size, edges, depth, and exudation.

[0224] 2. Feature Extraction

[0225] Feature extraction from macroscopic images:

[0226] Image processing techniques (such as edge detection) are used to extract joint contours (e.g., knees, ankles) from photographs.

[0227] As an easily identifiable feature;

[0228] Feature extraction from detailed photos:

[0229] Extract features such as the edge morphology of the ulcer and changes in the texture of the surrounding skin;

[0230] 3. Model matching

[0231] Model feature library construction:

[0232] A high-precision human body structure model is pre-built, and the features of each part are marked in the model, especially the anatomical landmarks of the lower leg, to form a model feature library;

[0233] Similarity calculation:

[0234] The features extracted from the overall and detailed photos are compared one by one with the features in the model feature library;

[0235] Use similarity metrics (such as Euclidean distance and cosine similarity) to calculate the degree of similarity between image features and model features;

[0236] 4. Determining the shooting location

[0237] The optimal matching position has been determined:

[0238] Based on the similarity calculation results, the position with the highest similarity is selected as the best matching position;

[0239] Determine the exact location of the photographed area (the ulcer on the outer side of the left calf) on the human body structure model; generate the corresponding location information, including the coordinates, size, and orientation of the ulcer on the model;

[0240] 5. Assisted Recognition

[0241] Based on the medical history record:

[0242] Use patient medical history records (such as prolonged standing work, ulcers that have persisted for a month, etc.) to assist in identification; take and label information:

[0243] Quickly locate the general image by combining the annotation information at the time of shooting (such as shooting angle, distance, etc.); manually confirm:

[0244] Medical staff manually confirmed the joint positions in the gross images and further corrected the position of the photographed areas on the model.

[0245] The detailed photos were precisely corrected to ensure that the ulcer site was accurately mapped onto the model.

[0246] The following is a specific example of the implementation of model matching based on imaging site recognition in a case of skin ulceration on the lower leg:

[0247] Patient basic information: Patient name: Zhang; Gender: Male; Age: 45 years old; Height: 175cm; Weight: 70kg.

[0248] Case Description

[0249] Chief complaint: Skin ulcer on the outer side of the left calf.

[0250] Ulcer description: Approximately 3cm x 4cm in size, accompanied by redness, swelling, pain, and slight oozing.

[0251] Specific implementation steps of model matching for image part recognition

[0252] 1. Feature Extraction

[0253] Feature extraction from macroscopic images:

[0254] Use the camera module of the visual examination recorder to take a gross photograph including the patient's lower leg area;

[0255] Image processing techniques (such as edge detection) are used to extract joint contours (such as knees and ankles) from photos as easily identifiable features;

[0256] Feature extraction from detailed photos:

[0257] High-resolution, high-precision detailed photographs were taken of the patient's ulcerated skin area on the lower leg.

[0258] Extract features such as the edge morphology of the ulcer and changes in the texture of the surrounding skin;

[0259] 2. Model Feature Library Construction

[0260] A high-precision human body structure model is pre-built, and the features of each part are marked in the model, especially the anatomical landmarks of the lower leg, forming a model feature library;

[0261] 3. Similarity Calculation

[0262] The features extracted from the overall and detailed photos are compared one by one with the features in the model feature library;

[0263] Use similarity metrics (such as Euclidean distance and cosine similarity) to calculate the degree of similarity between image features and model features;

[0264] 4. Determining the optimal matching position

[0265] Based on the similarity calculation results, the position with the highest similarity score is selected as the best matching position;

[0266] 5. Determine the exact location of the imaging site (the ulcer on the outer side of the left calf) on the human anatomy model; 6. Matching verification and adjustment.

[0267] Combine with medical history records: Use the patient's medical history records (such as prolonged standing work, ulcers that have lasted for a month) to assist in identification;

[0268] Shooting annotation information: Combine the annotation information at the time of shooting (such as shooting angle and distance) to quickly locate the general photo;

[0269] Manual verification: Medical staff manually verify the joint positions in the gross images to further correct the position of the photographed areas on the model; they also perform precise corrections on the detailed images to ensure that the ulcer sites are accurately mapped onto the model.

[0270] Example of the implementation of the medical history recording module in cases of skin ulcers on the lower leg.

[0271] Patient Basic Information: Patient Name: Zhang, Gender: Male, Age: 45, Height: 175cm, Weight: 70kg. Brief Medical History: Zhang presented with a skin ulcer on the lateral side of his left calf. The ulcer was approximately 3cm x 4cm in size, accompanied by redness, swelling, pain, and slight exudation. It had persisted for one month. He had no history of diabetes, but worked standing for long periods.

[0272] Implementation steps of the medical history recording module

[0273] Data structure design

[0274] Design a flexible data structure to store Zhang's medical history information. This structure includes text fields (such as symptom descriptions and diagnosis results), date fields (such as onset date and consultation date), numeric fields (such as ulcer size), and image and video fields (such as photos and video recordings of the ulcer site).

[0275] Ensure that the data structure can support the recording needs of different parts and different medical histories, so as to facilitate subsequent data entry and retrieval;

[0276] User interface design

[0277] Design an intuitive and easy-to-use user interface that includes a dedicated area for inputting and viewing information about the history of skin ulcers on the lower legs;

[0278] The interface includes input boxes, selection boxes, a date picker, and tables or lists for displaying and editing medical history information;

[0279] It provides the ability to quickly select and record medical history information related to skin ulcers on the lower legs, such as ulcer symptoms, treatment process, and medication use.

[0280] Medical history information entry and storage

[0281] Medical staff input Zhang's detailed medical history information through the user interface, including the initial symptoms of the ulcer, the diagnosis process, treatment experience, medication use, etc.

[0282] This information is precisely stored in a data structure and corresponds one-to-one with the location information of the shooting location (outer side of the left calf).

[0283] It supports the automatic import of Zhang's medical history data from electronic medical record systems or other medical information systems, and automatically classifies and integrates the data according to the shooting location, so as to achieve seamless data connection and efficient management;

[0284] Ensure that the entered medical history information is closely linked to Zhang's identification and the location information of the shooting site, so that relevant information can be quickly retrieved and viewed according to the location or medical history type in the future;

[0285] Medical records and location association

[0286] When recording medical history information, the system automatically associates the identified location information of the shooting site (outer side of the left calf) with the corresponding medical history information;

[0287] Medical staff can intuitively compare and analyze Zhang's medical history information with imaging data of specific areas (such as images of ulcer sites taken by 3D cameras);

[0288] It provides the function of classifying, filtering and viewing medical history information according to the location or type of medical history, so that medical staff can more easily understand Zhang's medical history and provide strong support for diagnosis and treatment;

[0289] Completeness of medical history information

[0290] The recorded medical history information covers Zhang's comprehensive medical history, including the historical symptoms of skin ulcers on his lower legs, disease progression, related influencing factors (such as prolonged standing work), diagnostic assessment results, and treatment response, etc.

[0291] Ensuring the completeness and accuracy of records provides a reliable basis for subsequent medical decisions and patient management.

[0292] Example of implementing treatment response tracking function in a case of lower leg skin ulcers: Implementation steps of treatment response tracking function:

[0293] 1. Comprehensive recording and management of treatment response data

[0294] Data structure design:

[0295] Add a field or sub-table specifically for recording treatment response data to the data structure of the medical history recording module. This includes the treatment date, specific treatment plan (such as drug treatment, physical therapy, etc.), improvement of symptoms after treatment (such as changes in ulcer size, reduction of redness and swelling), possible adverse reactions (such as allergic reactions, skin irritation, etc.), and related medical imaging data (such as photos of the ulcer site before and after treatment).

[0296] Data entry:

[0297] Medical staff used an intuitive and easy-to-use user interface to record Zhang's treatment response information in various ways, such as text descriptions, selection of preset response types, and recording key time points. For example, they recorded the situation after the first drug treatment, where the ulcer edges began to scab over and the redness and swelling decreased.

[0298] Data import and association:

[0299] It supports the automatic import of Zhang's treatment response data from electronic medical record systems or other medical information systems, and automatically classifies and integrates the data according to the shooting location (outer side of the left calf), ensuring that the data is closely related to the patient's identity and the location information of the shooting location.

[0300] 2. Visual presentation and in-depth comparison of treatment response data

[0301] Multimodal display:

[0302] The user interface simultaneously displays Zhang's treatment response data, images of the imaging site (such as images of the ulcer site taken with a 3D camera), and other relevant medical information. Split-screen and overlay displays are used to facilitate comparison and analysis of changes in the treatment response and ulcer site by medical staff.

[0303] Timeline function:

[0304] It provides a timeline function, allowing medical staff to dynamically view Zhang's treatment response record in chronological order of treatment. For example, they can clearly observe the trend of significant reduction in ulcer size and substantial subsidence of redness and swelling after two weeks of treatment.

[0305] Advanced comparative analysis tools:

[0306] It integrates advanced comparative analysis tools such as image difference analysis and data statistical analysis to automatically calculate and analyze differences before and after treatment, generating intuitive comparison reports. For example, it automatically generates reports on the percentage reduction in ulcer area, as well as statistical charts showing changes in redness and swelling.

[0307] Intelligent reminders and report generation

[0308] Automatic reminders:

[0309] The system automatically sends treatment response data recording reminders to medical staff according to preset rules or conditions (such as one week after each treatment) to ensure that Zhang's treatment response data is recorded in a timely and accurate manner.

[0310] Report generation:

[0311] It offers powerful report generation capabilities, allowing healthcare professionals to quickly generate detailed and accurate treatment response reports based on treatment response data and imaging data of the imaging sites. Reports can be exported to multiple formats, including PDF and Word, facilitating sharing and discussion. For example, it can generate comprehensive reports that include ulcer treatment progress, symptom improvement, and recommendations for the next treatment steps.

[0312] Specific Implementation of Further Optimizing Human Structure Models Using Medical Imaging Data: Patient Case Example: Cancerous Ulcer on the Back. I. Basic Patient Information: Patient Name: Ms. Li, Gender: Female, Age: 65, Height: 160cm, Weight: 55kg

[0313] II. Case Description

[0314] Chief complaint: Persistent back pain, accompanied by skin ulceration and bleeding.

[0315] Diagnosis: Cancerous ulcer on the back, approximately 5cm x 6cm in size, with irregular edges and necrotic tissue, suspected to be a metastasis of skin cancer.

[0316] III. Specific Implementation Steps for Further Optimizing Human Structural Models Using Medical Imaging Data

[0317] 1. Acquisition of image data

[0318] Visual examination recorder images:

[0319] The 3D camera in the visual examination recorder was used to record Li's back in three dimensions with high precision and efficiency, capturing subtle morphological changes in the ulcer site, including the size, edges, depth, and necrotic tissue condition of the ulcer. A polarized light camera was used to photograph the ulcer and surrounding skin, revealing the fine structural features of the subcutaneous tissue.

[0320] Infrared thermal imaging cameras are used to capture the temperature distribution on the back surface, accurately locating areas of abnormal temperature.

[0321] Other medical imaging data:

[0322] The hospital information system retrieved Li's CT and MRI images, which provided information on the deep structures of his back, including possible tumor metastases and vascular distribution.

[0323] 2. Image Preprocessing

[0324] All acquired image data undergoes preprocessing operations such as denoising and enhancement to improve image quality.

[0325] Unified spatial and temporal resolution processing is performed on images of different modalities to ensure the accuracy of subsequent fusion.

[0326] 3. Image and model registration

[0327] Initial model construction: Based on Li's basic information (height, weight, etc.), a preliminary human body structure model is constructed.

[0328] Registration process:

[0329] Advanced image registration technology is used to accurately register preprocessed medical images such as CT and MRI with preliminary human structural models.

[0330] Through key steps such as feature extraction (e.g., edges, corners, textures), feature matching, and transformation estimation, we ensure that image data is precisely aligned with the human body structure model in a unified spatial coordinate system.

[0331] 4. Model Optimization and Refinement

[0332] Morphological adjustment: Based on the registered image data, the shape and size of the human body structure model are adjusted to ensure that the model is highly consistent with the actual situation of Li's back.

[0333] Adding details: Add or adjust detailed features in the model, such as back muscles, blood vessels, necrotic tissue of cancerous ulcers, etc., especially the shape and depth information of the ulcer site, which must correspond precisely to the images captured by the visual examination recorder.

[0334] Texture optimization: By combining data from polarized light cameras and infrared thermal imaging cameras, the texture and temperature distribution information of the model surface are optimized to make the model more realistic and detailed.

[0335] 5. Model Validation and Confirmation

[0336] Comparative verification: The optimized human body structure model was compared with Li's actual imaging data to ensure that the model could accurately reflect the actual situation of the cancerous ulcer on the back and its surrounding tissues.

[0337] Expert evaluation: Medical experts are invited to evaluate the model to confirm its accuracy and reliability, providing a reliable basis for subsequent auxiliary diagnosis and evaluation.

[0338] Specific implementation of the image fusion module in a case of cancerous ulcer on the back

[0339] I. Basic Patient Information

[0340] Patient Name: Ms. Li, Gender: Female, Age: 65, Height: 160cm, Weight: 55kg

[0341] II. Case Description

[0342] Chief complaint: Persistent back pain, accompanied by skin ulceration and bleeding.

[0343] Diagnosis: Cancerous ulcer on the back, approximately 5cm x 6cm in size, with irregular edges and necrotic tissue, suspected to be a metastasis of skin cancer.

[0344] III. Specific Implementation Steps of the Image Fusion Module

[0345] (I) Image Data Acquisition and Preprocessing

[0346] Image data acquisition

[0347] Visual examination recorder images:

[0348] The 3D camera in the visual examination recorder was used to record Li's back in a high-precision and high-efficiency three-dimensional manner, capturing subtle morphological changes in the ulcer site, including the size, edges, depth, and necrotic tissue of the ulcer.

[0349] Using a polarized light camera to photograph the ulcer and surrounding skin, the fine structural features of the subcutaneous tissue were revealed.

[0350] Infrared thermal imaging cameras are used to capture the temperature distribution on the back surface, accurately locating areas of abnormal temperature.

[0351] Other medical imaging data:

[0352] The hospital information system retrieved Li's CT and MRI images, which provided information on the deep structures of his back, including possible tumor metastases and vascular distribution.

[0353] Image preprocessing

[0354] All acquired image data undergo preprocessing operations such as denoising and enhancement to improve image quality.

[0355] Unified spatial and temporal resolution processing is performed on images of different modalities to ensure the accuracy of subsequent fusion.

[0356] (II) Image and Model Registration

[0357] Initial model construction: Based on Li's basic information (height, weight, etc.), a preliminary human body structure model is constructed.

[0358] Registration process:

[0359] Advanced image registration technology is used to accurately register preprocessed medical images such as CT and MRI with preliminary human structural models.

[0360] Through key steps such as feature extraction (e.g., edges, corners, textures), feature matching, and transformation estimation, we ensure that image data is precisely aligned with the human body structure model in a unified spatial coordinate system.

[0361] (III) Image Deep Fusion and Data Integration

[0362] Deep integration strategy:

[0363] Based on the registration results, a highly integrated fusion strategy is adopted to deeply fuse image data from different modalities.

[0364] During the fusion process, feature information from various types of images is comprehensively integrated and analyzed, including but not limited to morphological structure (from 3D cameras), density distribution (from CT and MRI), temperature distribution (from infrared thermal imaging cameras), and fine texture (from polarized light cameras).

[0365] Data integration:

[0366] The fused image data is tightly integrated with the human body structure model, ensuring that all image data can be directly viewed on the model. For example, tumor metastasis information from CT images is overlaid with ulcer morphology from 3D morphological images, while areas of temperature abnormality shown by infrared thermal imaging are marked in the model.

[0367] (iv) Evaluation and optimization of fusion results

[0368] Evaluation method:

[0369] The fused results are rigorously evaluated and verified, using methods such as quantitative evaluation based on manually labeled samples or qualitative evaluation based on actual application scenarios.

[0370] Medical experts were invited to evaluate the fusion results to confirm the accuracy and clinical relevance of the fusion.

[0371] Optimization process:

[0372] Based on the evaluation results, the fusion strategy and optimization algorithm are continuously adjusted and optimized, such as adjusting the parameters of feature extraction and optimizing the registration algorithm, to ensure the accuracy and reliability of the fusion results.

[0373] (V) Display and Application of Integration Results

[0374] Display method:

[0375] The user interface displays the fused composite image, offering various display methods such as split-screen display, overlay display, and linked display. For example, it can simultaneously display the overlay effect of 3D morphological images, temperature distribution images, and CT images.

[0376] It allows medical staff to adjust parameters such as image transparency and contrast as needed for better observation and analysis.

[0377] Application scenarios:

[0378] Through the human body structure model docking function, medical staff can comprehensively examine the fused image data from a three-dimensional perspective, enabling unified viewing and comparison. For example, doctors can intuitively see the shape and depth of the ulcer site, as well as the temperature distribution of the surrounding tissue, and can further examine tumor metastasis information in CT images, conducting a comprehensive analysis from the surface to the depths.

[0379] The fusion results are not only used for display but also to assist doctors in diagnosis and evaluation, providing more comprehensive medical information support. For example, by combining the morphology and structure of the ulcer site, temperature distribution, and tumor metastasis information in CT images, doctors can more accurately determine the type, severity, and possible cause of the ulcer, thereby developing a more effective treatment plan.

Claims

1. A multi-functional, multi-dimensional visual examination recording system, comprising: The 3D camera acquisition module is specifically designed to perform high-efficiency human body surface morphology, accurately capture subtle morphological changes, and provide detailed basic data for constructing high-precision human body stereoscopic images. Human Body Structure Model Mapping Module: This module ensures that the captured human body parts are accurately mapped onto a high-precision human body structure model. Through this module, users can easily and comprehensively examine the specific condition of the captured parts from a three-dimensional perspective, achieving intuitive and three-dimensional observation and analysis. The specific implementation method of the human body structure model mapping module includes the following steps: a. Model building: First, a high-precision human body structure model is built, which includes detailed anatomical structures and physiological features to provide an accurate basis for subsequent mapping of the shooting parts; b. Camera Location Recognition: Using image recognition technology or manual annotation, identify the camera location and match it with the corresponding position in the human body structure model; c. Precise mapping: Precisely mapping the location information of the identified shooting parts in the image onto the human body structure model; d. Dynamic updates: As filming progresses, the mapping information on the human body structure model is updated in real time to reflect the latest status of the filmed area; e. 3D perspective viewing: Provides interactive operations such as rotation, zoom, and pan, allowing users to freely adjust the viewing angle and observation distance to gain a deeper understanding of the details and features of the shooting location; f. Assisted diagnosis and assessment: Combining medical knowledge and algorithms, the system automatically analyzes the imaged area and provides diagnostic suggestions or assessment reports to assist doctors in making decisions.

2. The specific steps of the model construction method in the human body structure model docking function according to claim 1 include: a. Basic information gathering and initial modeling: First, collect basic information about the patient, including height, weight, gender, age, etc. This information helps to provide basic parameters for subsequent modeling. Based on this information, a preliminary human anatomy model is constructed. This model can be a general model based on average parameters, and will be adjusted and optimized based on the patient's specific data in the future. b. 3D morphological data acquisition and integration: The 3D camera in the visual examination recorder is used to record the human body surface morphology in a high-precision and efficient manner. This provides detailed human morphological data, including but not limited to the size, shape, and outline of various body parts; By integrating these 3D morphological data with a preliminary human body structure model, and using data fusion techniques such as image registration and calibration, the 3D morphological data is accurately mapped onto the model, resulting in a more accurate and personalized patient human body model.

3. The identification of the shooting location in the human body structure model docking function according to claim 1, characterized in that, The specific method for identifying the imaged body part includes the following steps: a. Multimedia Acquisition: First, multimedia data is acquired through the camera module of the visual examination recorder, including taking gross photographs, detailed photographs, and videos. The gross photographs and videos should include easily identifiable parts of the human body, such as joints (shoulder, elbow, wrist, hip, knee, ankle), head, face, buttocks, back, perineum, forearm, etc., for quick preliminary positioning on the human body structure model; the detailed photographs and videos are taken at high resolution and high precision for specific shooting areas to capture more detailed features and dynamic changes; b. Feature extraction: In the acquired images and videos, representative features are extracted using image processing techniques. These features include, but are not limited to, edges, corners, textures, motion trajectories, and specific anatomical landmarks in the images and videos. c. Model matching: The extracted features are matched with the corresponding features in the pre-built or acquired high-precision human body structure model, and the best matching position is determined by similarity measurement methods (such as Euclidean distance, cosine similarity, etc.). d. Determining the shooting location: Based on the model matching results, determine the specific location of the shooting location on the human body structure model, and generate the corresponding location information, including coordinates, size, orientation, etc. e. Assisted Recognition: To improve recognition accuracy, the system can also combine other auxiliary information, such as the patient's medical history, annotations at the time of shooting, timestamps in the video, or manual confirmation by medical staff. It first quickly locates the general image and video, and then precisely corrects and confirms the shooting location in detailed images and videos. By comprehensively utilizing multimedia data, accurate identification and positioning of the shooting location can be achieved.

4. According to claim 3, a model matching method for identifying the imaging site in a multifunctional multidimensional visual examination recording system, characterized in that, Includes the following steps: a. Model feature library construction: Construct or acquire a high-precision human body structural model, which must include detailed anatomical structures and physiological characteristics; In the human body structure model, the feature points, contours and specific anatomical landmarks of each part are marked in detail, especially the detailed anatomical information of the target part, forming a comprehensive and detailed model feature library. b. Feature extraction and comparison: From the gross and detail photos captured by the visual examination recorder camera module, representative features are extracted using advanced image processing technology, including but not limited to edges, corners, textures, and specific anatomical landmarks. The extracted image features are compared one by one with the features in the model feature library to ensure that each feature point or region is accurately matched. c. Similarity calculation: Similarity measurement methods, such as Euclidean distance and cosine similarity, are applied to calculate the degree of similarity between image features and model features, and a similarity score is obtained. A comprehensive analysis of the similarity scores of all compared feature points is performed to determine the overall similarity level; d. Determining the optimal matching position: Based on the similarity calculation results, the position with the highest similarity score is selected as the best matching position of the shooting part on the human body structure model; To verify the accuracy of the matching results, factors such as the spatial distribution of feature points, morphological consistency, and the rationality of anatomical structures are considered to ensure the precision of the matching position. e. Auxiliary information fusion and correction: The matching results were further verified and corrected by combining the patient's medical history, the annotation information at the time of shooting, and the timestamps in the video. If necessary, invite medical personnel to manually confirm the location of the imaged area on the human anatomy model to ensure that the information is accurate.

5. The visual examination recorder according to claim 1, characterized in that, It also includes a medical history recording module, which comprehensively and meticulously records the patient's medical history information and closely links this information with the precise location of the imaging site on the human body structure model; The medical history recording module includes: a. A data structure design unit, configured to design a data structure for storing patient medical history information, wherein the data structure can flexibly contain various types of data such as text, date, numerical value, image and video to adapt to the recording needs of different parts and different medical histories; b. User interface design unit, configured to design an intuitive and easy-to-use user interface, which should include a dedicated area for inputting and viewing medical history information from different sites, including input boxes, selection boxes, date pickers for inputting medical history information, and tables or lists for displaying and editing medical history information, while providing functions to facilitate medical staff to quickly select and record relevant medical history information based on the imaging site; c. The medical history information entry and storage unit is configured as follows: Medical staff are allowed to input detailed medical history information for different parts of the patient through the user interface, including but not limited to symptom descriptions, diagnosis results, treatment process, medication use, etc., and this information is accurately stored in the data structure, forming a one-to-one correspondence with the location information of the imaging site; It supports the automatic import of patients' medical history data from electronic medical record systems or other medical information systems, and automatically classifies and integrates the data according to the shooting location, so as to achieve seamless data connection and efficient management; Ensure that the entered medical history information is closely linked to the patient's identification and the location information of the imaging site, so that relevant information can be quickly retrieved and viewed according to the location or type of medical history later; d. The medical record and location association unit is further configured as follows: When recording medical history information, the system automatically associates the identified location information of the imaging site with the corresponding medical history information, enabling medical staff to intuitively compare and analyze the patient's medical history information with the imaging data of specific sites. It provides the function of classifying, filtering and viewing medical history information according to the location or type of medical history, so that medical staff can more easily understand the patient's medical history and provide strong support for diagnosis and treatment; e. The medical history information should cover the patient's comprehensive medical history, including but not limited to historical symptoms in different parts of the body, disease progression, relevant influencing factors, diagnostic assessment results, and treatment response, to ensure the completeness and accuracy of the record.

6. According to claim 5, a medical history recording module including treatment response tracking function, characterized in that, Configured as follows: a. A comprehensive unit for recording and managing treatment response data: In the data structure of the medical history recording module, a field or sub-table is added specifically for recording treatment response data. The recorded data covers the treatment date, specific treatment plan, symptom improvement after treatment, possible adverse reactions, and related medical imaging data. It provides an intuitive and easy-to-use user interface, supporting medical staff to enter treatment response information in various ways, such as text description, selection of preset response types, and recording of key time points; It supports the automatic import of treatment response data from electronic medical record systems or other medical information systems, ensuring seamless data connection and efficient integration; Ensure that the entered treatment response information is closely linked to the patient's identification and the location information of the imaging site to guarantee the integrity and traceability of the data; b. Visual presentation and in-depth comparison of treatment response data: The user interface simultaneously displays the patient's treatment response data, imaging data of the imaging site, and other relevant medical information; It employs multiple display methods, such as split-screen display, overlay display, and linked display, to facilitate medical staff in comparing and analyzing changes in treatment response and imaging sites; It provides a timeline function, enabling medical staff to dynamically view the patient's treatment response records in chronological order of treatment time, thereby clearly observing the changing trend of the treatment response and the dynamic evolution of the imaging data of the imaging site. It integrates advanced comparative analysis tools such as image difference analysis and data statistical analysis to automatically calculate and analyze the differences before and after treatment and generate intuitive comparison reports; c. Intelligent reminder and report generation unit: The system automatically sends treatment response data recording reminders to medical staff based on preset rules or conditions, ensuring timely and accurate recording of treatment response data; It provides powerful report generation capabilities, enabling medical staff to quickly generate detailed and accurate treatment response reports based on treatment response data and imaging data of the imaging sites; It supports exporting reports to multiple formats such as PDF and Word for easy sharing and discussion.

7. A multi-functional, multi-dimensional visual examination recording system, further comprising a polarized light camera acquisition module, an infrared thermal imaging camera acquisition module, and a data fusion module; The polarization camera acquisition module reveals and adjusts the microstructural features under the skin tissue by finely controlling the polarization state of polarized light, thus enhancing the visualization ability of subcutaneous tissue structures. The infrared thermal imaging camera acquisition module is designed to capture and analyze the infrared radiation emitted from the human body surface, generate detailed and high-precision temperature distribution images, and then accurately locate areas of local temperature abnormalities or potential lesions. The data fusion module is responsible for deeply and accurately integrating the data information collected by the 3D camera, polarized light camera, and infrared thermal imaging camera to construct a three-dimensional image of the human body and record its dynamic changes in real time.

8. A multi-functional, multi-dimensional visual examination recording system, characterized in that, This also includes further optimizing the human body structure model using medical imaging data, specifically including the following steps: a. Acquisition of imaging data: Obtain other medical imaging data of patients from the hospital's information system, including but not limited to CT, MRI, X-ray, B-ultrasound, etc.; b. Image preprocessing: Preprocessing the acquired medical image data, including denoising and enhancement, to improve image quality and ensure that images of different modalities have uniform spatial and temporal resolution; c. Image and model registration: Using advanced image registration technology, the preprocessed medical image data is accurately registered with the human body structure model that has integrated 3D morphological data. Through key steps such as feature extraction, feature matching, and transformation estimation, the image data is ensured to be accurately aligned with the human body structure model in a unified spatial coordinate system. e. Model optimization and refinement: After integrating medical imaging data, the human body structure model is further optimized and refined. The shape, size, texture, etc. of the model are adjusted to ensure that the model is highly consistent with the actual situation of the patient. Detailed features of the model, such as blood vessels, nerves, and lesions, are added or adjusted to provide more comprehensive and accurate human body structure information. f. Model Validation and Confirmation: The constructed and optimized human structural model is rigorously validated and confirmed. By comparing it with the patient's actual imaging data or through evaluation by medical experts, it is ensured that the model can accurately reflect the patient's actual condition and provide a reliable basis for subsequent auxiliary diagnosis and evaluation.

9. A multi-functional multi-dimensional visual diagnosis recording system, further equipped with an image fusion module, which is configured to deeply fuse the image data of the visual diagnosis recording system with other medical image data and uniformly map it onto the created human body structure model to form a comprehensive image representation, which is convenient for medical staff to view and compare at the same time. At the same time, the data of all images are directly displayed on the fusion model, providing doctors with a more comprehensive and accurate basis for diagnosis and evaluation. The implementation method of the image fusion module includes the following steps: I. Image Data Acquisition and Preprocessing Image data acquisition: Obtain high-quality body surface images from the visual examination recording system, including but not limited to 3D morphology, polarized light images and infrared thermal imaging; Obtain other medical imaging data of the patient from the hospital's information system, such as CT and MRI scans; Image preprocessing: All acquired image data are preprocessed, including denoising and enhancement, to improve image quality; Unified spatial and temporal resolution processing is performed on images of different modalities to ensure the accuracy of subsequent fusion; II. Image and Model Registration Using advanced image registration technology, various image data are accurately registered with the created human body structure model; Through key steps such as feature extraction, feature matching, and transformation estimation, we ensure that the image data is accurately aligned with the human body structure model in a unified spatial coordinate system. The registered image data is mapped to the corresponding positions on the human body structure model to form a preliminary fused image; III. Image Deep Fusion and Data Integration Based on the registration results, a highly integrated fusion strategy is adopted to deeply fuse image data from different modalities; During the fusion process, the feature information of various images is fully integrated and analyzed, including but not limited to morphological structure, density distribution, temperature distribution, etc. The fused image data is closely integrated with the human body structure model to ensure that all image data can be directly viewed on the model; IV. Evaluation and Optimization of Fusion Results The fused results are rigorously evaluated and verified, using methods such as quantitative evaluation based on manually labeled samples or qualitative evaluation based on actual application scenarios. Based on the evaluation results, the fusion strategy and optimization algorithm are continuously adjusted and optimized to ensure the accuracy and reliability of the fusion results; V. Display and Application of Fusion Results The integrated image is displayed in the user interface, offering multiple display methods such as split-screen display, overlay display, and linked display. It allows medical staff to adjust parameters such as image transparency and contrast as needed for better observation and analysis; Through the human body structure model docking function, medical staff can comprehensively examine the fused image data from a three-dimensional perspective, enabling unified viewing and comparison. The fusion results are not only used for display, but also to assist doctors in diagnosis and evaluation, providing more comprehensive medical information support.

10. According to claim 4, the specific method and steps for feature extraction and comparison in the model matching method of a multifunctional multidimensional visual examination recording system are as follows:

1. Image Feature Extraction Feature extraction from macroscopic images: Use the camera module of the visual examination recorder to take photos that include the patient's gross body parts (such as easily identifiable parts like joints and head); Image processing techniques (such as edge detection and corner detection) are used to extract representative features from photos, such as joint contours and facial contours. Feature extraction from detailed photos: High-resolution, high-precision detailed photographs are taken of specific areas (such as skin ulcers, lesions, etc.). Extract detailed features, such as the edge morphology of the ulcer, changes in the texture of the surrounding skin, and specific anatomical landmarks (such as blood vessel distribution and skin folds); 2. Preparation of Model Feature Library Model feature annotation: In a high-precision human body structure model, the feature points, contours, and specific anatomical landmarks of each part are marked in detail in advance. For the target area being photographed, special attention should be paid to and its anatomical information should be annotated in detail to form a comprehensive and detailed model feature library; 3. Feature comparison process Compare one by one: Each feature point extracted from the overall and detailed photos will be compared one by one with the corresponding features in the model feature library; Ensure that every feature point or region (such as edges, corners, textures, etc.) has a corresponding match in the model feature library; similarity metric: Similarity measurement methods (such as Euclidean distance, cosine similarity, etc.) are applied to calculate the degree of similarity between image features and model features; For edge features, the spatial distance between edge points can be calculated; for texture features, texture similarity algorithms can be used for comparison. Comprehensive analysis: A comprehensive analysis of the similarity scores of all compared feature points is conducted, taking into account factors such as the number, distribution, morphological consistency, and rationality of the anatomical structure of the feature points. Determine the overall similarity level and assess the accuracy and reliability of feature matching; 4. Matching Validation and Adjustment Spatial consistency verification: Check whether the spatial distribution of the matching feature points is consistent with the actual anatomical structure, and ensure that the relative positional relationship of the feature points is correct; Morphological consistency verification: Verify whether the shape of the matched feature points matches the shape in the model feature library to ensure that there are no shape abnormalities caused by deformation or mismatch. Auxiliary information correction: The matching results were further verified and corrected by combining the patient's medical history, the annotation information at the time of shooting, and the timestamps in the video. Manual confirmation: If necessary, invite medical personnel to manually confirm the location of the imaged area on the human anatomy model to ensure that the information is accurate.