A software and hardware system for multi-modal imaging and multi-dimensional index evaluation of scars

Abstract

This invention discloses a multimodal imaging and multidimensional index evaluation software and hardware system for scars, involving an infrared, RGB, and stereoscopic three-modal imaging hardware system, and an intelligent calculation and monitoring software system for more than twenty scar evaluation indicators. The intelligent evaluation system for scars includes indicators such as perimeter, length, width, thickness, area, volume, color, roughness, consistency, elasticity, blood flow density, risk of continued scar growth, and color difference. This forms an objective, multidimensional, precise, and intelligent evaluation system for the scar diagnosis and treatment process, thereby assisting in diagnosis and treatment, as well as monitoring and tracking the entire life cycle of scars. This contributes to the visualization, objectification, and accuracy of the scar diagnosis and treatment process, providing doctors and patients with comprehensive diagnostic and treatment evidence and enabling tracking and tracing.

Description

Technical Field

[0001] This invention relates to the field of medical imaging and intelligent analysis technology, specifically to a system and method for objective and quantitative evaluation of human skin scars, and in particular to a multimodal imaging and multidimensional intelligent analysis system that integrates infrared, visible light and three-dimensional morphological information. Background Technology

[0002] Scars are a product of the skin's natural repair process after trauma, and their assessment is crucial for treatment planning, efficacy monitoring, and prognosis. Currently, clinical evaluation mainly relies on physicians' visual observation and palpation, using subjective scales such as the Vancouver Scar Scale. This method has significant limitations.

[0003] High subjectivity: Different doctors have different experience and judgment standards, resulting in poor consistency of assessment results.

[0004] Qualitative: The scale scores are mostly graded (such as "red, pink, purple"), lacking precise quantitative data.

[0005] Limited dimensionality: It is difficult to accurately assess multiple key dimensions of scars simultaneously, such as color, thickness (volume), texture, and blood flow.

[0006] Difficult to track: The lack of objective data records makes it difficult to accurately compare and analyze the long-term dynamic changes of scars.

[0007] Existing technologies include some devices that utilize single imaging techniques (such as laser speckle scanning and 3D scanning) to assist in scar assessment. However, these devices typically only acquire information in a single dimension (such as only 3D morphology or only blood perfusion), failing to comprehensively and holistically reflect the complex pathological state of scars. Therefore, developing an intelligent evaluation system that can integrate multimodal information and provide multidimensional, high-precision, and quantifiable indicators is an urgent need in clinical practice. Summary of the Invention

[0008] (a) Purpose of the invention This invention aims to overcome the aforementioned deficiencies of existing technologies and provide a multimodal imaging and multidimensional index evaluation system and method for scars. This system can automatically, synchronously, or quasi-synchronously acquire infrared blood flow imaging, high-resolution color (RGB) images, and three-dimensional morphological data of scars. Through integrated intelligent software, it calculates quantitative evaluation indicators covering multiple dimensions such as morphology, color, texture, and blood flow, forming an objective, accurate, repeatable, and traceable intelligent scar evaluation scheme to assist in clinical diagnosis and treatment decisions.

[0009] Achieve comprehensive, objective, and precise quantification in assessment. Construct electronic medical records for scars, enabling digital monitoring and tracking throughout their entire lifecycle, providing visualized and data-driven objective evidence for efficacy evaluation. Integrate these functions into a user-friendly hardware and software system to improve clinical diagnostic efficiency and the reliability of research data.

[0010] (II) Technical Solution To achieve the above objectives, the present invention adopts the following technical solution: A multimodal imaging and multidimensional index evaluation system for scars includes a hardware imaging subsystem and a software analysis subsystem.

[0011] The hardware imaging subsystem is used for integrated acquisition of multimodal image data of the scar area. It includes at least: Infrared imaging module: Employs an uncooled infrared focal plane detector to acquire infrared blood flow maps characterizing the surface of the scar area. The distribution of these images is closely related to subcutaneous blood perfusion and inflammatory activity.

[0012] RGB color imaging module: Employs a high-resolution visible light camera to acquire true color and texture images of scars under standard lighting conditions.

[0013] Stereoscopic 3D imaging module: used to acquire a precise 3D point cloud or mesh model of the scar area. In a preferred embodiment, the module is a structured light 3D scanner, including a pattern projector and at least one camera; in other embodiments, a binocular stereo vision system or a laser 3D scanner may also be used.

[0014] The hardware imaging subsystem can be designed as a portable all-in-one device, with the optical axes of the three modules precisely calibrated and fixed to ensure that spatially aligned multimodal data can be acquired in a single shot. Alternatively, a split design can be adopted, with a positioning device ensuring data registration.

[0015] The software analysis subsystem is communicatively connected to the hardware imaging subsystem and is used to receive and process the multimodal image data. Its core functions include: Data registration and fusion unit: Performs high-precision spatial registration of infrared images and RGB images from different sensors with the 3D model to generate a fused data model in which each 3D point is associated with color (RGB) and blood flow (IR) information.

[0016] Intelligent indicator calculation unit: Based on the fused data model, it automatically segments the scar area and calculates quantitative evaluation indicators across multiple dimensions. These indicators include, but are not limited to: Morphological indicators: perimeter, maximum length, maximum width, average thickness, two-dimensional projected area, three-dimensional surface area, and three-dimensional volume of the scar.

[0017] Colorimetric indicators: color difference between the scar area and the surrounding normal skin area (e.g., ΔE value based on CIE L*a*b* color space), and color consistency within the scar area (e.g., standard deviation of color values).

[0018] Texture metrics: surface roughness of the scar (e.g., standard deviation or fractal dimension based on the thickness distribution of a 3D point cloud), and / or elastic parameters estimated based on image analysis or integrated micro-force sensing modules.

[0019] Hemodynamic parameters: blood flow density (defined as the percentage of pixel area above a specific infrared threshold) obtained from infrared image analysis, and the degree of risk of continued scar growth reflecting the extent of active inflammation.

[0020] Data Management and Tracking Unit: This unit establishes independent files for patients and their scars, linking and storing multimodal images from all examinations, all calculated indicators, and evaluation reports. It provides comparative views of historical data and can plot trend curves of any indicator over time, enabling full lifecycle monitoring.

[0021] (III) Beneficial Effects Compared with the prior art, the system provided by the present invention has the following significant advantages: Objective and precise, comprehensive in quantification: It completely replaces subjective scales, providing more than 20 precise quantitative indicators that cover all key dimensions of scar evaluation, making diagnostic and treatment decisions based on data.

[0022] Information fusion for three-dimensional assessment: For the first time, morphological (3D), color (RGB), and functional (infrared blood flow) information are fused in the same coordinate system, realizing a panoramic depiction of the pathological state of scars in the form of a "3D color map", with information depth far exceeding that of single-modal devices.

[0023] Intelligent and efficient, one-click operation: The software automatically completes the entire process from data registration and region segmentation to indicator calculation, which greatly simplifies the operation steps, improves clinical work efficiency, and lowers the threshold for use.

[0024] Dynamic tracking and closed-loop management: Digitalized full-cycle records enable each assessment to be accurately compared with historical data, intuitively displaying the therapeutic effect, truly realizing refined and personalized management of scars, and promoting the upgrading of the diagnosis and treatment model.

[0025] It has a wide range of applications: it can be used for rapid screening and assessment in daily clinical practice, as well as for precise measurement in scientific research and objective efficacy verification of new drugs and therapies. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the system workflow described in this invention.

[0027] Figure 2 This is a schematic diagram of the overall hardware structure of one embodiment of the system described in this invention.

[0028] Figure 3 This is a flowchart of the main functional modules of the software analysis subsystem described in this invention.

[0029] Figure 4 This is a schematic diagram of multimodal image data registration and fusion in the embodiment.

[0030] Figure 5 This is a schematic diagram illustrating the principle of calculating scar volume based on a three-dimensional mesh model in the embodiment.

[0031] Figure 6 This is a schematic diagram of the indicator trend analysis report generated by the data management and tracking module in the embodiment. Detailed Implementation

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0033] Example 2: System Hardware Configuration like Figure 2 As shown, the system in this embodiment adopts an integrated handheld design. The housing integrates: A high-resolution RGB camera, An uncooled infrared focal plane detector A miniature structured light projector and two synchronized monocular cameras (together forming a structured light 3D scanning module).

[0034] All optical components have parallel optical axes and fixed positions, ensuring spatial consistency of multimodal data. The device connects to the workstation via a USB-C interface to run the software analysis subsystem.

[0035] Example 2: Software Analysis Process and Core Algorithm like Figure 3 As shown, the software workflow includes: data acquisition, registration and fusion, feature extraction, indicator calculation, report generation, and data archiving.

[0036] 1. Multimodal data registration: like Figure 4As shown, firstly, using the RGB image as a reference, the infrared image is spatially aligned using feature point matching (such as SIFT and ORB) and perspective transformation. Then, using known camera calibration parameters and the 3D-2D projection relationship, the 3D point cloud data is precisely registered with the RGB image, ensuring that each 3D point corresponds to an RGB color value and a blood flow value (from the registered infrared image), forming an integrated fusion data model of "color-blood flow-3D coordinates".

[0037] 2. Calculation method for key indicators (supporting claims 11-14): (1) Calculation of color difference ΔE (corresponding to claim 11): The software converts the RGB image to the device-independent CIE L*a*b* color space. The user selects a typical scar area (S) and an adjacent normal skin area (N) on the RGB image. The software calculates the average color values ​​of all pixels in both areas, resulting in (L_s, a_s, b_s) and (L_n, a_n, b_n) respectively. The color difference ΔE is calculated using the following formula: ΔE = √[(L_s - L_n)² + (a_s - a_n)² + (b_s - b_n)²] like Figure 5 As shown, this value represents the Euclidean distance between two color points in the color space. The larger the ΔE value, the more significant the color difference. This calculation is one of the standard color difference calculation methods, objectively quantifying the degree of erythema or pigmentation of scars.

[0038] (2) Surface roughness measurement (corresponding to claim 12): For the registered 3D point cloud of the scar region, a reference plane representing the "average skin surface" is first calculated using a planar or low-order surface fitting algorithm. Then, the perpendicular distance (thickness residual) z_i from all scar surface sampling points to this reference plane is calculated. Roughness is characterized by calculating the standard deviation σ of these thickness residuals. σ = √[ (1 / (N-1)) * Σ_{i=1}^{N} (z_i - ż)² ] Where N is the total number of sampling points in the scar area, and ż is the arithmetic mean of all z_i. The larger the σ value, the more severe the surface undulation and the rougher the scar.

[0039] (3) Calculation of blood flow density BD (corresponding to claim 13): The software analyzes the scar region in the registered infrared blood flow image. A blood flow threshold T_th is set (e.g., the average blood flow of normal skin plus twice the standard deviation). The number of pixels in the scar region with blood flow values ​​greater than T_th is counted, and their corresponding physical area is denoted as A_v (calculated according to the imaging resolution). Blood flow density BD is defined as: BD = (A_v / A_t) * 100% Where A_t is the total area of ​​the scar region (which can be obtained from RGB or 3D data). The BD value reflects the proportion of "active" blood flow areas within the scar that are above the basal metabolic rate, and is an important indicator for measuring scar congestion and inflammation.

[0040] (4) Calculation of scar volume V (corresponding to claim 14): like Figure 4 As shown, this is one of the core advantages of this system. First, in the fused data model, the three-dimensional triangular mesh surface M_s of the entire scar region is automatically or semi-automatically and accurately segmented based on color and / or texture features. Then, the "normal skin surface" M_n, assuming no scar, is reconstructed using algorithms (such as extrapolation or fitting based on the surrounding normal skin surface). The volume V of the scar is the volume of the closed space Ω enclosed by surfaces M_s and M_n. The calculation method uses integration over three-dimensional voxels or direct summation over tetrahedral meshes: V = ∫∫∫_{Ω} dx dy dz In practice, this can be achieved by calculating the algebraic sum of the volumes of all corresponding triangular prisms between M_s and M_n. This method is efficient and accurate, and can accurately reflect the raised volume of hypertrophic scars or the depressed volume of atrophic scars.

[0041] 3. Data Management and Tracking: The software creates a unique profile for each patient and each scar. The results of each assessment (original image, fusion model, and all indicator values) are saved in association with the assessment date. Figure 5 As shown, users can select any indicator (such as volume V, color difference ΔE), and the software will automatically generate a line graph showing the change of the indicator over time, clearly displaying the evolution trend of the scar during the treatment process and providing direct evidence for efficacy evaluation.

[0042] Example 3: Clinical Application Scenarios The doctor held the device approximately 30cm away from the patient's scar area and performed a single scan, which took about 2 seconds. The software automatically processed the data and generated a report within 10 seconds, showing: "Scar volume: 4.5 cm³; Color difference ΔE: 15.6 (significant erythema); Roughness σ: 0.32; Blood flow density: 0.55. Compared with data from three months ago, the volume decreased by 30%, ΔE decreased to 10.2, and blood flow density decreased to 18%, objectively demonstrating the effectiveness of anti-scar treatment."

[0043] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multimodal imaging and multidimensional index evaluation system for scars, characterized in that, include: Hardware imaging subsystem and software analysis subsystem; The hardware imaging subsystem is used to acquire multimodal image data of the scar area, and includes at least: an infrared imaging module, an RGB color imaging module, and a stereoscopic three-dimensional imaging module; The software analysis subsystem is communicatively connected to the hardware imaging subsystem and is used to receive and process the multimodal image data, and automatically calculate and output multiple-dimensional scar evaluation indicators based on the multimodal image data; the multiple-dimensional scar evaluation indicators include at least indicators selected from morphology, colorimetry, texture, and hemodynamics.

2. The system according to claim 1, characterized in that, The stereo imaging module is a binocular stereo vision system.

3. The system according to claim 1, characterized in that, The morphological parameters calculated by the software analysis subsystem include at least one or more of the following: perimeter, length, width, thickness, two-dimensional area, and three-dimensional volume of the scar.

4. The system according to claim 1, characterized in that, The colorimetric indicators calculated by the software analysis subsystem include at least one or more of the following: color difference between the scar and the surrounding normal skin, and color consistency of the scar area.

5. The system according to claim 1, characterized in that, The textural indices calculated by the software analysis subsystem include at least one or more of the following: surface roughness and elasticity of the scar.

6. The system according to claim 1, characterized in that, The hemodynamic parameters calculated by the software analysis subsystem include at least one or more of the following: blood flow density in the scar area obtained from infrared imaging data and the degree of risk of continued scar growth.

7. The system according to claim 1, characterized in that, The software analysis subsystem also includes a data management and tracking module, which is used to associate and store multimodal image data of the same scar collected at different time points and corresponding evaluation indicators to form a diagnosis and treatment record of the scar throughout its entire life cycle, and to provide a visual analysis of the trend of indicator changes.

8. The system according to any one of claims 1-7, characterized in that, The system is integrated into a portable device, or the hardware imaging subsystem and the software analysis subsystem are separate structures connected by wired or wireless means.

9. A method for multi-dimensional evaluation of scars based on the system described in any one of claims 1-8, characterized in that, Includes the following steps: S1: Use the hardware imaging subsystem to simultaneously or sequentially image the target scar area to acquire infrared images, RGB images and three-dimensional model data; S2: Transmit the multimodal image data obtained in step S1 to the software analysis subsystem; S3: The software analysis subsystem performs registration and fusion processing on the multimodal image data; S4: Based on the registered multimodal data, automatically calculate the preset scar evaluation indicators in multiple dimensions; S5: Output the calculation results of the indicators and generate a comprehensive evaluation report.

10. The method according to claim 9, characterized in that, Following step S5, step S6 is also included: comparing and analyzing the imaging data and indicator results of this assessment with historical data, tracking the changes of scars over time, and updating the scar treatment record.

11. The system according to claim 4, characterized in that, The color difference ΔE is calculated as follows: In the CIE L*a*b* color space, the average color value (L_s, a_s, b_s) of the scar area and the average color value (L_n, a_n, b_n) of the surrounding normal skin area are selected, and the result is calculated using the following formula: ΔE = √[(L_s - L_n)² + (a_s - a_n)² + (b_s - b_n)²].

12. The system according to claim 5, characterized in that, The roughness is quantified by a statistical index of surface undulation, specifically by calculating the standard deviation σ of the thickness of the scar's three-dimensional surface point cloud data relative to the reference fitted surface: σ = √[ (1 / (N-1)) * Σ_{i=1}^{N} (z_i - ż)² ] Where N is the number of sampling points, z_i is the thickness value of the i-th point, and ż is the average thickness of all points.

13. The system according to claim 6, characterized in that, The blood flow density BD is obtained by processing infrared blood flow imaging data, specifically by calculating the ratio of the pixel area A_v within the target scar region where blood flow is higher than a set threshold T_th to the total area A_t of that region: BD = (A_v / A_t) * 100%.

14. The system according to claim 3, characterized in that, The volume V is calculated using the triangular mesh model obtained by the stereoscopic 3D imaging module, and is obtained by voxel integration or tetrahedral segmentation. Its integral formula is expressed as: V = ∫∫∫_{\Omega} dx dy dz Wherein, Ω is the closed domain of the scar region in three-dimensional space, which is determined by integrating the space between the scar surface model and the simulated normal skin surface.

Images