Foot arch damage degree intelligent evaluation method and system based on multi-model fusion

Abstract

The invention discloses a multi-model fusion-based foot arch damage degree intelligent evaluation method and system, and belongs to the technical field of medical image processing, and the method comprises the steps: obtaining a foot load X-ray side position film; a DeepLabv3 + image segmentation model is adopted to process the foot load X-ray lateral position film, and a foot bone segmentation image is obtained; based on the foot bone segmentation image, identifying key anatomical points required by foot arch evaluation through a YOLOv8 anatomical point positioning model; according to the spatial position relation of the key anatomical points, arch related angle data such as an inner side longitudinal arch angle, an outer side longitudinal arch angle, a front arch angle and a rear arch angle are calculated; based on the foot arch related angle data, determining a foot arch damage degree grade in combination with a forensic judicial appraisal and evaluation system; according to the method, automation and standardization of foot arch damage assessment are achieved through multi-model fusion, the assessment efficiency is remarkably improved, and the measurement precision is improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method for analyzing foot X-ray images using deep learning technology. Specifically, it is an intelligent assessment method and system for the degree of arch damage based on multi-model fusion, applicable to forensic identification, judicial practice, and clinical medicine. Background Technology

[0002] The arch of the foot is a convex, arch-shaped structure formed by the tarsal and metatarsal bones of the foot, along with ligaments and tendons. It possesses a high degree of arch strength and elasticity, serving to cushion shocks. In forensic medical practice, the extent of damage to the arch structure is a crucial basis for determining the severity of foot injuries in cases such as traffic accidents, falls from heights, and injuries caused by falling heavy objects.

[0003] Currently, the commonly used method in forensic clinical practice to assess the degree of arch damage involves forensic examiners manually locating key anatomical points on lateral X-ray films of the foot, such as the lowest point of the calcaneus and the lowest point of the first metatarsal head. The degree of arch damage is then evaluated by measuring and calculating key data such as the medial longitudinal arch angle, lateral longitudinal arch angle, anterior arch angle, and posterior arch angle. However, this traditional manual measurement method has the following significant shortcomings:

[0004] 1. High subjectivity: The selection of measurement points and the measurement results of angles rely excessively on the professional knowledge and operational skills of the appraisers, and different people may draw biased conclusions;

[0005] 2. Inefficiency: The manual measurement process is time-consuming and cannot meet the timeliness requirements of large-scale identification;

[0006] 3. Limited accuracy: Manual operation inevitably introduces measurement errors, affecting the scientific nature and accuracy of the assessment opinions.

[0007] Although medical image processing technology has made significant progress in recent years, its application in the specialized field of foot arch damage assessment remains insufficient. Most existing computer-aided measurement systems are designed for general medical images and lack the specialized ability to identify the specific anatomical structures of the foot arch, making it difficult to provide sufficiently accurate assessments of the degree of arch damage. Furthermore, these systems typically use a single model for image processing, making it difficult to simultaneously ensure both accuracy and processing efficiency when facing high-precision applications such as forensic identification.

[0008] Therefore, there is an urgent need to develop an intelligent system that can objectively, efficiently, and accurately assess the degree of damage to the foot arch, providing reliable technical support for forensic identification. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing technologies by providing an intelligent assessment method and system for the degree of arch damage based on multi-model fusion. The aim is to establish an objective and efficient evaluation system for arch damage, providing accurate reference data for forensic identification and rating.

[0010] This invention proposes an intelligent assessment method for the degree of arch damage based on multi-model fusion, including:

[0011] Obtain lateral X-ray images of the foot bearing weight uploaded by the user;

[0012] The foot weight-bearing lateral X-ray film was processed using the DeepLabv3+ image segmentation model to obtain segmented images of the foot bones;

[0013] Based on the foot bone segmentation images, the key anatomical points required for foot arch assessment are identified using the YOLOv8 anatomical point localization model.

[0014] Calculate the relevant angle data of the foot arch based on the spatial relationship of the key anatomical points;

[0015] Based on the aforementioned arch-related angle data, and in conjunction with the forensic judicial appraisal and evaluation system, the degree of arch damage is determined.

[0016] Generate an evaluation report that includes the original image, annotation points, and calculation results.

[0017] Preferably, after obtaining the user-uploaded weight-bearing lateral X-ray film of the foot, the method further includes:

[0018] The weight-bearing lateral X-ray film of the foot is preprocessed, including image denoising, contrast enhancement and size normalization, to improve image quality;

[0019] The image denoising uses a Gaussian filtering algorithm, and the contrast enhancement uses histogram equalization technology.

[0020] Preferably, the process of using the DeepLabv3+ image segmentation model to process the foot weight-bearing X-ray lateral view specifically includes:

[0021] Image feature extraction based on the DeepLabv3+ architecture of the Xception backbone network;

[0022] Multi-scale contextual information is obtained through the void space pyramid pooling module;

[0023] Generate pixel-level classification results for the foot skeleton;

[0024] By refining the segmentation boundaries, segmentation images of different skeletal structures are obtained.

[0025] Preferably, the identification of key anatomical points required for arch assessment using the YOLOv8 anatomical point localization model specifically includes:

[0026] Receive the segmented image of the foot bone as input;

[0027] Multi-scale features are extracted using the CSPDarknet backbone network and feature pyramid structure.

[0028] Locating key anatomical landmarks includes the lowest point of the calcaneus, the highest point of the talonavicular joint, and the lowest point of the first metatarsal head;

[0029] Generate the spatial coordinates and confidence values ​​of key anatomical points.

[0030] Preferably, the location includes key anatomical landmarks such as the lowest point of the calcaneus, the highest point of the talonavicular joint, and the lowest point of the first metatarsal head, and also includes:

[0031] The effectiveness of the detected key points is verified based on anatomical relationships;

[0032] Key points with confidence levels below a preset threshold are repositioned or interpolated for correction.

[0033] Automatic correction is performed on abnormal positioning points to ensure accurate positioning of key anatomical points.

[0034] Preferably, the calculation of the arch-related angle data specifically includes:

[0035] The medial longitudinal arch angle is calculated based on the lowest point of the calcaneus, the highest point of the talonavicular joint, and the lowest point of the first metatarsal head.

[0036] The lateral longitudinal arch angle is calculated based on the lowest point of the calcaneus, the quadrate-cuboidal joint of the quadrate bone, and the lowest point of the fifth metatarsal head.

[0037] Calculate the anterior bow angle based on the arc formed by the lowest points of each metatarsal head;

[0038] The posterior bow angle is calculated based on the angle formed by the posterior parts of the calcaneus and talus.

[0039] Preferably, determining the degree of arch damage specifically includes:

[0040] Compare the calculated angle data with the normal range;

[0041] Based on the degree of angular deviation, the degree of arch damage is classified into mild, moderate, and severe:

[0042] When the angular deviation is less than 15°, it is judged as minor damage;

[0043] When the angular deviation is between 15° and 30°, it is judged as moderate damage;

[0044] When the angular deviation is greater than 30°, it is judged as severe damage.

[0045] Preferably, the generation of an evaluation report including the original image, labeled points, and calculation results includes:

[0046] The original weight-bearing lateral X-ray film of the foot, the segmented image, and the marked key anatomical points are integrated into the same interface;

[0047] Displays the specific values ​​of the inner longitudinal bow angle, outer longitudinal bow angle, front bow angle, and rear bow angle;

[0048] Generate a rating conclusion on the degree of arch damage and explain its corresponding clinical significance.

[0049] As a preferred option, it also includes:

[0050] The accuracy of the system evaluation results is verified by comparing them with the results of manual measurements by experts, and the system measurement error is calculated.

[0051] The system is considered stable and reliable when the fluctuation of repeated measurements of the same image is less than 3°.

[0052] Store evaluation data in a historical database for subsequent model optimization and performance improvement.

[0053] A multi-model fusion-based intelligent assessment system for the degree of arch damage includes:

[0054] The data input module is used to acquire weight-bearing lateral X-ray films of the foot uploaded by the user;

[0055] An image preprocessing module is used to perform noise reduction, enhancement, and normalization on the weight-bearing lateral X-ray film of the foot.

[0056] The DeepLabv3+ image segmentation module is used to process pre-processed weight-bearing lateral X-ray images of the foot to obtain segmented images of the foot bones;

[0057] The YOLOv8 anatomical point localization module is used to identify key anatomical points required for arch assessment based on the foot bone segmentation image.

[0058] An angle calculation module is used to calculate foot arch-related angle data based on the spatial relationship of the key anatomical points.

[0059] The rating and determination module is used to determine the degree of damage to the foot arch based on the relevant angle data of the foot arch and in conjunction with the forensic judicial appraisal evaluation system.

[0060] The results display module is used to generate an evaluation report that includes the original image, labeled points, and calculation results.

[0061] The modules are sequentially connected via a data transfer interface to form a complete process for assessing the degree of arch damage.

[0062] The beneficial effects of this invention are mainly reflected in:

[0063] 1. It has achieved automation and standardization of foot arch damage assessment, significantly improving assessment efficiency. The processing time for a single image has been reduced from 5-10 minutes using traditional manual methods to within 3 seconds, improving efficiency by approximately 100 times.

[0064] 2. Through the objective identification and calculation of the deep learning model, the subjective factors in manual measurement are eliminated, making the evaluation results more objective and reliable. The measurement angle error is controlled within ±3°, which is significantly better than the ±8° error range of manual measurement.

[0065] 3. By adopting a multi-model fusion strategy, the advantages of different models are fully utilized, and the image segmentation accuracy reaches over 95%, the key point localization accuracy exceeds 97%, and high-precision evaluation of the foot arch structure is achieved.

[0066] 4. A standardized grading system for foot arch damage has been established, providing objective and accurate data support for forensic identification and improving the scientific rigor and authority of the identification conclusions;

[0067] 5. The system has good scalability and transfer learning capabilities. Through continuous data accumulation and model optimization, the system performance can be continuously improved, adapting to a wider range of application scenarios. Attached Figure Description

[0068] Figure 1 This is a segmentation diagram of a lateral X-ray film of the foot after processing by the system of the present invention, which intuitively shows the segmentation results of different bone structures.

[0069] Figure 2 The foot bone segmentation image obtained by the DeepLabv3+ image segmentation model of this invention clearly shows the boundaries of each bone structure.

[0070] Figure 3 This is a system interface diagram of the present invention, which includes the original image, the processed image, and the calculated angle data. Detailed Implementation

[0071] Please refer to the attached document. Figure 1-3 The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific embodiments. Based on the general knowledge of those skilled in the art, various changes and modifications that can be understood within the scope of the technology disclosed in this invention fall within the protection scope of this invention.

[0072] This invention provides an intelligent assessment method for the degree of arch damage based on multi-model fusion, which mainly includes the following steps:

[0073] First, the system acquires weight-bearing lateral X-ray images of the foot uploaded by the user. In practice, the system receives weight-bearing lateral X-ray images of the foot in DICOM format or common image formats (such as JPG, PNG, etc.) as input data for subsequent processing. The system supports single-image processing or batch image processing to meet identification needs of different scales.

[0074] Next, the aforementioned weight-bearing lateral X-ray images of the foot are processed using the DeepLabv3+ image segmentation model to obtain segmented images of the foot bones. This step is one of the core technologies of this invention, automatically identifying and segmenting the skeletal structure of the foot through deep learning technology, laying the foundation for subsequent key point localization. Figure 1 and Figure 2 The segmentation effect is shown, with different bone structures marked with different colors, making them clearly distinguishable.

[0075] Subsequently, based on segmented images of the foot bones, the YOLOv8 anatomical point localization model was used to identify key anatomical points required for foot arch assessment. These key points include the lowest point of the calcaneus, the highest point of the talonavicular joint, and the lowest point of the first metatarsal head, which are fundamental to calculating the foot arch angle. The system automatically identifies these points using deep learning technology, avoiding subjective errors from manual labeling.

[0076] Based on the spatial relationships of key anatomical points, the system calculates relevant arch angle data, including the medial longitudinal arch angle, lateral longitudinal arch angle, anterior arch angle, and posterior arch angle. These angles are core indicators for assessing the degree of arch damage and are automatically derived through mathematical calculations, ensuring the consistency and accuracy of measurements.

[0077] Based on the calculated arch angle data and combined with the forensic medical evaluation system, the system automatically determines the degree of arch damage. The rating result objectively reflects the damage status of the arch structure and provides a reference for subsequent forensic identification.

[0078] Finally, the system generates an evaluation report containing the original image, labeled points, and calculation results, visually displaying the evaluation process and conclusions for easy review and use by evaluators. Figure 3 As shown, the system interface centrally displays the original image, the processed image, and the calculated angle data.

[0079] After obtaining the weight-bearing lateral X-ray film of the foot uploaded by the user, the present invention also includes a step of preprocessing the image to improve the accuracy of subsequent processing.

[0080] Specifically, the system first performs image denoising on the weight-bearing lateral X-ray film of the foot. Noise is inevitably introduced into X-ray images during the imaging process, affecting image quality and subsequent analysis. This invention employs a Gaussian filtering algorithm for denoising, which effectively removes Gaussian noise while preserving image edge information. In practice, a Gaussian kernel size of 5×5 and a standard deviation of 0.8 were chosen; these parameters were determined through extensive experimentation to be optimal, maximizing the preservation of bone edge details while reducing noise.

[0081] Secondly, the system performs contrast enhancement processing on the image. X-ray images typically have low contrast, and the boundary between bone structure and soft tissue is not clearly defined. This invention uses histogram equalization technology to improve image contrast, making bone structure clearer and more discernible. In practical applications, the system employs an adaptive histogram equalization algorithm, sets the window size to 1 / 8 of the image size, and limits the contrast enhancement threshold to 3.0. These parameter settings effectively avoid noise amplification caused by over-enhancement.

[0082] In addition, the system performs image size standardization. Since X-ray images captured by different devices may vary in size, to ensure processing consistency, the system uniformly adjusts all input images to a standard size of 512×512 pixels. This size setting satisfies the input requirements of the deep learning model while preserving sufficient image detail.

[0083] Through the above preprocessing steps, the system significantly improves the quality of the input image, laying a solid foundation for subsequent image segmentation and keypoint localization. The preprocessed image exhibits approximately 80% less noise, 40% higher contrast, clearer image details, and more defined skeletal boundaries, thus enhancing the accuracy of subsequent processing.

[0084] This invention employs the DeepLabv3+ image segmentation model to process weight-bearing lateral X-ray images of the foot, achieving accurate segmentation of the foot bones. DeepLabv3+ is an advanced semantic segmentation model, and this invention has specifically optimized it to adapt to the characteristics of foot X-ray images.

[0085] In its implementation, this invention extracts image features based on the DeepLabv3+ architecture of the Xception backbone network. The Xception network employs a depthwise separable convolutional structure, significantly reducing computational cost while maintaining model performance. The system is set with an input size of 512×512×1 (grayscale image) and a backbone network depth of 65 layers. This depth setting ensures feature extraction capability while avoiding the gradient vanishing problem that may occur with excessively deep networks.

[0086] To acquire multi-scale contextual information, this invention introduces a dilated spatial pyramid pooling (ASPP) module into the DeepLabv3+ model. This module uses dilated convolutions with different dilation rates to effectively expand the receptive field and capture image features at different scales. In this embodiment, the ASPP module employs three sets of dilated convolutions with dilation rates of 6, 12, and 18, along with a global average pooling branch and a 1×1 convolution branch, forming a total of five parallel processing paths, each with 256 output channels. This design allows the model to simultaneously focus on local details and global structure, making it particularly suitable for processing complex anatomical structures such as the foot bones.

[0087] The mathematical expression for dilated convolution is as follows:

[0088] ,

[0089] in: To output feature map at position The value; Input feature map; These are the kernel weights; The dilation rate controls the spacing between elements in the convolution kernel. `index` represents the kernel index, indicating its position within the kernel. This is achieved by adjusting the dilation rate. It can expand the receptive field and capture a wider range of contextual information without increasing the number of parameters.

[0090] After feature extraction, the model generates pixel-level classification results for the foot bones. This invention classifies the foot bones into six categories: calcaneus, fibula, cuboid, phalanges, tibia, and background. The classification uses a softmax function to calculate the probability of each pixel belonging to each category.

[0091] ,

[0092] in: For pixels Category The probability of; The category corresponding to this pixel The network output value, i.e., the original score before activation; This represents the total number of categories, which is 6 in this example. This is a category index, with values ​​ranging from 1 to... ; is the base of the natural logarithm.

[0093] To improve the accuracy of the segmentation boundaries, the system refines the initial segmentation results. Specifically, the Conditional Random Field (CRF) algorithm is used to optimize the segmentation boundaries, defining an energy function:

[0094] ,

[0095] in: Let be the energy function, representing the segmentation result. The total energy value; the smaller the value, the higher the segmentation quality. The unary potential energy term represents the pixel. Marked as The cost is usually calculated based on the probability value output by the classifier; This is a binary potential energy term, representing adjacent pixels. and Marker compatibility is typically related to differences in pixel values ​​and spatial distance; and For pixel index; and Representing pixels and The category labeling. CRF optimization minimizes the energy function, making the segmentation results more consistent with the actual shape of the bone boundaries.

[0096] After the above processing, the system obtained high-quality foot bone segmentation images, with different bone structures clearly marked in different colors. In actual testing, the segmentation model of this invention achieved a pixel-level accuracy of 95.3% on a foot X-ray image dataset, and the bone boundary IoU (Intersection over Union) index reached 0.918, which is superior to existing general medical image segmentation methods. This provides a reliable foundation for subsequent keypoint localization.

[0097] This invention, based on segmented images of the foot bones, identifies key anatomical points required for foot arch assessment using the YOLOv8 anatomical point localization model. YOLOv8 is an efficient target detection model, and this invention adapts it to suit medical key point localization tasks.

[0098] In its implementation, the system first receives a segmented image of the foot bones obtained after processing by the DeepLabv3+ model as input. Different skeletal structures in the segmented image are clearly identified, providing a clear anatomical reference for key point localization.

[0099] The system extracts multi-scale features through the CSPDarknet backbone network and feature pyramid structure. CSPDarknet adopts a CrossStage Partial Network design, which effectively alleviates the gradient loss problem and improves feature extraction efficiency. In this embodiment, the CSPDarknet depth is set to 53 layers, and the input size is consistent with the segmentation model output, which is 512×512×3 (color segmentation map). The Feature Pyramid Network (FPN) fuses feature maps from different levels through top-down paths and lateral connections to form a multi-scale feature representation. The FPN is designed with 5 scale levels, namely 64×64, 32×32, 16×16, 8×8, and 4×4, with 256 channels in each level.

[0100] The mathematical representation of multi-scale features is as follows:

[0101] ,

[0102] in: The first in the feature pyramid The feature map of a layer is a two-dimensional feature tensor; Feature maps of the corresponding layers of the backbone network; for Convolution operations are used to adjust the number of channels; For upsampling operations, nearest neighbor interpolation is typically used. This represents an element-wise addition operation. The formula starts from the top of the feature pyramid and constructs each layer of feature map from top to bottom, combining semantic information at different scales.

[0103] Based on the extracted multi-scale features, the system locates key anatomical landmarks, including the lowest point of the calcaneus, the highest point of the talonavicular joint, and the lowest point of the first metatarsal head. These points are fundamental to calculating the arch angle of the foot, therefore, the accuracy of the location directly affects the reliability of the assessment results. In this embodiment, the system locates a total of 10 key anatomical points, including: the lowest point of the calcaneus, the posterior end of the calcaneus, the supraorbital point of the talus, the talonavicular joint, the supraorbital point of the navicular bone, the supraorbital point of the first cuneiform, the supraproximal point of the first metatarsal, the lowest point of the first metatarsal head, the proximal end of the fifth metatarsal, and the lowest point of the fifth metatarsal head.

[0104] For each keypoint, the system generates its spatial coordinates and confidence score. Keypoint localization uses a heatmap regression method to generate a two-dimensional Gaussian distribution heatmap for each target point.

[0105] ,

[0106] in: For the first A key point is in the location The heatmap value indicates the probability that the location is a key point of the target; The actual coordinates of the key points; The standard deviation of the Gaussian kernel is used to control the diffusion range of the heatmap; The base of the natural logarithm The exponential function; Point To the key point The square of the Euclidean distance. In this implementation, The value is set to the image width. The pixel size is approximately 5 pixels. This setting ensures positioning accuracy while taking into account a certain positional tolerance.

[0107] During training, the system uses the mean squared error loss function to optimize heatmap prediction:

[0108] ,

[0109] in: The heatmap loss represents the degree of difference between the predicted heatmap and the target heatmap. For predicting heatmaps; For target heatmap; This represents the total number of key points, which is 10 in this example. The number of valid pixels; Indicates all Sum of the key points; This represents the summation of all pixel positions in the heatmap; This indicates the location of the predicted heatmap and the target heatmap. The mean square error.

[0110] During the inference phase, the system extracts the coordinates of key points from the predicted heatmap:

[0111] ,

[0112] in: The coordinates of the key points to be predicted; This indicates a search for the function that maximizes the value. coordinate; For the first Predictive heatmaps of key points.

[0113] To further improve positioning accuracy, the system employs a sub-pixel-level positioning algorithm, refining the coordinates near the maximum value in the heatmap through weighted averaging.

[0114] ,

[0115] ,

[0116] in: and The coordinates are refined and have subpixel accuracy; and These are the initial integer coordinates for positioning. Indicates heatmap in The value of the position; other terms are calculated similarly; the numerator calculates the gradient in the horizontal or vertical direction; the denominator... This is the normalization factor.

[0117] In addition, the system calculates the confidence value for each key point for subsequent validity verification:

[0118] ,

[0119] in: For the first The confidence level of each key point ranges from [0,1]. This indicates that the maximum value in the heatmap is taken. The higher the confidence level, the more reliable the location of that point.

[0120] In actual testing, the YOLOv8 keypoint localization model of this invention achieved a point detection accuracy of 97.2% (average error less than 3 pixels) on foot X-ray images, which is higher than the 92.1% accuracy of traditional HOG features and random forest methods, providing high-precision point data for foot arch angle calculation.

[0121] After locating key anatomical landmarks, including the lowest point of the calcaneus, the highest point of the talonavicular joint, and the lowest point of the first metatarsal head, this invention also conducted effectiveness verification and correction, further improving the accuracy and reliability of the location.

[0122] First, the system validates the detection of key points based on anatomical relationships. The foot's skeletal structure has specific anatomical positional relationships; for example, the lowest point of the calcaneus should be located on the posterolateral aspect of the foot, and the lowest point of the first metatarsal head should be located on the anterior aspect of the foot. The system verifies the relative positional relationships of each key point using preset anatomical rules, detecting possible abnormal locations. Specific rules include:

[0123] 1. The x-coordinate of the lowest point of the calcaneus should be less than the x-coordinate of the point on the talus;

[0124] 2. The y-coordinate of the distance from the navicular joint should be less than the y-coordinate of the lowest point of the calcaneus and the lowest point of the first metatarsal head;

[0125] 3. The x-coordinate of the lowest point of the first metatarsal head should be greater than the x-coordinate of the distance from the navicular joint;

[0126] 4. The x-coordinate of the lowest point of the fifth metatarsal head should be greater than the x-coordinate of the proximal end of the fifth metatarsal.

[0127] When a key point is detected that violates the above rules, the system will mark these points as potential anomalies.

[0128] Secondly, the system relocates or interpolates keypoints with confidence levels below a preset threshold. In practice, the system sets a confidence threshold of 0.75; keypoints below this threshold are considered to have potential positioning errors. For points with low confidence levels but still within the 0.5-0.75 range, the system uses a local search strategy to relocate them in the surrounding area; for points with confidence levels below 0.5, the system uses an interpolation method based on anatomical relationships for location estimation.

[0129] The mathematical expression for interpolation correction is as follows:

[0130] ,

[0131] in, The coordinates of the point that needs to be corrected are a two-dimensional vector. and These are the coordinates of the relevant reference points, which are also two-dimensional vectors. These are interpolation coefficients, determined based on anatomical relationships, and are typically in the range of 0.3-0.7; they are scalars. This represents the vector from reference point 1 to reference point 2. For example, if the talonavicular joint is missing, it can be estimated by interpolation based on the suprascapular and scaphoid points, in which case α is approximately 0.5.

[0132] In addition, the system automatically corrects outlier locations. For keypoints marked as potential outliers, the system combines anatomical constraints and statistical models for correction. The statistical model, based on a large number of correctly labeled samples, establishes a spatial relationship model between keypoints.

[0133] ,

[0134] in: The corrected point position is a two-dimensional vector. This represents the original predicted point location, which is also a two-dimensional vector. A reference point can be a single point or a set of multiple points; The average value of the reference points is a standard location obtained based on training data statistics; The correlation matrix represents the degree of influence of changes in the reference point on the target point. It is a 2×2 matrix obtained through data learning. This indicates the offset of the reference point relative to its standard position.

[0135] Through the above verification and correction steps, the system significantly improves the reliability of key point localization and effectively handles localization errors that may occur under complex image conditions. In practical applications, the accuracy of key point localization after correction increases from the original 97.2% to 99.1%, providing more accurate basic data for subsequent angle calculations.

[0136] This invention calculates relevant angle data of the foot arch based on the spatial relationship of key anatomical points, and uses this data as a core indicator to assess the degree of damage to the foot arch.

[0137] First, the system calculates the medial longitudinal arch angle based on the lowest point of the calcaneus, the highest point of the talonavicular joint, and the lowest point of the first metatarsal head. The medial longitudinal arch angle is an important indicator for assessing the medial structure of the foot arch, and the normal value is usually between 120° and 130°. The calculation formula is as follows:

[0138] ,

[0139] in: The inner longitudinal arch angle is expressed in degrees. The vector pointing from the highest point of the navicular joint to the lowest point of the calcaneus; Let be the vector pointing from the highest point of the navicular joint to the lowest point of the first metatarsal head; Represents the dot product of two vectors; and Let these represent the magnitudes of the two vectors respectively; This represents the inverse cosine function, converting cosine values ​​into angles.

[0140] In practical implementation, vectors and It can be represented as:

[0141] ,

[0142] ,

[0143] in: The coordinates of the lowest point of the calcaneus; The coordinates are the distance from the highest point of the navicular joint; The coordinates are the lowest point of the first metatarsal head.

[0144] Secondly, the system calculates the lateral longitudinal arch angle based on the lowest point of the calcaneus, the quadrate-cuboidal joint of the quadrate bone, and the lowest point of the fifth metatarsal head. The lateral longitudinal arch angle reflects the arch of the lateral structures of the foot, and the normal value is usually between 150° and 170°. The calculation method is similar to that of the medial longitudinal arch angle.

[0145] ,

[0146] in: The outer longitudinal bow angle, in degrees; The vector pointing from the quadriceps-cuboidal joint to the lowest point of the calcaneus; This is the vector pointing from the quadriceps cuboidal joint to the lowest point of the fifth metatarsal head; other symbols have the same meaning as above.

[0147] Next, the system calculates the anterior arch angle based on the arc formed by the lowest points of each metatarsal head. The anterior arch angle represents the shape of the transverse arch of the forefoot, with a normal value of approximately 160°-170°. The calculation formula is as follows:

[0148] m

[0149] in: The forward bow angle is expressed in degrees. This is the vector pointing from the third metatarsal head to the first metatarsal head; This is the vector pointing from the third metatarsal head to the fifth metatarsal head; other symbols have the same meaning as above.

[0150] Finally, the system calculates the posterior arch angle based on the angle formed by the posterior parts of the calcaneus and talus. The posterior arch angle reflects the structural morphology of the heel, with a normal value of approximately 20°-30°. The calculation formula is as follows:

[0151] ,

[0152] in: The rear bow angle is expressed in degrees. The coordinates of the calcaneal tuberosity; The coordinates of the lowest point of the calcaneus; These are the coordinates of a point on the talus. This represents the arctangent function, which converts ratios into angles.

[0153] Through the above calculations, the system obtained comprehensive arch angle data, which objectively reflects the morphological characteristics of the arch structure and provides a quantitative basis for assessing the degree of arch damage. In practical applications, the angle error calculated by the system is controlled within ±3°, meeting the accuracy requirements of forensic identification.

[0154] This invention, based on calculated arch angle data and combined with a forensic medical evaluation system, determines the degree of arch damage and provides objective evidence for forensic identification.

[0155] First, the system compares the calculated angle data with the normal range. Based on extensive clinical statistical data, this invention sets normal reference ranges for each arch angle: medial longitudinal arch angle 120°-130°, lateral longitudinal arch angle 150°-170°, anterior arch angle 160°-170°, and posterior arch angle 20°-30°. The system calculates the deviation between the actual measured value and the median normal value as the basis for rating.

[0156] Secondly, based on the degree of angular deviation, the system classifies the severity of arch damage into three levels: mild, moderate, and severe. The classification criteria are as follows:

[0157] When the angular deviation is less than 15°, it is considered minor damage. Minor damage typically manifests as a basically intact arch structure with remaining elasticity, slightly limited function, and minimal impact on daily activities. In forensic assessments, minor damage usually corresponds to minor injury, with a lower compensation ratio.

[0158] When the angular deviation is between 15° and 30°, it is considered moderate damage. Moderate damage is characterized by significant deformation of the arch structure, a marked decrease in elasticity, and moderate functional limitation, significantly impacting daily activities. In forensic assessment, moderate damage usually corresponds to minor injury, with a moderate proportion of compensation.

[0159] An angular deviation greater than 30° is considered severe damage. Severe damage manifests as severe collapse or deformation of the foot arch structure, loss of elasticity, and severely limited function, potentially leading to long-term walking difficulties. In forensic assessments, severe damage typically corresponds to serious injury, resulting in a higher compensation ratio.

[0160] In its implementation, the system employs a weighted comprehensive scoring method, considering the combined impact of multiple indicators:

[0161] ,

[0162] Where: Score is a comprehensive score, dimensionless; , , and These are the median values ​​of the normal values ​​for each angle, in degrees; , , and These are the critical thresholds for each angle (usually set to 15°), in degrees; , , and These are weighting coefficients, dimensionless, set based on clinical experience, and are typically... , , , This reflects the dominant role of the medial longitudinal arch angle in foot arch assessment; This indicates taking the absolute value.

[0163] The final rating criteria are:

[0164] ,

[0165] Where: Grade is the final rating result; Score is the aforementioned comprehensive score; 1 and 2 are rating thresholds, which are dimensionless.

[0166] This rating method enables an objective and quantitative assessment of the degree of arch damage, providing a scientific basis for forensic identification. In practical applications, the rating method achieves a consistency rate of 92.5% with professional forensic grading, significantly higher than the 78.3% consistency rate of traditional manual methods, demonstrating its practical value in judicial identification.

[0167] This invention generates an evaluation report that includes the original image, labeled points, and calculation results, visually displaying the evaluation process and conclusions, making it easy for evaluators to understand and use.

[0168] Specifically, the system first integrates the original weight-bearing lateral X-ray film of the foot, the segmented image, and the marked key anatomical points into a single interface. For example... Figure 3 As shown, the system interface displays the original X-ray image on the left and the processed segmented image on the right, with the two displayed correspondingly for easy comparison. On the segmented image, the system marks key anatomical points with different colors and connects related points with line segments, visually demonstrating the basis of angle calculation.

[0169] Secondly, the system displays the specific values ​​of the medial longitudinal arch angle, lateral longitudinal arch angle, anterior arch angle, and posterior arch angle. These angle data are presented in tabular form, including measured values, normal reference values, and deviation values, enabling assessors to clearly understand the specific changes in the foot arch structure. In the actual interface, the system uses color coding to mark abnormal angles: angles close to normal are displayed in green, slightly abnormal angles in yellow, and severely abnormal angles in red, enhancing data readability.

[0170] Finally, the system generates a rating conclusion on the degree of arch damage and its corresponding clinical significance interpretation. The rating conclusion directly states the level of arch damage (mild, moderate, or severe), along with corresponding clinical interpretations. For example, mild arch damage is characterized by a basically intact arch structure with a slight decrease in elasticity, and regular follow-up examinations are recommended. Severe arch damage is characterized by severe collapse of the arch structure and severely limited function, which may require surgical intervention. These interpretations help assessors understand the practical significance of the rating results and provide a reference for subsequent assessment work.

[0171] In addition, the system provides an export function for assessment reports, supporting multiple formats such as PDF and DOCX, facilitating archiving and sharing. The exported reports contain complete image data, measurement results, and rating conclusions, meeting the standardized requirements for forensic identification documents.

[0172] Through this intuitive and comprehensive report presentation method, the system greatly improves the interpretability and usability of the assessment results, enabling appraisers to easily understand and apply the automated assessment results, thereby improving work efficiency and appraisal quality.

[0173] The present invention also includes a method for verifying and optimizing system evaluation results to ensure the accuracy and reliability of the system.

[0174] First, the system verifies the accuracy of the evaluation results by comparing them with expert manual measurements to calculate the system's measurement error. During the verification process, 100 foot X-ray images were randomly selected, and three senior forensic experts independently performed manual measurements. The average value was used as a reference standard and compared with the system's measurement results. The error calculation formula is as follows:

[0175] ,

[0176] in: The average error is expressed in degrees. This represents the sample size, which is 100 in this example. The first measurement of the system Angle values ​​for each sample, in degrees; The first measurement by experts Angle values ​​for each sample, in degrees; This represents the absolute error between the system's measurement results and those of experts. This represents summing over all N samples; This indicates that the average value has been calculated.

[0177] The verification results show that the average error of the inner longitudinal arch angle measured by the system is 2.3°, the average error of the outer longitudinal arch angle is 2.8°, the average error of the front arch angle is 2.5°, and the average error of the rear arch angle is 2.1°, all of which are controlled within the error range of ±3°, which is better than the ±8° error range of the traditional manual method, proving the high accuracy of the system measurement.

[0178] Secondly, the system's stability was evaluated through repeated testing. The same image was processed 10 times, and the standard deviation of the angle measurements was calculated.

[0179] ,

[0180] in: Standard deviation, expressed in degrees, represents the degree of dispersion of measurement results; For the first The measurement results are in degrees. The average of 10 measurements, in degrees; This represents the summation of 10 measurements; This indicates the calculation of the average value; This indicates the calculation of the square root.

[0181] The system is considered stable and reliable when the fluctuation of repeated measurements of the same image is less than 3°. In actual testing, the average standard deviation of repeated measurements was 1.2°, with a maximum of no more than 2.5°, indicating that the system has good stability and repeatability.

[0182] In addition, the system stores evaluation data in a historical database for subsequent model optimization and performance improvement. The historical database contains complete information such as original images, segmentation results, keypoint coordinates, angle data, and rating results, providing a data foundation for continuous model learning. The system employs an incremental learning strategy, periodically updating model parameters with new data to continuously improve model performance. The weight update formula for incremental learning is as follows:

[0183] ,

[0184] in The updated model weights are a multi-dimensional vector; The model weights before the update are also a multi-dimensional vector; The learning rate is a scalar that controls the update step size, typically set in the range of 0.001 to 0.01. In new data The gradient of the loss function calculated above is a vector with the same dimension as the weight vector; + indicates vector addition.

[0185] Through the aforementioned verification and optimization methods, the system has continuously improved the accuracy and reliability of its assessments, meeting the stringent requirements of forensic identification. In practical applications, after three rounds of incremental learning, the system's assessment accuracy increased from the initial 89.7% to 95.2%, demonstrating the effectiveness of continuous optimization.

[0186] This invention also provides an intelligent assessment system for the degree of arch damage based on multi-model fusion, including a data input module, an image preprocessing module, a DeepLabv3+ image segmentation module, a YOLOv8 anatomical point localization module, an angle calculation module, a rating judgment module, and a result display module. These modules are sequentially connected through a data transfer interface to form a complete assessment process for the degree of arch damage.

[0187] The data input module is responsible for acquiring weight-bearing lateral X-ray images of the foot uploaded by the user. This module supports multiple image formats, including DICOM, JPG, and PNG, and provides batch processing capabilities to meet the needs of different application scenarios. The data input interface adopts a REST API design for easy integration with other systems.

[0188] The image preprocessing module performs denoising, enhancement, and normalization on the weight-bearing lateral X-ray film of the foot. As mentioned earlier, this module uses Gaussian filtering for denoising, histogram equalization to improve contrast, and normalizes the image to a size of 512×512 pixels, laying the foundation for subsequent processing. The output of the preprocessing module is the improved, normalized image, which serves as the input to the segmentation module.

[0189] The DeepLabv3+ image segmentation module processes pre-processed weight-bearing lateral X-ray images of the foot to obtain segmented images of the foot bones. This module is one of the core components of the system, employing advanced deep learning technology to achieve pixel-level segmentation, enabling accurate identification and labeling of different skeletal structures. The output of the segmentation module is a color-coded segmented image, with different bones identified by different colors, which serves as the input to the localization module.

[0190] The YOLOv8 anatomical point localization module identifies key anatomical points required for foot arch assessment based on segmented foot bone images. This module is another core component of the system, employing an improved YOLOv8 architecture to achieve high-precision point localization, accurately identifying all key points needed for foot arch assessment. The output of the localization module is the spatial coordinates and confidence values ​​of the key anatomical points, which serve as input to the angle calculation module.

[0191] The angle calculation module calculates relevant arch angle data based on the spatial relationships of key anatomical points. This module achieves accurate calculation of the medial longitudinal arch angle, lateral longitudinal arch angle, anterior arch angle, and posterior arch angle, providing quantitative indicators for assessing the degree of arch damage. The output of the angle calculation module is a set of angle data, which serves as input to the rating and determination module.

[0192] The rating and determination module, based on foot arch angle data and combined with the forensic medical evaluation system, determines the degree of foot arch damage. This module compares angle data with normal values, calculates deviations, and classifies grades, providing objective evidence for forensic identification. The output of the rating and determination module is the degree of foot arch damage and related clinical interpretation, which serves as input to the results display module.

[0193] The results display module generates an evaluation report that includes the original image, labeled points, and calculation results. This module provides an intuitive visualization interface that centrally displays the evaluation process and conclusions, and supports report export for easy archiving and sharing.

[0194] These modules are sequentially connected through standardized data interfaces to form a complete processing flow. Data transfer between modules uses memory mapping, avoiding frequent disk I / O operations and improving system efficiency. Simultaneously, the system is designed with an exception handling mechanism to detect and handle potential anomalies at each stage, ensuring system stability and reliability.

[0195] In practical applications, the system processes a single image in about 2.5 seconds on a regular computer with an Intel Core i7 processor and 16GB of memory, which is about 100 times faster than traditional manual methods, while maintaining higher measurement accuracy, significantly improving the efficiency and quality of forensic identification work.

[0196] The intelligent assessment method and system for the degree of arch damage based on multi-model fusion of the present invention has significant industrial applicability, mainly reflected in the following aspects:

[0197] 1. Wide range of applications: Applicable to multiple fields such as forensic identification, medical diagnosis, and insurance claims, with broad application prospects;

[0198] 2. Mature technology: It adopts mature deep learning technology and standardized software architecture, which has good stability and reliability;

[0199] 3. Significant cost-effectiveness: Automated processing greatly improves work efficiency, reduces labor costs, and has obvious economic benefits;

[0200] 4. Moderate deployment requirements: The system can run on ordinary computers without special hardware, making it easy to promote and apply;

[0201] 5. Continuous optimization capability: Through data accumulation and model updates, the system performance can be continuously improved, which has long-term application value.

[0202] Based on the above characteristics, the present invention possesses the basic conditions for industrial application and can provide practical technical support for forensic identification and medical diagnosis.

[0203] This invention provides an intelligent assessment method and system for the degree of arch damage based on multi-model fusion. By integrating the DeepLabv3+ image segmentation model and the YOLOv8 anatomical point localization model, it achieves automatic analysis of foot X-ray images and objective assessment of the degree of arch damage. The system can automatically process weight-bearing lateral X-ray films of the foot, accurately identify key anatomical points, calculate relevant arch angle data, and provide a determination of the degree of arch damage, providing objective, efficient, and accurate technical support for forensic identification.

[0204] Compared with traditional manual measurement methods, this invention significantly improves assessment efficiency and accuracy, reduces subjective errors, and establishes a standardized assessment system, demonstrating significant practical value and promising prospects for wider application. Through this invention, the assessment of the degree of arch damage will be more objective, efficient, and accurate, providing reliable reference data for forensic identification and rating.

[0205] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. An intelligent evaluation method for the degree of arch destruction based on multi-model fusion, characterized in that, The method comprises the following steps: Obtain the foot weight-bearing X-ray lateral film uploaded by the user; Process the foot weight-bearing X-ray lateral film using a DeepLabv3+ image segmentation model to obtain a foot bone segmentation image; Based on the foot bone segmentation image, identify the key anatomical points required for arch evaluation using a YOLOv8 anatomical point positioning model; Calculate the arch-related angle data based on the spatial positional relationship of the key anatomical points; Determine the arch damage degree level based on the arch-related angle data and the forensic judicial appraisal system; Generate an evaluation report containing the original image, labeled points, and calculation results.

2. The method of claim 1, wherein, After obtaining the foot weight-bearing X-ray lateral film uploaded by the user, the method further comprises the following steps: Image preprocessing is performed on the foot weight-bearing X-ray lateral film, including image denoising, contrast enhancement, and size standardization to improve image quality; The image denoising uses a Gaussian filter algorithm, and the contrast enhancement uses a histogram equalization technique.

3. The method of claim 1, wherein, The processing of the foot weight-bearing X-ray lateral film using a DeepLabv3+ image segmentation model specifically includes the following steps: Extract image features based on the DeepLabv3+ architecture of the Xception backbone network; Obtain multi-scale context information through the empty space pyramid pooling module; Generate pixel-level classification results of foot bones; Refine the segmentation boundaries to obtain segmentation images of different bone structures.

4. The method of claim 1, wherein, The identification of key anatomical points required for arch evaluation using a YOLOv8 anatomical point positioning model specifically includes the following steps: Receive the foot bone segmentation image as input; Extract multi-scale features through the CSPDarknet backbone network and feature pyramid structure; Locate key anatomical landmark points including the lowest point of the calcaneus, the highest point of the talonavicular joint, and the lowest point of the first metatarsal head; Generate spatial coordinates and confidence values of the key anatomical points.

5. The method of claim 4, wherein, After locating the key anatomical landmark points including the lowest point of the calcaneus, the highest point of the talonavicular joint, and the lowest point of the first metatarsal head, the method further comprises the following steps: Verify the effectiveness of the detected key points based on anatomical relationships; Reposition or interpolate and correct key points with confidence values below a preset threshold; Automatically correct abnormal positioning points to ensure accurate positioning of key anatomical points.

6. The method of claim 1, wherein, The calculation of arch-related angle data specifically includes the following steps: Calculate the medial longitudinal arch angle based on the lowest point of the calcaneus, the highest point of the talonavicular joint, and the lowest point of the first metatarsal head; Calculate the lateral longitudinal arch angle based on the lowest point of the calcaneus, the cuboid joint, and the lowest point of the fifth metatarsal head; Calculate the forefoot arch angle based on the arc formed by the lowest points of the metatarsal heads; Calculate the rear arch angle based on the angle formed by the calcaneus and the posterior talus.

7. The method of claim 1, wherein, The determination of the arch damage degree level specifically includes the following steps: Compare the calculated angle data with the normal value range; According to the degree of angle deviation, divide the arch damage degree into mild, moderate, and severe: When the angle deviation is less than 15°, it is determined as mild damage; When the angle deviation is between 15° and 30°, it is determined as moderate damage; When the angle deviation is greater than 30°, it is determined as severe damage.

8. The method of claim 1, wherein, The generation of the evaluation report containing the original image, labeled points, and calculation results includes the following steps: Integrate the original foot weight-bearing X-ray lateral film, the segmented image, and the labeled key anatomical points into the same interface; Display the specific values of the medial longitudinal arch angle, the lateral longitudinal arch angle, the anterior arch angle, and the posterior arch angle; Generate the rating conclusion of the foot arch damage degree and its corresponding clinical significance interpretation.

9. The method according to any one of claims 1 to 8, characterized in that, Also includes: Accuracy verification of system evaluation results, compare with expert manual measurement results, calculate system measurement error; When the fluctuation of repeated measurement results of the same image is less than 3°, it is determined that the system is stable and reliable; Store evaluation data in the historical database for subsequent model optimization and performance improvement.

10. An intelligent evaluation system for arch breakage degree based on multi-model fusion, for implementing the method of any one of claims 1-9, characterized in that, Includes: Data input module, for obtaining the foot weight-bearing X-ray lateral film uploaded by the user; Image preprocessing module, for denoising, enhancing and standardizing the foot weight-bearing X-ray lateral film; DeepLabv3+ image segmentation module, for processing the preprocessed foot weight-bearing X-ray lateral film to obtain the foot bone segmentation image; YOLOv8 anatomical point positioning module, for identifying the key anatomical points required for foot arch evaluation based on the foot bone segmentation image; Angle calculation module, for calculating the foot arch related angle data according to the spatial position relationship of the key anatomical points; Rating determination module, for determining the foot arch damage degree rating based on the foot arch related angle data and combining the forensic judicial appraisal evaluation system; Result display module, for generating the evaluation report containing the original image, labeled points and calculation results; The modules are sequentially connected through data flow interfaces to form a complete foot arch damage degree evaluation process.

Images