Vision-based insole stitch anomaly detection method and system

Abstract

The application provides a visual-based insole stitch anomaly detection method and system, and relates to the field of quality detection. The method comprises the following steps: obtaining an original image of an insole to be detected, performing denoising and light balance processing to obtain a standardized detection image. Feature fusion processing is performed to generate a detection fusion feature tensor. The standardized image of a normal insole sample is obtained, the normal fusion feature tensor is generated through feature fusion processing, and a normal feature distribution model library is constructed. The detection fusion feature tensor is compared and calculated with the normal feature distribution model library to generate an anomaly heat map. The detection image is divided into detection regions to generate regional anomaly statistical data. According to the regional anomaly statistical data and a preset global judgment threshold, it is judged whether the insole to be detected is a qualified product, and a detection report is output. The technical problems of hidden nature, insufficient recognition ability of microscopic defects and lack of stability of artificial quality inspection are solved.

Description

Technical Field

[0001] This application relates to the field of quality inspection, and in particular to a vision-based method and system for detecting abnormal stitches in insoles. Background Technology

[0002] In the large-scale production process of insoles, manual sewing is the core step in stitch processing. Influenced by multiple factors such as operator skill, working hours, visual fatigue, and changes in ambient lighting, stitch defects such as missed stitches, off-center stitches, dirt spots, and broken threads are frequent problems in production quality control. These defects are often characterized by their high degree of concealment (e.g., minor missed stitches, slight off-center stitches at the edges), random distribution (appearing in different areas of the insole such as the forefoot, arch, and heel), and diverse types, becoming key factors affecting the quality of finished insoles. Currently, the industry generally relies on traditional manual visual inspection for detecting stitch defects in insoles. Quality inspectors must visually screen each insole, judging the presence of defects by recognizing the arrangement, density, color, and integrity of the stitches.

[0003] Quality inspectors are prone to visual fatigue after prolonged continuous work, significantly reducing their sensitivity to minor defects. Furthermore, individual differences in visual discrimination ability and understanding of defect judgment standards among inspectors lead to inconsistencies in the test results for the same batch and type of defects, resulting in "different judgments for the same product." Manual inspection is also highly dependent on ambient lighting conditions; excessively strong, dim, or uneven lighting in the workshop can easily lead to missed or misjudged defects, ultimately resulting in a lack of unified and stable judgment standards for stitch inspection results, hindering the formation of standardized quality inspection results. Secondly, in areas with high stitch density (such as the forefoot pressure area of ​​insoles and edge stitching areas), it is difficult for the naked eye to accurately distinguish minute missed stitches and broken threads, and it is also impossible to quantify stitch color deviations (such as localized color abnormalities caused by dirt). Only obvious, macroscopic stitch defects can be identified, causing a large number of hidden defects to flow into subsequent processes or the final product stage, creating potential product quality risks. Subsequent customer complaints or rework actually increase production remediation costs. Summary of the Invention

[0004] This application provides a vision-based method and system for detecting abnormal stitches in insoles, which solves the technical problems of insufficient concealment and microscopic defect identification capabilities of existing manual quality inspection methods, as well as the lack of stability.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, a vision-based method for detecting abnormal insole stitching includes: acquiring the original image of the insole to be detected, performing denoising and illumination equalization processing to obtain a standardized detection image; performing feature fusion processing on the detection image to generate a detection fusion feature tensor, which includes extracting feature maps of the detection image at multiple different depth levels through a deep convolutional neural network, performing spatial scale alignment and feature channel stitching, with each spatial location in the detection fusion feature tensor corresponding to a feature vector; acquiring standardized images of normal insole samples, generating a normal fusion feature tensor through feature fusion processing, and constructing a normal feature distribution model library; comparing the detection fusion feature tensor with the normal feature distribution model library to generate an anomaly heatmap; dividing the detection image into detection regions based on the physical structural characteristics of the insole, and statistically analyzing the distribution of abnormal pixels within each detection region using the anomaly heatmap to generate regional anomaly statistics; and determining whether the insole to be detected is a qualified product based on the regional anomaly statistics and a preset global judgment threshold, and outputting a detection result image containing anomaly location markers and a structured detection report.

[0006] In conjunction with the first aspect mentioned above, in one possible implementation, the deep convolutional neural network is a ResNet18 residual network. Forward propagation is performed on sequentially arranged first, second, and third residual block groups, corresponding to the extraction of first, second, and third feature maps with decreasing resolution and increasing channel number. The first feature map represents the texture details of the pin. The second feature map represents the structural semantic information of the pin. The third feature map represents the global abstract semantic information of the pin.

[0007] In conjunction with the first aspect mentioned above, in one possible implementation, the process of generating a detection fusion feature tensor by performing feature fusion processing on the detected image specifically includes: inputting the detected image into a ResNet18 residual network, and extracting a first feature map, a second feature map, and a third feature map through a first residual block group, a second residual block group, and a third residual block group, respectively. Performing a first upsampling operation on the third feature map to generate a first upsampled feature map. Concatenating the first upsampled feature map and the second feature map using a first feature channel to generate a first intermediate fusion feature tensor. Performing a second upsampling operation on the first intermediate fusion feature tensor to generate a second upsampled feature map. Concatenating the second upsampled feature map and the first feature map using a second feature channel to generate a second intermediate fusion feature tensor. Performing cross-channel information interaction and feature dimensionality reduction on the second intermediate fusion feature tensor through convolutional layers to generate a detection fusion feature tensor. The spatial location in the detection fusion feature tensor corresponds to a multi-dimensional feature vector, and the multi-dimensional feature vector includes texture details and structural semantics.

[0008] In conjunction with the first aspect mentioned above, in one possible implementation, the construction process of the normal feature distribution model library specifically includes: extracting fixed spatial locations within the normal fusion feature tensor, summarizing the channel number dimension feature vectors of all normal samples at the fixed spatial locations, and constructing a set of normal feature vectors for the spatial locations. The normal feature vector set is then fitted with a multivariate Gaussian distribution, and the standard mean vector and standard covariance matrix of all normal feature vectors are calculated. Finally, the standard mean vector and standard covariance matrix are indexed and stored according to their corresponding fixed spatial locations to obtain the normal feature distribution model library.

[0009] In conjunction with the first aspect mentioned above, one possible implementation involves comparing the detection fusion feature tensor with a normal feature distribution model library to generate an anomaly heatmap. Specifically, this includes: receiving the detection fusion feature tensor and extracting the standard mean vector from the normal feature distribution model library; calculating the detection mean fusion value vector of the detection fusion feature tensor and the deviation vector between the detection mean fusion value vector and the standard mean vector; extracting the standard covariance matrix from the normal feature distribution model library; calculating the Mahalanobis distance based on the deviation vector and the standard covariance matrix; traversing all spatial locations of the detection fusion feature tensor or the Mahalanobis distance at each location to construct an initial anomaly matrix; and performing adaptive smoothing processing on the initial anomaly matrix based on the insole material characteristics to generate the anomaly heatmap.

[0010] In conjunction with the first aspect mentioned above, in one possible implementation, the process of dividing the detection image into detection regions based on the physical structural characteristics of the insole specifically includes: acquiring the global spatial coordinate matrix of the insole image; dividing the insole into physiological regions based on the biomechanical characteristics of the human foot, including the forefoot pressure zone, arch support zone, heel buffer zone, and edge stitching zone; and dividing each physiological region into process sub-regions based on the stitch density distribution characteristics of the insole, including high-density stitch sub-regions and low-density stitch sub-regions.

[0011] In conjunction with the first aspect mentioned above, in one possible implementation, the process of statistically analyzing the distribution of abnormal pixels within each detection region and generating regional anomaly statistics specifically includes: configuring an anomaly sensitivity threshold set for each process sub-region, and generating a sub-region mask image for each process sub-region based on its spatial range. The sub-region mask image has the same spatial size as the anomaly heatmap, and pixels belonging to the process sub-region are assigned a value of 1, while pixels not belonging to the process sub-region are assigned a value of 0. A pixel-by-pixel multiplication operation is performed between the anomaly heatmap and the sub-region mask image to extract a subset of abnormal heatmap values ​​for each process sub-region, and the total number and percentage of abnormal pixels with heatmap values ​​exceeding the anomaly sensitivity threshold within each process sub-region are calculated. Connectivity analysis is performed on the abnormal pixels in the process sub-regions, the number of abnormal connected components and the maximum connected component area within each process sub-region are counted, and the spatial distribution entropy value of the abnormal pixels within the sub-region is calculated. The percentage of abnormal pixels, the number of abnormal connected components, the maximum connected component area, and the spatial distribution entropy value of the process sub-regions are combined to generate a regional anomaly feature vector. The regional anomaly feature vectors of all process sub-regions under the same physiological region are weighted and fused to generate a comprehensive anomaly assessment index, which is then encapsulated into regional anomaly statistical data in sync with the regional anomaly feature vectors.

[0012] In conjunction with the first aspect mentioned above, in one possible implementation, the process of determining whether the insole to be tested is a qualified product based on regional anomaly statistics and a preset global judgment threshold specifically includes: configuring a sub-region judgment threshold for each process sub-region and a regional judgment threshold for each physiological region. Based on the sub-region judgment threshold and the regional judgment threshold, a first-level local judgment is performed on the regional anomaly feature vector and the comprehensive anomaly evaluation index. The first-level local judgment includes comparing each regional anomaly feature vector with the corresponding sub-region judgment threshold and comparing each comprehensive anomaly evaluation index with the corresponding regional judgment threshold, filtering data where neither the process sub-region nor the physiological region exceeds its corresponding judgment threshold. After satisfying the first-level local judgment, a global anomaly score is calculated based on the comprehensive anomaly evaluation index using a preset weight coefficient for the physiological region. The global anomaly score is compared with the preset global judgment threshold to determine whether the insole to be tested is a qualified product.

[0013] In conjunction with the first aspect mentioned above, one possible implementation also includes a parallel-deployed auxiliary detection channel. This auxiliary detection channel performs pin anomaly detection on the detection image based on a color matrix. Specifically, it includes: dividing the detection image into a grid, generating an M-row × N-column grid cell set, where each grid cell corresponds to a local pin region in the original image, and M and N are positive integers; extracting the color feature matrix of each grid cell in a preset color space, the color feature matrix including at least the mean brightness, mean color channel value, and standard deviation of all pixels within the grid cell; selecting K% of the grid cells, and removing the grid cells with the highest and lowest brightness, and statistically analyzing the color distribution baseline interval for normal pins, the color distribution baseline interval including at least the brightness baseline range and the baseline range of each color channel, where K is a preset positive integer; comparing the color feature matrix with the color distribution baseline interval item by item to identify abnormal grid cells; visually annotating the abnormal grid cells and statistically analyzing the total number and distribution of abnormal grid cells to generate an auxiliary detection statistical report; and fusing the auxiliary detection statistical report with the detection report to output a comprehensive quality inspection result.

[0014] Secondly, a vision-based insole stitch anomaly detection system is provided, comprising: a data acquisition module for acquiring the original image of the insole to be detected, performing denoising and illumination equalization processing to obtain a standardized detection image; a feature tensor generation module for performing feature fusion processing on the detection image to generate a detection fusion feature tensor, which includes extracting feature maps of the detection image at multiple different depth levels through a deep convolutional neural network, performing spatial scale alignment and feature channel stitching, with each spatial location in the detection fusion feature tensor corresponding to a feature vector; a standard feature construction module for acquiring standardized images of normal insole samples, generating a normal fusion feature tensor through feature fusion processing, and constructing a normal feature distribution model library; a comparison calculation module for comparing the detection fusion feature tensor with the normal feature distribution model library to generate an anomaly heatmap; and an anomaly detection module for dividing the detection image into detection regions based on the physical structural characteristics of the insole, combining the anomaly heatmap to statistically analyze the distribution of abnormal pixels within each detection region, and generating regional anomaly statistics. The judgment and analysis module is used to determine whether the insole to be tested is a qualified product based on the regional anomaly statistics and the preset global judgment threshold, and outputs a detection result image containing anomaly location markers and a structured detection report.

[0015] This application provides a vision-based method and system for detecting stitch anomalies in insoles. By denoising and equalizing the original image, it eliminates interference from uneven lighting and image noise in the production environment, generating standardized detection images. This avoids the problems of manual detection being affected by lighting and visual discrimination being easily interfered with, establishing a unified and stable image foundation for subsequent detection and solving the underlying problem of detection stability. Simultaneously, a deep convolutional neural network is used to extract multi-level feature maps and complete scale alignment and channel stitching to generate a detection fusion feature tensor that integrates texture details and deep semantics. This enables accurate extraction of microscopic features of the stitches, overcoming the limitation of the human eye being unable to identify minute stitch defects and solving the core problem of insufficient microscopic defect recognition capability. Furthermore, a normal feature distribution model library is constructed, and anomaly heatmaps are generated through Mahalanobis distance comparison to achieve accurate localization of hidden defects, making microscopic anomalies visible. The detection area is divided based on the physical structure of the insole, and the distribution of abnormal pixels is statistically analyzed to form standardized regional anomaly statistics, replacing subjective manual judgment. Finally, standardized test results and reports are output through multi-level threshold judgment, realizing the standardization and quantification of test judgment, and completely solving the problem of lack of judgment stability caused by individual differences and visual fatigue in manual inspection. The entire process uses standardized and intelligent technical means to achieve stable and accurate detection of the concealment of pins and micro-defects.

[0016] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0017] Figure 1 A schematic flowchart illustrating the vision-based insole stitch anomaly detection method provided in this application embodiment; Figure 2 A flowchart illustrating the step of generating a detection fusion feature tensor by performing feature fusion processing on the detected image in the vision-based insole stitch anomaly detection method provided in this application embodiment; Figure 3 A flowchart illustrating the steps of constructing a normal feature distribution model library in the vision-based insole stitch anomaly detection method provided in this application embodiment; Figure 4 The flowchart of the step of generating an anomaly heatmap by comparing the detection fusion feature tensor with the normal feature distribution model library in the vision-based insole stitch anomaly detection method provided in the embodiments of this application is shown. Figure 5 This is a flowchart illustrating the step of generating regional anomaly statistics by statistically analyzing the distribution of abnormal pixels in each detection area in the vision-based insole stitch anomaly detection method provided in this application embodiment. Figure 6 This is a schematic diagram of the structure of a vision-based insole stitch anomaly detection system provided in an embodiment of this application. Detailed Implementation

[0018] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0019] like Figure 1 As shown in the embodiments of this application, the vision-based insole stitch anomaly detection method includes: Step 101: Obtain the original image of the insole to be tested, perform noise reduction and illumination equalization processing to obtain a standardized test image.

[0020] In some implementations, the image acquisition module directly calls the camera to capture the original image of the insole to be detected, and automatically detects the main area of ​​the insole in the image and crops it to remove background interference.

[0021] After acquiring the original image, it can be directly scaled to a preset uniform size, which matches the number of rows and columns of the subsequent grid division, ensuring that each grid corresponds to a fixed physical area.

[0022] After size normalization is completed, Gaussian filtering algorithm is applied to the image for noise reduction. By calculating the weighted average of each pixel and its neighboring pixels, random noise points in the image are eliminated while preserving the edge details of the pins.

[0023] Finally, the denoised image undergoes illumination equalization processing. A histogram equalization algorithm is used to redistribute the image's brightness values, enhancing the detail information in areas that were originally too dark or too bright, resulting in a standardized detection image with uniform brightness distribution. This standardized detection image serves as input data for subsequent grid division and color feature extraction. Its size consistency ensures the stability of grid segmentation, while its noise reduction and brightness equalization characteristics lay the foundation for accurately calculating the mean color and brightness values ​​of each grid.

[0024] Taking actual production line inspection as an example, a quality inspector photographs a white insole. Due to the warm yellow lighting in the workshop, the original image appears yellowish overall, and there is a shadow on the heel area of ​​the insole. The image acquisition module automatically crops the main body of the insole and first scales the image to a preset size of 640×320 pixels. Then, a 5×5 Gaussian filter is applied for noise reduction, eliminating small particles caused by noise from the mobile phone sensor. Finally, histogram equalization is performed to correct the warm yellow tone to a white closer to natural light, and the stitch texture in the shadowed heel area becomes clearer. This transforms the unevenly lit original image into a standardized inspection image with uniform brightness and noise reduction, providing reliable input data for accurately determining whether there are missing stitch defects in that area.

[0025] Step 102: Perform feature fusion processing on the detection image to generate a detection fusion feature tensor. The feature fusion processing includes extracting feature maps of the detection image at multiple different depth levels through a deep convolutional neural network, performing spatial scale alignment and feature channel stitching. Each spatial position in the detection fusion feature tensor corresponds to a feature vector.

[0026] In some implementations, the standardized detection image is first input into a pre-trained deep convolutional neural network. When the image propagates forward in the deep convolutional neural network, feature maps of three different depth levels are extracted from the outputs of the first residual block group, the second residual block group, and the third residual block group, respectively. The shallow feature maps have higher resolution but fewer channels, preserving the edge and texture details of the pins, while the deep feature maps have lower resolution but more channels, encoding the structure and arrangement semantics of the pins.

[0027] Since these three feature maps have different spatial dimensions, the deep feature map can be upsampled to match the spatial resolution of the shallow feature map, thus achieving spatial scale alignment. Subsequently, the aligned three feature maps are concatenated along the channel dimension, merging the shallow texture features and deep semantic features into a unified feature tensor. At this point, the detection fusion feature tensor has the same spatial resolution as the shallow feature map, with each spatial location corresponding to a local region in the original image. The feature vector at that location is composed of feature values ​​from three levels, fusing complete visual information from detail to structure, for subsequent anomaly distribution comparison.

[0028] Step 103: Obtain standardized images of normal insole samples, generate normal fusion feature tensors through feature fusion processing, and construct a normal feature distribution model library.

[0029] Among them, a normal insole sample refers to an image of a standard, qualified insole that has been manually inspected and confirmed to be free of any stitch defects. A normal fusion feature tensor refers to a multi-level comprehensive feature representation generated by fusing features from the standardized image of a normal insole sample using a deep convolutional neural network. A normal feature distribution model library refers to a set of parameterized models describing the distribution patterns of normal stitch features, established through statistical analysis of the fusion feature tensors of a large number of normal insole samples.

[0030] In some implementations, a set of normal insole sample images (typically dozens to hundreds of images to cover various variations under normal conditions) that have been manually inspected and confirmed to be qualified is collected. Then, normalization preprocessing and feature fusion processing are performed on each normal insole sample image in turn to generate the corresponding normal fusion feature tensor.

[0031] Since the feature fusion tensor of all samples has the same spatial size and number of channels, it is possible to collect the feature vectors of all samples at each spatial location, forming a feature vector set.

[0032] At this point, for each spatial location, the arithmetic mean of all feature vectors in the set is calculated to obtain the standard mean vector for that location; simultaneously, the covariance between each dimension of all feature vectors in the set is calculated to obtain the standard covariance matrix for that location. The standard mean vector and the standard covariance matrix together constitute the normal distribution model for that location, where the standard mean vector defines the baseline value of the normal features, and the standard covariance matrix defines the acceptable range of variation for the normal features and the correlation structure between the various feature dimensions.

[0033] This allows for the combination of the standard mean vector and standard covariance matrix of all spatial locations to form a complete normal feature distribution model library. This model library will comprehensively describe the expected feature performance and inherent variation law of normal insoles in different spatial locations, providing a comparison benchmark for subsequent anomaly detection.

[0034] Step 104: Compare and calculate the detection fusion feature tensor with the normal feature distribution model library to generate an anomaly heatmap.

[0035] The comparison calculation refers to the process of measuring the similarity or difference between the feature vector of the image to be detected and the distribution parameters at the corresponding positions in the normal feature distribution model library. The anomaly heatmap is a two-dimensional grayscale image generated through comparison calculation that corresponds to the spatial position of the original image. The value of each pixel represents the degree to which the feature vector at that position deviates from the normal distribution model; the higher the value, the greater the probability of an anomaly.

[0036] In some implementations, the detection fusion feature tensor of the insole to be detected is first obtained. At this time, each spatial location of the tensor contains a multi-dimensional feature vector, which can simultaneously read the standard mean vector and standard covariance matrix of the same spatial location from the normal feature distribution model library.

[0037] For each spatial location, the difference between the feature vector to be detected and the standard mean vector at that location is calculated to obtain the deviation vector, which reflects the absolute difference between the feature to be detected and the normal benchmark value in each dimension.

[0038] Then, the inverse covariance matrix at that location is read (pre-calculated and stored during the model building phase to measure the correlation structure between the dimensions of the features). The transpose of the deviation vector is then multiplied by the inverse covariance matrix, and then multiplied again by the deviation vector to obtain the Mahalanobis distance at that location. This Mahalanobis distance value contains both information about the magnitude of the deviation vector and the correlation information between the feature dimensions; the greater the deviation of the detected feature vector from the normal distribution, the larger the Mahalanobis distance value.

[0039] The above calculation process is repeated for all spatial locations to obtain a two-dimensional numerical matrix with the same size as the feature fusion tensor space. Each value in the matrix corresponds to the degree of anomaly in a local region of the original image. This two-dimensional numerical matrix is ​​normalized to a grayscale range of 0 to 255 and then upsampled to the original image size to generate the final anomaly heatmap, where areas with higher brightness indicate a more severe anomaly.

[0040] Step 105: Based on the physical structure characteristics of the insole, divide the detection image into detection areas, combine with the abnormal heat map, and statistically analyze the distribution of abnormal pixels in each detection area to generate regional abnormal statistics.

[0041] The process of dividing the detection image into detection regions based on the physical structural characteristics of the insole specifically includes: acquiring the global spatial coordinate matrix of the insole image; dividing the insole into physiological regions based on the biomechanical characteristics of the human foot, including the forefoot pressure zone, arch support zone, heel buffer zone, and edge stitching zone; and further dividing each physiological region into process sub-regions based on the stitch density distribution characteristics of the insole, including high-density stitch sub-regions and low-density stitch sub-regions.

[0042] The detection area segmentation refers to the process of dividing the image space into sub-regions with different functional attributes and technological requirements based on the physical structural characteristics of the insole. The forefoot pressure zone refers to the area at the front of the insole corresponding to the metatarsal region of the human body; this area bears the main pressure during walking. The arch support zone refers to the area in the middle of the insole corresponding to the arch region of the human foot; this area mainly provides support. The heel buffer zone refers to the area at the rear of the insole corresponding to the calcaneus region of the human body; this area bears the greatest impact force. The edge stitching zone refers to the outer perimeter edge area of ​​the insole, where the insole is sewn to the sole. High-density stitching sub-regions refer to local areas within the same physiological region where the stitch density is higher than a preset density threshold, typically corresponding to areas of concentrated stress. Low-density stitching sub-regions refer to local areas within the same physiological region where the stitch density is lower than or equal to a preset density threshold, typically corresponding to non-stressed areas.

[0043] In some implementations, the detection image of the insole to be detected and its corresponding anomaly heatmap are first obtained. The anomaly heatmap has the same spatial size as the original image, and the gray value of each pixel represents the degree of anomaly at that location.

[0044] The detection area can be divided according to the preset physical structure mapping table of the insole. This mapping table divides the insole image into four primary physiological areas based on the biomechanical characteristics of the human foot: the forefoot pressure area, the arch support area, the heel buffer area, and the edge stitching area. Each primary area has a clear coordinate range definition.

[0045] Further reading of the stitch density distribution map in the production process document, which was obtained by offline analysis of the stitch positions of a large number of normal insole samples, identifies the local locations in each primary region where the stitch density is higher than a preset threshold. Based on this, each primary physiological region is subdivided into high-density stitch sub-regions and low-density stitch sub-regions, forming eight secondary process sub-regions.

[0046] Once the regions are divided, a binarized region mask image can be generated for each secondary process sub-region. The abnormal heat map is then processed with the mask image of each sub-region to extract a subset of abnormal heat values. For each sub-region, the number of pixels whose heat values ​​exceed a preset anomaly detection threshold is counted as the total number of abnormal pixels, and the proportion of this number to the total number of pixels in the sub-region is calculated as the percentage of abnormal pixels.

[0047] Simultaneously, eight-neighbor connectivity analysis is performed on abnormal pixels within each sub-region to identify abnormal connected regions formed by adjacent abnormal pixels, and the number and maximum area of ​​abnormal connected regions within each sub-region are counted. The percentage of abnormal pixels, the number of abnormal connected regions, and the maximum area of ​​the maximum connected region in each sub-region are combined to form a multi-dimensional abnormal feature for that sub-region. Finally, by merging and aggregating the abnormal features of each sub-region under the same primary physiological region, a comprehensive abnormality index for each primary region is generated, and the statistical results of all primary regions and secondary sub-regions are encapsulated into a structured JSON format regional abnormality statistics package.

[0048] Step 106: Based on the regional anomaly statistics and the preset global judgment threshold, determine whether the insole to be tested is a qualified product, and output the detection result image containing anomaly location markers and a structured detection report.

[0049] The global judgment threshold refers to a pre-set critical value for comprehensively judging the overall degree of abnormality of the entire insole based on historical quality inspection data and process quality standards. A qualified product refers to an insole where, after comprehensive judgment, all abnormal indicators in all tested areas do not exceed the corresponding standards and the overall degree of abnormality is lower than the global judgment threshold. Abnormal location marking refers to the process of visually marking areas judged as abnormal on the original inspection image using specific colors or symbols.

[0050] The process of determining whether an insole to be tested is a qualified product includes: configuring a sub-region judgment threshold for each process sub-region and a region judgment threshold for each physiological region; based on the sub-region judgment threshold and the region judgment threshold, performing a first-level local judgment on the regional abnormal feature vector and the comprehensive abnormality evaluation index. The first-level local judgment includes comparing each regional abnormal feature vector with the corresponding sub-region judgment threshold and comparing each comprehensive abnormality evaluation index with the corresponding region judgment threshold, and filtering data where neither the process sub-region nor the physiological region exceeds its corresponding judgment threshold; after satisfying the first-level local judgment, calculating a global abnormality score based on the comprehensive abnormality evaluation index using a preset weight coefficient for the physiological region; comparing the global abnormality score with a preset global judgment threshold to determine whether the insole to be tested is a qualified product.

[0051] In some implementations, the regional anomaly statistics data package is obtained, and a pre-configured global judgment threshold table is read. The first level of local judgment can be performed: the proportion of abnormal pixels in the comprehensive abnormality assessment index of each physiological region is compared with the region alarm threshold. If the proportion of abnormal pixels in any region exceeds its region alarm threshold, the insole is immediately judged as a defective product, and the over-standard area identifier and the over-standard index value are recorded.

[0052] If the comprehensive abnormality assessment index of all physiological regions does not exceed the alarm threshold of their respective regions (i.e., after satisfying the first level of local judgment), then the second level of global judgment is executed: calculate the weighted average of the proportion of abnormal pixels in all physiological regions as the overall abnormality score. The weight coefficient is preset according to the area proportion of each region. Compare the overall abnormality score with the global judgment threshold. If it exceeds the global judgment threshold, it is judged as a non-conforming product; otherwise, it is judged as a qualified product.

[0053] After the judgment is completed, based on the coordinate information of abnormal pixels in the regional anomaly statistics package, the abnormal pixels are rendered on the original detection image using preset colors. For example, pixels with abnormal thermal values ​​exceeding the anomaly judgment threshold are highlighted in red, and abnormal connected regions are outlined with red contour lines. The final judgment result text is then overlaid on the top of the image. Simultaneously, the detection time, image file name, four-dimensional anomaly feature vector of the process sub-region, comprehensive anomaly evaluation index of the physiological region, local judgment result, global judgment result, and final pass / fail judgment conclusion are encapsulated into a structured inspection report in JSON format. This report is output to a specified storage path and displayed in real time on the quality inspection terminal interface for quality inspectors to view and confirm.

[0054] The global judgment threshold table was established through offline analysis of a large amount of historical quality inspection data. It includes the regional alarm threshold for each primary physiological area and the global judgment threshold for the entire insole. The thresholds for different areas are set differently according to their physiological function importance. For example, the regional alarm threshold for the forefoot pressure area is set to 60, the regional alarm threshold for the edge suture area is set to 50, and the global judgment threshold is set to 55.

[0055] In another possible implementation, a parallel auxiliary detection channel is also included. This auxiliary detection channel is used to perform pin anomaly detection on the detection image based on a color matrix. Specifically, this includes: dividing the detection image into a grid, generating a set of M rows × N columns of grid cells, where each grid cell corresponds to a local pin region in the original image, and M and N are positive integers; extracting the color feature matrix of each grid cell in a preset color space, the color feature matrix including at least the mean brightness, mean color channel, and standard deviation of all pixels within the grid cell; selecting K% of the grid cells, and removing the grid cells with the highest and lowest brightness, and statistically analyzing the color distribution baseline interval for normal pins, the color distribution baseline interval including at least the brightness baseline range and the baseline range of each color channel, where K is a preset positive integer; comparing the color feature matrix with the color distribution baseline interval item by item to identify abnormal grid cells; visually annotating the abnormal grid cells, and statistically analyzing the total number and distribution of abnormal grid cells to generate an auxiliary detection statistical report; and fusing the auxiliary detection statistical report with the detection report to output a comprehensive quality inspection result.

[0056] The auxiliary detection channel refers to a detection process based on traditional image processing techniques that runs in parallel with the main detection channel (feature fusion and anomaly detection based on deep learning). During its operation, it primarily utilizes color matrix analysis algorithms and can serve as a supplement or backup to the main detection channel, suitable for the rapid detection of color deviation defects (such as dirt spots and color differences).

[0057] In some implementations, the detection image can be uniformly divided into a set of grid cells with M rows and N columns. For example, if M=20 and N=40, 800 grid cells of the same size are generated, and each grid cell corresponds precisely to a local pin area of ​​a fixed size in the original image.

[0058] In short, for each grid cell, all pixels within it are traversed, and the color feature matrix of that grid cell is calculated in a preset color space (such as RGB color space or Lab color space which is more in line with human eye perception). At this time, the matrix contains at least three core statistics: the mean brightness of all pixels (reflecting the overall brightness of the area), the mean of each color channel (reflecting the color tone of the area), and the standard deviation of each color channel (reflecting the uniformity of color and texture complexity of the area).

[0059] Once the feature extraction of all grid cells is complete, the automatic construction phase of the normal color distribution benchmark begins.

[0060] Assuming that most areas of the insole image to be tested are normal, K% (e.g., 80%) of the grid cells are selected from all grid cells as normal candidate grid cells to eliminate interference from extreme lighting or local dirt. Furthermore, the grid cells with the highest and lowest extreme values ​​of brightness among these candidate cells are removed. The remaining normal candidate grid cells are then statistically analyzed to calculate the overall minimum and maximum values ​​of brightness as the brightness reference range. This allows for the calculation of the overall minimum and maximum values ​​of each color channel as the reference range for each color channel, forming a distribution reference interval describing the color characteristics of normal stitches.

[0061] Then, anomaly detection logic is executed for each grid cell: each feature value (such as the average brightness value, the average R channel value, etc.) in the color feature matrix of the current grid cell is compared item by item with the corresponding color distribution reference interval: If the characteristic value falls within the baseline range, then the item is considered normal; If a feature value exceeds the baseline range, the deviation of the excess portion is calculated and compared with a pre-configured dynamic tolerance threshold for that feature dimension. Only when the feature value exceeds the baseline range and the deviation is greater than the dynamic tolerance threshold is the mesh cell ultimately determined to be an abnormal mesh cell. For example, if a mesh cell has an R-channel mean of 150, while the baseline R-channel range is 100-140, and the excess is 10, and the dynamic tolerance threshold is set to 8, then 10 > 8, and the mesh cell is judged to be abnormal.

[0062] After completing the above judgment for all grid cells, the locations of all grid cells judged as abnormal can be visually marked on the original detection image using preset marker symbols (such as red dots or red rectangles), intuitively displaying the abnormal locations. Simultaneously, the total number of all abnormal grid cells is counted, and based on the physiological region division results, the number and proportion of abnormal grid cells in each physiological region are calculated, generating a structured auxiliary detection statistical report. This report is encapsulated in JSON format and includes information such as the total number of abnormalities, the distribution of abnormalities in each region, and the abnormality type of each grid cell (such as brightness abnormality, color shift abnormality).

[0063] Finally, the auxiliary inspection statistical report generated by the auxiliary inspection channel and the structured inspection report generated by the main inspection channel are merged to generate a comprehensive quality inspection result that includes structural morphological abnormalities (such as missing needles or misaligned needles) and color appearance abnormalities (such as dirt spots or color differences). This result is then output to the quality inspection terminal for quality inspectors to view and confirm.

[0064] Based on the above technical solution, by utilizing deep convolutional neural networks for multi-level feature fusion, a detection fusion feature tensor containing texture details and deep semantics is extracted from the original image. This tensor is then compared with a normal feature distribution model library based on Mahalanobis distance to generate an anomaly heatmap capable of accurately locating microscopic anomalies. This solves the technical challenge of identifying hidden stitch defects using traditional manual visual inspection or simple image processing, elevating defect identification from simply detecting presence to precise location. Simultaneously, the insole is divided into physiological regions based on foot biomechanical characteristics, and further subdivided into process sub-regions with different stitch densities. By generating masks for each sub-region and combining them with the anomaly heatmap for regional statistical analysis, multi-dimensional regional anomaly statistics are generated, including anomaly percentage and connected component count. This overcomes the application bottleneck of a unified detection standard failing to adapt to the functional importance and process complexity of different regions, making the detection process more closely aligned with actual production processes and providing more refined data support for subsequent analysis. Ultimately, a three-tiered progressive judgment logic of sub-region, region, and global was constructed. By accurately capturing subtle defects in key areas through local thresholds, and then conducting overall quality assessment through weighted fusion, the management problems of high misjudgment rate and inability to locate the source of defects by single global thresholds were solved. Thus, while ensuring the quality of products leaving the factory, high-risk process sub-regions can be accurately fed back to the production end, realizing the leap from result judgment to process improvement and improving the level of intelligent production management.

[0065] In another possible implementation of the embodiments of this application, combined with Figure 1-2 As shown, the process of generating a detection fusion feature tensor by performing feature fusion processing on the detection image can be achieved through the following steps 201 to 204, which are explained in detail below: The deep convolutional neural network is a ResNet18 residual network. Forward propagation is performed using three sequentially arranged residual block groups: the first, second, and third residual block groups. These groups extract feature maps with decreasing resolution and increasing channel count, respectively. The first feature map has the highest resolution and fewest channels, used to represent the texture details of the pins; the second feature map has the next highest resolution and a moderate number of channels, used to represent the structural semantic information of the pins; and the third feature map has the lowest resolution and the most channels, used to represent the global abstract semantic information of the pins.

[0066] Step 201: Input the detected image into the ResNet18 residual network, and extract the first feature map, the second feature map, and the third feature map through the first residual block group, the second residual block group, and the third residual block group, respectively.

[0067] The ResNet18 residual network refers to an 18-layer residual neural network model pre-trained on the ImageNet large-scale image dataset. Internally, it solves the gradient vanishing problem in deep networks through residual connections, effectively extracting hierarchical features of images. The first feature map is the feature map output after processing the first residual block group during the forward propagation of the ResNet18. It is characterized by high spatial resolution and few channels, capable of finely representing the texture, edges, and other details of the stitches. The second feature map is the feature map output after processing the second residual block group, with moderate resolution and number of channels, encoding intermediate semantic information such as stitch arrangement and structural morphology. The third feature map is the feature map output after processing the third residual block group, with the lowest resolution and the most channels, containing global abstract semantic information about the stitch region.

[0068] In some implementations, the preprocessed, standardized detection image is input into a ResNet18 residual network pre-loaded with weights pre-trained from the ImageNet dataset. The image then propagates forward layer by layer through the network. When the data flows through the first residual block, the output of that layer can be captured and saved as the first feature map via hook functions or direct access, preserving rich stitch textures and edge details. Subsequently, the data continues to the second residual block, where a second feature map is captured at its output, abstracting the structure and arrangement of the stitches. Finally, a third feature map is captured at the output of the third residual block, encoding the global context and high-level semantics of the stitch region. This hierarchical capture mechanism allows for the simultaneous acquisition of three different levels of feature representation—from concrete to abstract and from local to global—during the same forward propagation process.

[0069] For example, suppose the system acquires an image of an insole with a minor stitch break. When this normalized image is input into a ResNet18, in the first feature map output by the first residual block group, the texture interruption at the break point is highlighted as a sudden change in edge. In the second feature map, the discontinuity in the stitch arrangement at this point is captured. In the third feature map, although the resolution is reduced, the context of the surrounding stitches confirms that the feature vector at this point does not conform to the high-level semantics of "complete stitches" that should be present in normal areas. Through the synergistic effect of these three levels of features, the existence of the stitch break defect can be confirmed from three levels: detail, structure, and semantics.

[0070] Step 202: Perform a first upsampling operation on the third feature map to generate a first upsampled feature map; perform a first feature channel concatenation on the first upsampled feature map and the second feature map to generate a first intermediate fused feature tensor.

[0071] The first upsampling operation refers to the process of expanding the spatial size of the low-resolution third feature map to the same resolution as the second feature map using algorithms such as bilinear interpolation or transposed convolution, thereby generating the first upsampled feature map. The first feature channel concatenation refers to merging two feature maps with the same spatial size in the channel dimension to form a new feature tensor with an increased number of channels, which can integrate feature information from different levels.

[0072] In some implementations, the third feature map output by the third residual block group of the ResNet18 residual network is obtained first. Then, a predefined upsampling module is called, and a bilinear interpolation algorithm is used to calculate the value of each new pixel position according to the preset target size (i.e., the spatial size of the second feature map). The first upsampled feature map is generated by the weighted average of the four surrounding neighboring pixels, so that its spatial resolution is completely consistent with that of the second feature map.

[0073] Subsequently, a new tensor container is created with the number of channels equal to the sum of the number of channels in the first upsampled feature map and the second feature map, while keeping the spatial size unchanged. Then, the channel data of the two feature maps are sequentially filled into the new tensor. This completes the concatenation operation of the first upsampled feature map and the second feature map after spatial size alignment in the channel dimension. The resulting first intermediate fusion feature tensor can combine global semantic information with intermediate structural semantic information.

[0074] For example, suppose that when detecting a shoe insole, the size of the third feature map is 16×16×256 (height×width×number of channels), and the size of the second feature map is 32×32×128. We can first perform a first upsampling operation on the third feature map, enlarging it to 32×32 using bilinear interpolation, resulting in a first upsampled feature map of size 32×32×256. At this point, each spatial location in this feature map corresponds to global semantic information of a broader region in the original image. Then, this 32×32×256 first upsampled feature map is concatenated with the 32×32×128 second feature map along the channel dimension, generating a first intermediate fusion feature tensor of size 32×32×384. The 384-dimensional feature vector at each 32×32 spatial location in this tensor contains both the structural arrangement information of the pins in that region (from the second feature map) and a broader range of contextual semantic information (from the upsampled third feature map), allowing subsequent anomaly detection to simultaneously reference both local structure and global context.

[0075] Step 203: Perform a second upsampling operation on the first intermediate fusion feature tensor to generate a second upsampled feature map; concatenate the second upsampled feature map and the first feature map using the second feature channel to generate a second intermediate fusion feature tensor.

[0076] In some implementations, a first intermediate fusion feature tensor is obtained, and a preset upsampling module is called. A bilinear interpolation algorithm is used to calculate the value of each new pixel position based on the target size, i.e., the spatial resolution of the first feature map. A second upsampled feature map is generated by weighted averaging of the four surrounding neighboring pixels, so that its spatial resolution is improved to be completely consistent with the first feature map.

[0077] After spatial alignment, a new tensor container is created with its spatial size remaining unchanged. The number of channels is set to the sum of the number of channels in the second upsampled feature map and the number of channels in the first feature map. Then, all channel data from the two feature maps are sequentially filled into the new tensor, thus completing the second feature channel concatenation operation between the second upsampled feature map and the first feature map directly extracted from the first residual block group of the ResNet18 residual network. This yields the second intermediate fused feature tensor. This allows for the complete fusion of shallow texture details, mid-level structural semantics, and deep global abstract information at the highest spatial resolution, enabling the feature vectors of each spatial location to possess comprehensive visual expression capabilities from micro to macro.

[0078] For example, suppose that in the current processing, the size of the first intermediate fused feature tensor is 32×32×384, and the size of the first feature map is 64×64×64. We can first perform a second upsampling operation on the first intermediate fused feature tensor, enlarging it to 64×64 through bilinear interpolation, resulting in a second upsampled feature map of size 64×64×384. At this point, each 64×64 spatial location in this feature map corresponds to a small local region in the original image, while preserving the intermediate structure and global semantic information after fusion.

[0079] The second upsampled feature map (64×64×384) is then concatenated with the first feature map (64×64×64) using the second feature channel, generating a second intermediate fused feature tensor of size 64×64×448. Each 448-dimensional feature vector at a 64×64 spatial location in this fused tensor fully contains information from all three levels: 64-dimensional fine texture details from the first feature map (such as the edge direction of the stitches and the continuity of the lines); 128-dimensional structural semantics derived from the second feature map (such as the arrangement density and spacing pattern of the pins); 256-dimensional global abstract semantics from the third feature map (such as the functional context of the region in the whole insole).

[0080] Step 204: Perform cross-channel information interaction and feature dimensionality reduction on the second intermediate fusion feature tensor through convolutional layers to generate a detection fusion feature tensor. The spatial location in the detection fusion feature tensor corresponds to a multi-dimensional feature vector, and the multi-dimensional feature vector includes texture details and structural semantics.

[0081] Among them, the detection fusion feature tensor refers to the feature representation finally generated after processing by the convolutional layer. It has the same spatial resolution as the first feature map, but the number of channels has been optimized and compressed. The multi-dimensional feature vectors at each spatial location have been fully fused and refined, and the multi-level visual information from texture details to structural semantics at that location has been compactly and completely encoded.

[0082] In some implementations, the second intermediate fusion feature tensor is obtained by setting the number of convolutional kernels to the target number of channels after dimensionality reduction (e.g., 256), the kernel size to 1×1 or 3×3, the stride to 1, and the padding method to maintain the same spatial dimensions. Each convolutional kernel performs a sliding window calculation across all channels of the input feature map, weighting and summing all channel values ​​within the local area covered by the window to generate a single channel value in the output feature map. This calculation method allows the information from the originally independent 448 channels in the input feature map to be fully mixed, enabling interactive fusion of features from different levels, while compressing the number of channels to 256 dimensions. The detection fusion feature tensor generated after convolutional layer processing retains its spatial resolution (still 64×64), but each spatial location corresponds to a 256-dimensional feature vector.

[0083] It should be noted that the weight parameters of the convolutional layer are indirectly determined during the training phase of the anomaly detection system through the feature distribution modeling process of normal insole samples. Since the Padim algorithm itself does not perform backpropagation training, this convolutional layer is used as part of the feature extractor ResNet18. Its weights are inherited from the ImageNet pre-trained model and kept frozen, and only its fixed feature transformation capabilities are used for information fusion and dimensionality reduction.

[0084] For example, suppose we generate a 64×64×448 second intermediate fused feature tensor that contains all information from texture to semantics. This tensor can then be fed into a convolutional layer consisting of 256 3×3 convolutional kernels.

[0085] During the computation, each 3×3 convolutional kernel performs a weighted summation of the values ​​of all 448 channels within its surrounding 3×3 neighborhood at each location in the input feature map, generating an output value. This convolutional kernel then slides across the entire feature map for computation, ultimately producing a 64×64 feature map.

[0086] 256 such convolutional kernels are computed in parallel to generate a 64×64×256 detection fusion feature tensor. At this point, redundant information in the original 448-dimensional features is compressed, and key features are enhanced. The 256-dimensional feature vector at each spatial location in this final tensor, such as the feature vector corresponding to a certain stitch position in the forefoot pressure zone of the insole, has integrated information such as shallow stitch texture, middle-layer stitch density and arrangement, and deep regional functional context.

[0087] Based on the above technical solution, by employing a deep convolutional neural network to extract feature maps at multiple depth levels, shallow networks can capture the texture details and edge information of the stitches, solving the problem of subtle defects such as missing stitches and broken threads that are difficult to identify with the naked eye; mid-level networks can extract the structural arrangement features of the stitches, addressing the difficulty in quantifying structural defects such as misaligned stitches; and deep networks can encode global abstract semantics and establish contextual understanding of the stitch region. This hierarchical extraction mechanism achieves complete coverage from local details to overall semantics at the low-level information level of the image. Secondly, by aligning the feature maps at different levels spatially and stitching the feature channels together, high-resolution texture details and low-resolution semantic information are organically integrated to generate multi-dimensional feature vectors corresponding to each spatial location. This solves the problem of incomplete feature information at a single level, ensuring that the feature representation of each local region simultaneously includes fine texture and structural semantics, thus improving the ability to identify complex stitch defects. The final generated detection fusion feature tensor serves as the basis for subsequent anomaly comparison, ensuring that the feature vector at each spatial location contains multi-dimensional visual information from micro to macro levels. This provides high-quality data support for accurately locating pin defects and generating anomaly heatmaps, effectively replacing subjective human judgment and achieving standardization and intelligence in the detection process.

[0088] In another possible implementation of the embodiments of this application, combined with Figure 1-3 As shown, the process of constructing a normal feature distribution model library can be achieved through steps 301 to 303, which are explained in detail below: Step 301: Extract the fixed spatial location within the normal fusion feature tensor, and summarize the channel number dimension feature vectors of all normal samples at the fixed spatial location to form a set of normal feature vectors for the spatial location.

[0089] In this context, a fixed spatial location refers to the pixel position with definite row and column coordinates in the normal fusion feature tensor. This position forms a one-to-one spatial mapping relationship with a specific local region in the original image. The channel-dimensional feature vector is a multidimensional array composed of all feature values ​​arranged along the channel dimension at a fixed spatial location. This vector comprehensively represents the visual attributes of the pin at that location. The normal feature vector set is a vector group formed by summarizing the feature vectors of all normal samples at the same fixed spatial location. This set is used to describe the statistical distribution pattern of the normal pin features at that location.

[0090] In some implementations, the normal fusion feature tensors corresponding to all normal insole samples that have completed feature fusion processing are loaded. At this point, the tensors have the same spatial size and number of channels. Then, a traversal algorithm is used to sequentially access each fixed spatial position in each tensor. For the specific spatial position currently being processed, the multidimensional feature vector of that position is extracted from the feature tensor of the first normal sample and stored in a temporary cache. Then, the feature vectors of the same spatial position are extracted from the feature tensors of the second, third, and so on up to the Nth sample. All the extracted feature vectors can be stacked vertically according to the sample order, and finally a normal feature vector set consisting of N multidimensional feature vectors is formed at that spatial position. At this point, the set completely covers the feature performance of that position in different normal samples, providing basic data for subsequent multivariate Gaussian distribution fitting.

[0091] Repeat the above extraction and summarization operations for all spatial locations until the feature vector set for all locations within the entire feature tensor space is constructed.

[0092] Step 302: Fit the normal feature vector set to a multivariate Gaussian distribution and calculate the standard mean vector and standard covariance matrix of all normal feature vectors.

[0093] The standard covariance matrix is ​​an important parameter matrix obtained during the construction phase of the normal feature distribution model library. It is obtained by fitting the feature vector set of all normal samples at a fixed spatial location to a multivariate Gaussian distribution. The matrix is ​​a square matrix with d rows and d columns. The elements on its diagonal represent the variance of each feature dimension itself, that is, the fluctuation range of the normal value of that dimension. The elements off the diagonal represent the covariance between different feature dimensions, which quantifies the degree of linear correlation and synchronicity of change between two feature dimensions.

[0094] In some implementations, all N multidimensional feature vectors, each with dimension d, are read from the normal feature vector set. These N d-dimensional feature vectors are then organized into an N x d matrix, and the arithmetic mean of the feature values ​​in each column is calculated. These d averages are then combined in their original order according to the feature dimensions to form a d-dimensional standard mean vector. This vector accurately describes the central tendency of the normal pin features at that spatial location across all dimensions.

[0095] After calculating the mean, for each element in the matrix, the covariance between the corresponding two feature dimensions is calculated. Specifically, for the i-th and j-th dimensions, all N samples are iterated through to obtain the values ​​in those two dimensions, and the mean of each dimension is subtracted from the mean to obtain the deviation value. The two deviation values ​​are multiplied, summed, and then divided by the sample size N minus 1 to obtain the unbiased estimated covariance. This calculation is repeated for all d-row, d-column positions, ultimately generating a d-row, d-column standard covariance matrix. This matrix fully records the correlation structure between feature dimensions, reflecting the inherent variation law of normal stitch features. Finally, the above multivariate Gaussian distribution fitting operation is performed sequentially for all spatial locations, generating a unique standard mean vector and standard covariance matrix for each location.

[0096] It should be noted that the estimation quality of the standard covariance matrix is ​​quite sensitive to the sample size. When the sample size is less than the feature dimension, the standard covariance matrix may become a singular matrix that cannot be inverted. Therefore, in practical applications, it is necessary to ensure that the number of normal samples is significantly greater than the feature dimension. At the same time, the standard covariance matrix is ​​calculated using an unbiased estimator, i.e., divided by N minus 1 instead of N, in order to reduce the estimation bias caused by sample sampling.

[0097] Step 303: Index and store the standard mean vector and standard covariance matrix according to their corresponding fixed spatial locations to obtain the normal feature distribution model library.

[0098] In some implementations, a set of standard mean vectors and standard covariance matrices corresponding to each spatial location is obtained, and an empty data container is created to hold the entire model library. Each fixed spatial location is then accessed sequentially according to the spatial traversal order of the feature fusion tensor. For a specific location being processed, its row coordinates i and column coordinates j are read as two-dimensional index values. A key-value pair mapping is established between this index value and the corresponding standard mean vector, and another key-value pair mapping is established between this index value and the corresponding standard covariance matrix. These two mappings are stored in the corresponding location entry in the data container. After completing the storage operation for one spatial location, the system moves to the next spatial location and repeats the above storage process until the standard mean vectors and standard covariance matrices of all spatial locations are completely stored in the data container. At this point, the data container contains the distribution parameters of all H rows and W columns of spatial locations from coordinates (0,0) to coordinates (H-1,W-1), and each location can be uniquely retrieved using its row and column coordinates to obtain the corresponding standard mean vector and standard covariance matrix. Finally, the data container is serialized into a binary file format and written to a persistent storage medium, while a memory-mapped copy is retained to accelerate subsequent online detection access, thus completing the construction of the normal feature distribution model library.

[0099] Based on the above technical solution, by extracting feature vectors from each fixed spatial location within the normal fusion feature tensor and summarizing all normal samples, a set of normal feature vectors corresponding to each spatial location is constructed. This allows for the alignment and aggregation of visual features from different samples at the same physical location, solving the problem of inconsistent judgments for the same product due to individual differences and visual fatigue in manual quality inspection. Simultaneously, a multivariate Gaussian distribution is fitted to this set to calculate the standard mean vector and standard covariance matrix for each location. The standard mean vector defines the visual benchmark for a normal pin in that region, while the standard covariance matrix quantifies the fluctuation range and correlation between feature dimensions. This transforms the originally vague and subjective normal standard into calculable and repeatable mathematical parameters, fundamentally solving the technical challenge of lacking quantitative judgment criteria for concealment and microscopic defects. Finally, the standard mean vectors and standard covariance matrices of all spatial locations are indexed and stored according to spatial coordinates to construct a complete normal feature distribution model library. This model library can be used as a comparison benchmark for subsequent online detection, enabling the system to quickly and accurately locate any subtle anomalies that deviate from the normal distribution. This achieves a leap from relying on human experience to relying on standard data models, improving the stability, consistency, and ability to identify micro-defects in production quality inspection.

[0100] In another possible implementation of the embodiments of this application, combined with Figure 1-4As shown, the process of comparing and calculating the detection fusion feature tensor with the normal feature distribution model library to generate an anomaly heatmap can be achieved through steps 401 to 404, which are explained in detail below: Step 401: Receive the detection fusion feature tensor and extract the standard mean vector in the normal feature distribution model library. Calculate the detection mean fusion value vector of the detection fusion feature tensor and the deviation vector between the detection mean fusion value vector and the standard mean vector.

[0101] The detection mean fusion value vector refers to an ordered list of values ​​formed by extracting the specific values ​​of each dimension of the corresponding multi-dimensional feature vector for each fixed spatial location in the detection fusion feature tensor of the insole to be detected. The bias vector is a new vector obtained by subtracting the detection mean fusion value vector from the standard mean vector element by element. Each component value in this vector can quantitatively reflect the absolute difference between the detected feature and the normal benchmark in the current dimension.

[0102] In some implementations, upon receiving the detection fusion feature tensor of an insole to be detected, a spatial location-by-spatial traversal process can be initiated. First, the first spatial location in the tensor is locked, and a multi-dimensional vector consisting of 256 feature values ​​at that location is extracted; this is the detection mean fusion value vector for that location. Then, using the row and column coordinates of the current spatial location as an index, the standard mean vector stored at that coordinate can be quickly retrieved and read from a pre-built and memory-loaded normal feature distribution model library.

[0103] Subsequently, using the vector arithmetic unit, element-wise subtraction is performed on the two vectors with the same dimension. The difference between each feature value in the detection mean fusion vector and the corresponding feature value in the standard mean vector is calculated. These differences are then combined in the original order to generate the deviation vector for that spatial location. This vector can then fully describe the specific deviation of the detected feature from the normal baseline in each dimension.

[0104] For example, suppose that when processing a specific coordinate point (24, 15) in the forefoot area of ​​an insole, the detection mean fusion vector of the detection fusion feature tensor at that location is [0.82, 0.15, 0.43, ...], totaling 256 values. That is, using coordinate (24, 15) as a key, a pre-stored standard mean vector can be retrieved from the normal feature distribution model library, let's say [0.80, 0.10, 0.40, ...]. By calculation: the difference in the first dimension is 0.82 - 0.80 = 0.02, the difference in the second dimension is 0.15 - 0.10 = 0.05, the difference in the third dimension is 0.43 - 0.40 = 0.03, and so on. The differences in all dimensions together constitute the deviation vector at that location [0.02, 0.05, 0.03, ...]. This vector indicates that the features of the pin at that location are higher than the normal baseline in multiple dimensions.

[0105] Step 402: Extract the standard covariance matrix from the normal feature distribution model library, and calculate the Mahalanobis distance based on the bias vector and the standard covariance matrix.

[0106] Mahalanobis distance is a distance metric based on multivariate statistics. By introducing the inverse of the standard covariance matrix as weights, it can effectively eliminate the influence of different dimensions and correlation issues between feature dimensions when calculating the deviation between the detected feature vector and the normal standard mean vector. The result is a scalar value that comprehensively reflects the overall degree to which the feature vector of the current spatial location deviates from the normal distribution model. The larger the value, the higher the probability of an anomaly.

[0107] In some implementations, after calculating the deviation vector of the current spatial location, the inverse of the standard covariance matrix, which has been pre-stored and inverted, is immediately retrieved from the normal feature distribution model library using the row and column coordinates of that spatial location as indices. A pre-defined Mahalanobis distance calculation function is then called, treating the deviation vector as a column vector. Its transpose is calculated, and then multiplied by the inverse of the standard covariance matrix to obtain a new row vector. This new row vector is then multiplied a second time by the original standard deviation vector, ultimately yielding a single scalar value—the Mahalanobis distance of that spatial location.

[0108] During the construction phase of the normal feature distribution model library, the standard covariance matrix of each spatial location is calculated by summarizing the feature vector set of all normal samples at that location. In order to ensure that the standard covariance matrix is ​​invertible in subsequent calculations, it is necessary to ensure that the number of training samples is significantly greater than the feature dimension, and to pre-compute the matrix inversion operation before storage, so that it can be directly read and used during online detection, avoiding repeated calculations and improving detection efficiency.

[0109] For example, suppose at a certain spatial location, the deviation vector is [2.0, 2.0], indicating that both dimensions deviate from the normal baseline by 2 units. If the two feature dimensions at this location are completely independent, and the standard covariance matrix is ​​the identity matrix, then the Mahalanobis distance is calculated as follows: However, if there is a strong positive correlation between the two dimensions at this position, the off-diagonal elements in the standard covariance matrix will be large, and its inverse matrix will be adjusted accordingly. In this case, the Mahalanobis distance calculated for the same bias vector [2.0, 2.0] may only be 1.5, indicating that this co-variance is within the normal distribution range. Conversely, if the bias vector is [2.0, -2.0], which contradicts the positive correlation structure, even if the absolute values ​​are the same, the Mahalanobis distance may be as high as 4.0, thus being accurately identified as an anomaly.

[0110] Step 403: Detect all spatial locations of the fused feature tensor or the Mahalanobis distance of each location to construct the initial anomaly matrix.

[0111] The initial anomaly matrix is ​​a two-dimensional numerical matrix formed by traversing all spatial locations of the detection fusion feature tensor, using the Mahalanobis distance calculated at each location as matrix elements, and arranging them according to the row and column order of the original spatial locations. The spatial resolution of this matrix is ​​the same as the size of a certain intermediate layer feature map before feature fusion processing, and each value in the matrix precisely corresponds to the degree of feature deviation of a certain local region in the original detection image.

[0112] In some implementations, after calculating the Mahalanobis distance of a certain spatial location in the detection fusion feature tensor, the value is temporarily stored in a memory buffer, and its corresponding row coordinate i and column coordinate j are recorded.

[0113] A double-loop traversal mechanism is initiated, where the outer loop scans row by row from top to bottom, and the inner loop scans column by column from left to right, sequentially visiting each spatial location in the detection fusion feature tensor. For each visited spatial location, the complete process of coordinate indexing, feature vector extraction, mean vector reading, bias vector calculation, covariance inverse matrix reading, and Mahalanobis distance calculation is repeated. The Mahalanobis distance value obtained in each calculation is then filled into the corresponding position in a pre-allocated two-dimensional matrix data structure with the same spatial size as the feature map, according to the row and column coordinates of the current spatial location.

[0114] Once the traversal mechanism has finished processing the spatial position of the last row and last column, all cells of this two-dimensional matrix are filled with the Mahalanobis distance value calculated for the corresponding position. At this point, the matrix is ​​constructed as a complete initial anomaly matrix, and the value of each element in the matrix represents the anomaly score of the corresponding local region in the original image.

[0115] Step 404: Perform adaptive smoothing processing on the initial anomaly matrix based on the material properties of the insole to generate an anomaly heatmap.

[0116] The adaptive smoothing process based on the material characteristics of the insole refers to the process of spatially filtering the initial anomaly matrix by dynamically adjusting the kernel size and weight coefficients of the smoothing filter, taking into full account the different noise characteristics and anomaly patterns of different areas of the insole due to differences in material density, texture roughness, and stitching techniques. The anomaly heatmap is a two-dimensional grayscale image generated after numerical normalization and scaling of the initial anomaly matrix following adaptive smoothing. The grayscale value of each pixel in this image (usually mapped to the range of 0-255) can intuitively reflect the degree to which the stitches at the corresponding position in the original image deviate from the normal distribution model; a higher grayscale value indicates a greater likelihood of an anomaly.

[0117] In some implementations, a pre-stored insole material property mapping table (which records the material type, texture roughness, and stitch density level corresponding to different physiological regions and process sub-regions) is read. Based on the physiological region and process sub-region identifiers of each spatial location in the initial anomaly matrix, the appropriate smoothing parameters for that location can be dynamically determined. For areas with complex textures and high pin density, such as high-density pin sub-areas, a larger smoothing kernel (e.g., 5×5) and a higher weight attenuation coefficient are configured to suppress normal fluctuation noise caused by the complexity of the texture itself. For areas with uniform material and sparse stitches, such as low-density stitch sub-areas, a smaller smoothing kernel (e.g., 3×3) and a lower weight attenuation coefficient are configured to preserve potential small anomaly details.

[0118] Gaussian filtering can then be applied to the initial anomaly matrix to selectively preserve genuine anomaly signals while suppressing material-related background noise. After adaptive smoothing filtering, the resulting matrix is ​​linearly normalized, mapping all values ​​to an integer grayscale range of 0-255. Then, bilinear interpolation is used to upsample the matrix to the same resolution as the original detection image, ultimately generating an anomaly heatmap that perfectly corresponds to the original image space. The highlighted areas in this heatmap indicate the potential locations of pin anomalies.

[0119] It should be noted that the weight distribution of the filter kernel in the Gaussian filtering algorithm should be adaptively adjusted according to the material characteristics of the region to which the current processing position belongs.

[0120] For example, suppose that the high-density stitch area in the forefoot pressure zone of the insole exhibits some numerical fluctuations in the initial anomaly matrix due to the dense stitches, ranging from 20 to 30; while the low-density stitch area in the edge stitching zone of this area has a smooth material and lower background noise, with fluctuations ranging from 5 to 10.

[0121] During adaptive smoothing, the system can identify high-density stitch areas, call preset material property parameters, and use a 5×5 Gaussian kernel for smoothing, effectively suppressing normal fluctuations of 20-30 to 10-15. At the edge stitching area, a 3×3 Gaussian kernel is used to preserve any minor anomalous signals that may exist in that area.

[0122] After normalization and upsampling, the noise caused by the texture in the forefoot area is eliminated, and only the real stitch abnormalities will be highlighted. Even very small abnormal points in the edge area can be clearly presented, thus achieving a precise balance of abnormal response in all areas of the insole.

[0123] Based on the above technical solution, by utilizing the standard mean vector and covariance matrix of each spatial location in the normal feature distribution model library, the Mahalanobis distance of the feature vector to be detected is calculated. This accurately captures minute defects such as missing needles, off-center needles, and broken threads that are indistinguishable to the naked eye, transforming the subjective and vague concept of "normal" into an objective and calculable statistical model. This fundamentally solves the problem of inconsistent judgments of the same product due to individual differences among quality inspectors. Simultaneously, by traversing all spatial locations to construct an initial anomaly matrix, the scattered anomaly metrics are organized into structured data according to spatial adjacency relationships, ensuring that the location information of each micro-defect is completely preserved, providing a spatial index basis for precise positioning. Finally, an adaptive smoothing process based on the insole material characteristics is introduced. The filtering parameters are dynamically adjusted according to the stitch density and texture complexity of different areas. This effectively suppresses normal texture noise caused by process complexity in high-density stitch areas while retaining possible subtle anomaly signals in low-density areas. This allows the anomaly heatmap to achieve balanced anomaly response sensitivity across different functional areas on the same insole. This enables precise positioning and provides a visualized defect distribution map for the production end, which can directly guide process improvement and enhance the stability of quality inspection and product yield in large-scale production.

[0124] In another possible implementation of the embodiments of this application, combined with Figure 1-5 As shown, the process of statistically analyzing the distribution of abnormal pixels within each detection area and generating regional anomaly statistics can be achieved through steps 501 to 505, which are explained in detail below: Step 501: Configure anomaly sensitivity threshold set for process sub-regions, and generate sub-region mask images for each process sub-region according to the spatial range of the process sub-regions. The spatial size of the sub-region mask image is the same as that of the anomaly heatmap, and the pixel positions in the sub-region mask that belong to the process sub-region are assigned a value of 1, while the pixel positions that do not belong to the process sub-region are assigned a value of 0.

[0125] Among them, the anomaly sensitivity threshold set is a set of judgment parameters configured separately for each process sub-region, used to measure the degree to which abnormal thermal values ​​in the region are considered valid anomalies; the sub-region mask image is a binary spatial index template, which identifies the valid detection area belonging to the process sub-region by a pixel value of 1, and excludes interference from outside the region by a pixel value of 0, thereby achieving accurate alignment and decoupling between the abnormal thermal map and the specific physical region.

[0126] In some implementations, a set of abnormality sensitivity thresholds is configured differently for each divided process sub-region (such as a high-density pin sub-region) based on the production process document. This set may contain threshold values ​​for multiple dimensions such as abnormal pixel judgment and connected region area.

[0127] This allows for pixel-level filling based on the precise spatial coordinates of each process sub-region on the detection image, according to the polygonal outline of the sub-region. All pixels belonging to the current process sub-region are assigned the value "1," while background or other region pixels not belonging to this sub-region are assigned the value "0," thus forming a sub-region mask image with the exact same size as the anomaly heatmap. This mask image then serves as a spatial filter for subsequent precise calculations, ensuring that all statistical operations are strictly confined within the target process sub-region, eliminating mutual interference between regions, and laying the foundation for generating accurate regional anomaly statistics.

[0128] Step 502: Perform pixel-by-pixel multiplication on the abnormal heat map and the sub-region mask image to extract the subset of abnormal heat values ​​in each process sub-region, and count the total number and percentage of abnormal pixels whose heat values ​​exceed the abnormality sensitivity threshold in the process sub-region.

[0129] In some implementations, the binarized mask image and anomaly heatmap corresponding to the sub-region of the process to be detected are obtained. Then, a pixel-by-pixel multiplication operation is performed between the anomaly heatmap and the mask image of that sub-region. This involves traversing every pixel position in the image and multiplying the pixel value of the anomaly heatmap at that position with the pixel value of the mask image at that position. Since the pixel values ​​in non-sub-region areas of the mask image are 0, the result of the multiplication at these positions is 0, while the mask value in the sub-region is 1. The multiplication preserves the original anomaly heatmap value, thus accurately extracting a subset of anomaly heatmap values ​​belonging only to that specific sub-region of the process.

[0130] At this point, the pre-configured anomaly sensitivity threshold for this process sub-region is read, and the subset of abnormal thermal values ​​is traversed and counted. Each thermal value is compared with the threshold, and all pixels with thermal values ​​greater than the threshold are recorded, resulting in a total number of abnormal pixels. Finally, the total number of pixels with a value of 1 in the mask image of this process sub-region is obtained as the total area of ​​the sub-region. The ratio of the total number of abnormal pixels to the total area of ​​the sub-region is calculated to obtain the proportion of abnormal pixels. These two statistical values ​​are then encapsulated and output as important components of the multidimensional anomaly features of this sub-region.

[0131] For example, suppose the system has detected an abnormal heatmap of the "high-density pin sub-region" in the forefoot pressure zone, and the mask image for this sub-region is ready. The mask size is 100×100 pixels, with 5000 pixels having a pixel value of 1 within the sub-region. The two images can then be multiplied pixel-by-pixel to obtain a new image matrix that retains only the heatmap values ​​of this sub-region. The preset anomaly sensitivity threshold for this sub-region is read as 150 (grayscale value). Then, the 5000 extracted heatmap values ​​are iterated, and 45 pixels are found to have heatmap values ​​greater than 150, resulting in a total of 45 abnormal pixels. The total pixel area of ​​this sub-region is 5000, and the percentage of abnormal pixels is calculated to be 45 / 5000 = 0.9%. These two values ​​will be recorded as key anomaly indicators for the current batch of detection for this sub-region, used for subsequent local judgment and global quality assessment.

[0132] Step 503: Perform connected component analysis on abnormal pixels in the process sub-regions, count the number of abnormal connected components and the maximum connected component area in each process sub-region, and calculate the spatial distribution entropy value of abnormal pixels in the sub-regions.

[0133] Connected component analysis refers to the process of clustering adjacent pixels in a binary image, grouping interconnected foreground pixels into the same independent region; the number of anomalous connected components refers to the total number of independent anomalous regions formed by connecting adjacent anomalous pixels within a certain process sub-region; the maximum connected component area refers to the total number of pixels in the connected component containing the most pixels among all anomalous connected components; spatial distribution entropy is a statistical indicator used to measure the randomness or disorder of the distribution of anomalous pixels within a sub-region. A higher entropy value indicates a more dispersed distribution of anomalous points, while a lower entropy value indicates a more concentrated cluster of anomalous points.

[0134] In some implementations, based on a subset of abnormal thermal values ​​in a process sub-region, pixels with thermal values ​​exceeding a threshold are assigned a value of 1, while other pixels are assigned a value of 0, resulting in a binary anomaly distribution map for that sub-region. Then, an eight-neighbor connectivity algorithm is used to traverse the image, marking adjacent pixels with values ​​of 1 as the same connected component, and assigning a unique identifier to each newly discovered connected component.

[0135] After traversal, the total number of marked connected components within the sub-region is counted to obtain the number of abnormal connected components. Simultaneously, the number of pixels contained in each connected component is recorded. By comparison, the connected component containing the most pixels is identified, and its pixel count is recorded as the area of ​​the largest connected component.

[0136] Next, the coordinates of all anomalous pixels with a value of 1 are statistically analyzed in the binarized anomaly distribution map. The sub-region is then divided into a predetermined number of equally divided grids (e.g., 10×10 grids). The proportion of anomalous pixels in each grid to the total number of anomalous pixels is calculated as the probability distribution, and then substituted into the information entropy formula. Calculate the spatial distribution entropy value of this sub-region. ,in This represents the percentage of outliers within the i-th grid.

[0137] For example, suppose we are detecting a "low-density stitching sub-area" in the arch support zone. First, a binary anomaly distribution map of this sub-area is generated, containing 80 anomalous pixels. Then, after performing eight-neighbor connected component analysis, 12 independent anomalous connected components are identified. The largest connected component contains 23 pixels, therefore the total number of anomalous connected components is 12, and the largest connected component area is 23. Next, this sub-area is divided into 25 grids of 5×5. The number of anomalous pixels in each grid is counted, and the percentage of anomalous pixels in each grid is calculated. Substituting these values ​​into the information entropy formula yields a spatial distribution entropy value of 3.2. These three values ​​indicate that the anomalous points in this sub-area are scattered into 12 small patches, with the largest patch containing 23 pixels. The distribution is relatively dispersed (high entropy value), possibly corresponding to scattered, point-like dirt rather than concentrated, large-area damage.

[0138] Step 504: Combine the percentage of abnormal pixels, the number of abnormal connected components, the area of ​​the largest connected component, and the spatial distribution entropy value of the process sub-region to generate a regional abnormal feature vector.

[0139] In some implementations, the following parameters are obtained: the percentage of abnormal pixels reflecting the degree of abnormal density, the number of abnormal connected regions reflecting the splitting of abnormal regions, the area of ​​the largest connected region reflecting the size of the largest abnormality, and the spatial distribution entropy value reflecting the degree of disorder in the abnormal distribution.

[0140] The four values ​​can be sequentially filled into the fixed-dimensional array positions according to the predefined feature vector structure template (e.g., the first dimension is fixed as the proportion of abnormal pixels, the second dimension is the number of abnormal connected components, the third dimension is the area of ​​the largest connected component, and the fourth dimension is the spatial distribution entropy value, so as to ensure that the meaning of each dimension can be correctly interpreted during subsequent processing), forming a four-dimensional feature vector.

[0141] After vector assembly is completed, the current process sub-region identification information, such as sub-region number and physiological region to which it belongs, is added to the feature vector to form a complete labeled regional anomaly feature vector. This vector is then temporarily stored in memory, waiting to be weighted and fused with the feature vectors of other sub-regions under the same physiological region.

[0142] Step 505: Weight and fuse the regional anomaly feature vectors of all process sub-regions under the same physiological region to generate a comprehensive anomaly assessment index, and encapsulate it in the same way as the regional anomaly feature vectors into regional anomaly statistical data.

[0143] In some implementations, the regional anomaly feature vectors of all process sub-regions under the same physiological region are obtained. The fusion weight parameters for that physiological region are then read from a preset weight configuration table, which contains two levels: First, there is the sub-region weight for each process sub-region, which reflects the importance of that sub-region in the entire physiological region. It is usually preset based on the sub-region area ratio or functional importance. Second, the dimensional weights of the four dimensions of the feature vector are used to balance the contribution of different dimensional indicators to the comprehensive evaluation.

[0144] This allows for direct dimensional weighting of the feature vectors of each process sub-region, by multiplying the values ​​of the four dimensions by their corresponding dimensional weight coefficients to obtain the weighted feature vectors of the sub-regions.

[0145] Next, sub-region weighted fusion is performed on the weighted feature vectors of all process sub-regions. That is, the four dimensions of the weighted feature vector of each sub-region are multiplied by the sub-region weight of that sub-region, and then the results of all sub-regions are added together according to the dimensions to obtain the fusion value of the four dimensions.

[0146] Finally, these four fusion values ​​are combined again to form a four-dimensional comprehensive abnormality assessment index to reflect the overall condition of the physiological region in different abnormality dimensions.

[0147] After the fusion calculation is completed, the comprehensive anomaly assessment index of the physiological region can be encapsulated with the original regional anomaly feature vectors of all process sub-regions in the region, organized together according to a predefined JSON data structure, and metadata such as physiological region identifiers and timestamps can be added to finally generate a complete regional anomaly statistical data package for subsequent local judgment and global quality assessment.

[0148] Based on the above technical solution, by configuring differentiated judgment thresholds for each process sub-region and physiological region, the practical production problem of not being able to adopt a unified testing standard for different regions due to differences in functional importance and process complexity is solved. This allows for a first-level local judgment, comparing the multi-dimensional feature vector of each process sub-region with its sub-region threshold, and simultaneously comparing the comprehensive evaluation index of each physiological region with its region threshold. This accurately identifies and filters out all local regions that do not exceed the standards, effectively solving the management problem that macro-judgment may mask subtle flaws in key areas, ensuring that the quality of each functional region meets design requirements. After passing the first-level local judgment, a global anomaly score is calculated based on the comprehensive evaluation index of each physiological region and its preset weight coefficients to reflect the overall quality of the entire insole, overcoming the application bottleneck that a single local indicator cannot comprehensively evaluate the overall product quality. Finally, the global anomaly score is compared with the preset global judgment threshold to obtain a clear conclusion of whether it is qualified or not. It not only achieves accurate capture and location of hidden micro-defects, but also transforms subjective human judgment into objective quantitative multi-level threshold comparison, completely solving the problem of lack of judgment stability caused by individual differences and visual fatigue in manual inspection, and finally outputting standardized and traceable quality inspection results.

[0149] The above primarily describes the solutions of the embodiments of this application from the perspective of device implementation. It is understood that each device, such as a vision-based insole stitch anomaly detection system, includes at least one of the hardware structures and software modules corresponding to each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0150] When using integrated units, Figure 6 The vision-based insole stitch anomaly detection system described in the above embodiments is shown, including: a data acquisition module, a feature tensor generation module, a standard feature construction module, a comparison calculation module, an anomaly detection module, and a judgment and analysis module.

[0151] The data acquisition module is used to acquire the original image of the insole to be tested, perform noise reduction and illumination equalization processing, and obtain a standardized test image.

[0152] The feature tensor generation module is used to perform feature fusion processing on the detection image and generate a detection fusion feature tensor. The feature fusion processing includes extracting feature maps of the detection image at multiple different depth levels through a deep convolutional neural network, performing spatial scale alignment and feature channel stitching. Each spatial position in the detection fusion feature tensor corresponds to a feature vector.

[0153] The standard feature construction module is used to obtain standardized images of normal insole samples, generate normal fused feature tensors through feature fusion processing, and build a normal feature distribution model library.

[0154] The comparison calculation module is used to compare and calculate the detection fusion feature tensor with the normal feature distribution model library to generate an anomaly heatmap.

[0155] The anomaly detection module is used to divide the detection image into detection areas based on the physical structural characteristics of the insole, and combine the anomaly heatmap to statistically analyze the distribution of abnormal pixels in each detection area, generating regional anomaly statistics.

[0156] The judgment and analysis module is used to determine whether the insole to be tested is a qualified product based on the regional abnormal statistical data and the preset global judgment threshold, and outputs the detection result image containing abnormal location markers and a structured detection report.

[0157] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and variations.

Claims

1. A vision-based method for detecting abnormal stitches in shoe insoles, characterized in that, include: The original image of the insole to be tested is acquired, and noise reduction and illumination equalization processing are performed to obtain a standardized test image. The detected image is subjected to feature fusion processing to generate a detection fusion feature tensor. The feature fusion processing includes extracting feature maps of the detected image at multiple different depth levels through a deep convolutional neural network, performing spatial scale alignment and feature channel stitching. Each spatial position in the detection fusion feature tensor corresponds to a feature vector. Obtain standardized images of normal insole samples, generate normal fusion feature tensors through the aforementioned feature fusion processing, and construct a normal feature distribution model library; The detection fusion feature tensor is compared and calculated with the normal feature distribution model library to generate an anomaly heatmap; Based on the physical structure characteristics of the insole, the detection image is divided into detection areas. Combined with the abnormal heat map, the distribution of abnormal pixels in each detection area is statistically analyzed to generate regional abnormal statistical data. Based on the regional anomaly statistics and the preset global judgment threshold, it is determined whether the insole to be tested is a qualified product, and the detection result image containing anomaly location markers and a structured detection report are output.

2. The vision-based insole stitch anomaly detection method according to claim 1, characterized in that, The deep convolutional neural network is a ResNet18 residual network, and the first residual block group, the second residual block group and the third residual block group arranged in sequence are propagated forward to extract the first feature map, the second feature map and the third feature map with decreasing resolution and increasing number of channels in sequence. The first feature map is used to characterize the texture details of the pins; The second feature map is used to characterize the structural semantic information of the pin; The third feature map is used to represent the global abstract semantic information of the pin.

3. The vision-based insole stitch anomaly detection method according to claim 2, characterized in that, The process of performing feature fusion processing on the detected image to generate a detection fusion feature tensor specifically includes: The detected image is input into the ResNet18 residual network, and the first feature map, the second feature map, and the third feature map are extracted through the first residual block group, the second residual block group, and the third residual block group, respectively. The third feature map is subjected to a first upsampling operation to generate a first upsampling feature map; the first upsampling feature map and the second feature map are spliced ​​together by a first feature channel to generate a first intermediate fusion feature tensor. A second upsampling operation is performed on the first intermediate fusion feature tensor to generate a second upsampled feature map; the second upsampled feature map and the first feature map are concatenated using a second feature channel to generate a second intermediate fusion feature tensor. The second intermediate fusion feature tensor is subjected to cross-channel information interaction and feature dimensionality reduction through convolutional layers to generate a detection fusion feature tensor. The spatial location in the detection fusion feature tensor corresponds to a multi-dimensional feature vector, and the multi-dimensional feature vector includes texture details and structural semantics.

4. The vision-based insole stitch anomaly detection method according to claim 1, characterized in that, The construction process of the normal feature distribution model library specifically includes: Extract the fixed spatial location within the normal fusion feature tensor, and summarize the channel number dimension feature vectors of all normal samples at the fixed spatial location to form the normal feature vector set of the spatial location; Perform multivariate Gaussian distribution fitting on the set of normal feature vectors, and calculate the standard mean vector and standard covariance matrix of all the normal feature vectors; The standard mean vector and standard covariance matrix are indexed and stored according to their corresponding fixed spatial locations to obtain a normal feature distribution model library.

5. The vision-based insole stitch anomaly detection method according to claim 1, 3, or 4, characterized in that, The process of comparing the detected fusion feature tensor with the normal feature distribution model library to generate an anomaly heatmap specifically includes: Receive the detection fusion feature tensor and extract the standard mean vector in the normal feature distribution model library, calculate the detection mean fusion value vector of the detection fusion feature tensor and the deviation vector between the detection mean fusion value vector and the standard mean vector; Extract the standard covariance matrix from the normal feature distribution model library, and calculate the Mahalanobis distance based on the bias vector and the standard covariance matrix; By traversing all spatial locations of the detection fusion feature tensor or the Mahalanobis distance at each location, an initial anomaly matrix is ​​constructed; An adaptive smoothing process based on the material properties of the insole is applied to the initial anomaly matrix to generate an anomaly heatmap.

6. The vision-based method for detecting abnormal insole stitches according to claim 1, characterized in that, The process of dividing the detection image into detection regions based on the physical structural features of the insole specifically includes: The global spatial coordinate matrix of the insole image is obtained, and the insole is divided into physiological regions according to the biomechanical characteristics of the human foot. The physiological regions include the forefoot pressure zone, the arch support zone, the heel buffer zone, and the edge stitching zone. Based on the stitch density distribution characteristics of the insole, each physiological region is divided into process sub-regions, which include high-density stitch sub-regions and low-density stitch sub-regions.

7. The vision-based insole stitch anomaly detection method according to claim 6, characterized in that, The process of statistically analyzing the distribution of abnormal pixels within each detection area and generating regional anomaly statistics specifically includes: Configure an anomaly sensitivity threshold set for the process sub-region, and generate a sub-region mask image for each process sub-region according to the spatial range of the process sub-region. The sub-region mask image has the same spatial size as the anomaly heat map, and the pixel positions in the sub-region mask that belong to the process sub-region are assigned a value of 1, and the pixel positions that do not belong to the process sub-region are assigned a value of 0. The abnormal heat map and the sub-region mask image are multiplied pixel by pixel to extract the abnormal heat values ​​subset of each process sub-region, and the total number and percentage of abnormal pixels in the process sub-region whose heat values ​​exceed the abnormality sensitivity threshold are counted. Connectivity analysis is performed on the abnormal pixels in the process sub-regions to count the number of abnormal connected components and the maximum area of ​​the connected component in each process sub-region, and the spatial distribution entropy value of the abnormal pixels in the sub-regions is calculated. The abnormal pixel ratio, abnormal connected component number, maximum connected component area, and spatial distribution entropy value of the process sub-region are combined to generate a regional abnormal feature vector; The regional anomaly feature vectors of all process sub-regions under the same physiological region are weighted and fused to generate a comprehensive anomaly assessment index, which is then encapsulated into regional anomaly statistics along with the regional anomaly feature vectors.

8. The vision-based insole stitch anomaly detection method according to claim 7, characterized in that, The process of determining whether the insole to be tested is a qualified product based on the regional anomaly statistics and the preset global judgment threshold specifically includes: Configure a sub-region determination threshold for each of the process sub-regions, and configure a region determination threshold for each of the physiological regions; Based on the sub-region determination threshold and the region determination threshold, a first-level local determination is performed on the region abnormal feature vector and the comprehensive abnormality evaluation index. The first-level local determination includes comparing each region abnormal feature vector with the corresponding sub-region determination threshold and comparing each comprehensive abnormality evaluation index with the corresponding region determination threshold, and filtering out data in which neither the process sub-region nor the physiological region exceeds its corresponding determination threshold. After the first level of local judgment is met, the global anomaly score is calculated based on the comprehensive anomaly assessment index and the preset weight coefficient of the physiological region. The global anomaly score is compared with a preset global judgment threshold to determine whether the insole to be tested is a qualified product.

9. The vision-based method for detecting abnormal insole stitches according to claim 1 or 6, characterized in that, It also includes a parallel-deployed auxiliary detection channel, which is used to perform pin anomaly detection on the detection image based on a color matrix, specifically including: The detected image is divided into grids to generate a set of grid cells with M rows and N columns, and each grid cell corresponds to a local pin region in the original image, where M and N are positive integers; Extract the color feature matrix of each grid cell in a preset color space. The color feature matrix includes at least the mean brightness, mean color channel, and standard deviation of all pixels in the grid cell. After filtering out the grid cells with a ratio of K% and removing the grid cells with the highest and lowest brightness, the color distribution reference range of normal pins is statistically analyzed. The color distribution reference range includes at least the brightness reference range and the reference range of each color channel. K is a preset positive integer. The color feature matrix is ​​compared item by item with the color distribution reference interval to identify abnormal grid cells; The abnormal grid cells are visually labeled, and the total number and distribution of the abnormal grid cells are counted to generate an auxiliary detection statistical report; The auxiliary detection statistical report is merged with the detection report to output a comprehensive quality inspection result.

10. A vision-based insole stitch anomaly detection system, characterized in that, The vision-based insole stitch anomaly detection method according to any one of claims 1-9, wherein the insole stitch anomaly detection system specifically includes: The data acquisition module is used to acquire the original image of the insole to be tested, perform noise reduction and illumination equalization processing, and obtain a standardized test image. The feature tensor generation module is used to perform feature fusion processing on the detection image to generate a detection fusion feature tensor. The feature fusion processing includes extracting feature maps of the detection image at multiple different depth levels through a deep convolutional neural network, performing spatial scale alignment and feature channel stitching. Each spatial position in the detection fusion feature tensor corresponds to a feature vector. The standard feature construction module is used to acquire standardized images of normal insole samples, generate normal fusion feature tensors through the feature fusion processing, and construct a normal feature distribution model library. The comparison calculation module is used to compare and calculate the detection fusion feature tensor with the normal feature distribution model library to generate an anomaly heatmap; An anomaly detection module is used to divide the detection image into detection areas based on the physical structural characteristics of the insole, and, in conjunction with the anomaly heatmap, to statistically analyze the distribution of abnormal pixels in each detection area and generate regional anomaly statistics. The judgment and analysis module is used to determine whether the insole to be tested is a qualified product based on the abnormal statistical data of the region and the preset global judgment threshold, and output the detection result image containing abnormal location markers and the structured detection report.

Images