Intelligent detection and real-time grading method for stone surface image defects

Abstract

The application discloses a kind of stone surface image defect intelligent detection and real-time grading method, by the RGB image of the stone surface to be detected, thermal radiation image and depth image are synchronously collected, three modal images are respectively input corresponding feature extraction network and extract multi-scale hierarchical features, with cross-modal gate fusion module Channel inter-attention weight and spatial attention weight of each modal feature map are calculated respectively for each feature level and dynamically weighted fusion, generate unified cross-modal defect representation, then through grading decision network based on four semantic attributes mapping to continuous quality score space, configurable grading threshold output discrete quality level.The application utilizes the complementary characteristics of multi-modal information, solves the problem that single visible light image is difficult to accurately distinguish natural texture from real defect under complex texture background, while completely decouples defect objective attribute description and grading subjective standard application, significantly reduces deployment and maintenance cost.

Description

Technical Field

[0001] This invention relates to the field of image recognition and processing technology, and in particular to an intelligent detection and real-time grading method for image defects on stone surfaces. Background Technology

[0002] As a high-end material widely used in architectural decoration, home countertops, and artistic carving, the surface quality of natural stone is a core factor determining its grade, added value, and applicable scenarios. During the processing of natural stone, including quarrying, sawing, grinding, and polishing, various defects such as cracks, stains, holes, scratches, and edge defects are inevitably produced. Accurately detecting these defects and conducting reasonable quality grading is a crucial step in achieving standardized production and quality control in the stone processing industry.

[0003] Currently, machine vision-based stone defect detection methods have gradually replaced manual visual inspection and become the mainstream solution in the industry. However, existing methods generally suffer from the following two shortcomings. At the information perception level, detection systems typically rely solely on a single visible light RGB image, extracting texture features through convolutional neural networks for defect identification. Natural stone itself has complex and varied texture patterns, and some texture directions and color band distributions exhibit highly similar visual features to real defects in RGB images. A single modal information source is insufficient to effectively distinguish between natural texture variations and structural damage, resulting in persistently high false positive and false negative rates.

[0004] At the grading decision-making level, existing methods generally use single indicators such as the proportion of defect area or the number of defects, and classify grades by setting fixed thresholds. However, the quality grade of stone is the result of a combination of factors such as defect type, location criticality, and structural integrity. A single indicator cannot fully reflect the impact of defects on the actual use value of the slab. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide an intelligent detection and real-time grading method for defects in stone surface images, which can accurately distinguish between real defects and texture variations in complex natural texture backgrounds, and at the same time realize configurable quality grading based on multi-dimensional defect attributes, so that the grading standard can be flexibly adjusted according to application requirements without retraining the model.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: Firstly, the present invention provides an intelligent detection and real-time grading method for image defects on stone surfaces, characterized by comprising the following steps: Step 1: Acquire multimodal images of the surface of the stone to be inspected. The multimodal images include RGB images, thermal radiation images, and depth images, which are acquired simultaneously in the same field of view. Step 2: Input the RGB image into the first feature extraction network, the thermal radiation image into the second feature extraction network, and the depth image into the third feature extraction network. The three feature extraction networks extract the hierarchical features of each modality. The hierarchical features include multi-scale feature maps from shallow to deep layers. Feature maps with the same spatial resolution in different modalities belong to the same feature level. Multiple feature levels form a hierarchical sequence according to resolution from high to low. Step 3: Through the cross-modal gating fusion module, for each feature level, the inter-channel attention weight and spatial attention weight of each modal feature map are calculated respectively. The inter-channel attention weight and spatial attention weight are used to dynamically weight and fuse the RGB feature map, thermal radiation feature map and depth feature map of the same feature level to generate the cross-modal fusion feature of that feature level. The cross-modal fusion features of each feature level are aggregated through skip connections to form a unified cross-modal defect representation. Step four: Based on the cross-modal defect characterization, a quality grading result for the stone surface is generated through a hierarchical decision network. The hierarchical decision network includes a multi-task semantic parsing branch and a scoring network branch. The semantic parsing branch is used to predict four semantic attributes: defect type, defect area ratio, defect location criticality, and structural influence index, and concatenate them into an embedding vector. The scoring network branch maps the embedding vector to a continuous quality scoring space and converts the continuous score into a discrete quality level output through a configurable grading threshold.

[0007] Optionally, in step two, the first feature extraction network adopts the ConvNeXt architecture, the second feature extraction network adopts the EfficientNet architecture, and the third feature extraction network adopts the PointNet variant architecture. Each feature extraction network is initialized with weights pre-trained on the ImageNet dataset and jointly adjusted on the stone multimodal dataset to ensure that the hierarchical features output by each modal feature extraction network are consistent in channel dimension and spatial resolution, and to enable the cross-modal gating fusion module to perform feature alignment.

[0008] As described above, the first feature extraction network adopts the ConvNeXt architecture, which effectively captures the macroscopic direction and structural information of stone texture through large convolutional kernels and inverse bottleneck structures. The second feature extraction network adopts the EfficientNet architecture, which extracts temperature distribution features in thermal radiation images while maintaining low computational overhead through its composite scaling strategy. The third feature extraction network adopts a variant of the PointNet architecture, which can effectively handle the spatial distribution characteristics of 3D point clouds in depth images. All three networks are initialized with ImageNet pre-trained weights and jointly adjusted on the stone multimodal dataset, so that the features of each modality achieve consistent output in terms of channel dimension and spatial resolution. This provides a directly alignable feature foundation for the cross-modal gating fusion module, avoiding additional interpolation or projection errors introduced by the mismatch of feature dimensions between modalities, and improving fusion accuracy and training efficiency.

[0009] Optionally, in step three, the cross-modal gated fusion module includes multiple gated fusion units, each corresponding to a feature level. For the RGB feature map, thermal radiation feature map, and depth feature map of the current feature level, channel gate vectors for each modality are generated through global average pooling and fully connected layers, while spatial gate matrices for each modality are generated through convolutional layers. The channel gate vectors and spatial gate matrices are multiplied by the feature map of the corresponding modality through channel-wise multiplication and spatial position-wise multiplication, respectively, to obtain weighted modal feature maps. The weighted modal feature maps are concatenated along the channel dimension and fused through convolutional layers to output the cross-modal fusion feature of that feature level.

[0010] As described above, each gated fusion unit consists of two parallel attention mechanisms: channel gating and spatial gating. The channel-gated vector is generated through global average pooling and fully connected layers, enabling it to assess the overall importance of each modality's feature channels from the global receptive field, adaptively enhancing channels beneficial to the current task while suppressing noisy channels. The spatial gating matrix is ​​generated through convolutional layers, preserving spatial location information and assigning pixel-wise weights to different locations in the feature map, allowing the fusion module to automatically adjust the contribution ratio of each modality in areas with uniform texture and defect edges. The two mechanisms are then concatenated and fused after weighting the feature map through channel-wise multiplication and spatial location-wise multiplication, achieving dual adaptive adjustment of the channel and spatial dimensions. The resulting cross-modal fused features possess both discriminative power and robustness.

[0011] Optionally, in step four, the multi-task semantic parsing branch takes the cross-modal defect representation as input and connects in parallel a defect classification head, an area regression head, a location criticality assessment head, and a structural influence index assessment head. The defect classification head outputs the probability distribution of five defect types: cracks, spots, holes, scratches, and edge defects. The area regression head outputs the ratio of the defect area to the total board area. The location criticality assessment head outputs a criticality score based on the distance of the defect from the edge of the board and the stress characteristics of the area. The structural influence index assessment head outputs a structural integrity influence index based on the defect type and depth information. The outputs of the defect classification head, area regression head, location criticality assessment head, and structural influence index assessment head are normalized and concatenated into a fixed-dimensional defect semantic embedding vector.

[0012] As described above, by parallelly connecting the defect classification head, area regression head, location criticality assessment head, and structural impact index assessment head, this branch can simultaneously extract four semantic attributes of different properties from the same cross-modal defect representation. The defect classification head identifies the type of defect, the area regression head quantifies the spatial proportion of defects, the location criticality assessment head assesses the impact of defects on the usability of the board material based on the distance of defects from the edge and the stress characteristics of the area, and the structural impact index assessment head combines depth information to determine whether defects affect the structural integrity of the board material. The four outputs are normalized and concatenated into a fixed-dimensional defect semantic embedding vector, enabling the classification decision to be based on a comprehensive assessment of four dimensions: defect type, scale, location, and structural impact. This overcomes the one-sidedness of classification based on a single area index and significantly improves the rationality of the classification results.

[0013] Optionally, the scoring network branch specifically consists of two fully connected layers and a Sigmoid activation layer, used to map the defect semantic embedding vector to a continuous quality score between 0 and 1; the configurable grading threshold includes three boundary values, and the grading standard parameters are received through the user interface, the continuous quality score is compared with the boundary values, and the grading result of Grade A, Grade B, Grade C, Grade D or scrap is output.

[0014] As described above, the scoring network branch consists of two fully connected layers and one sigmoid activation layer. It uses a concise mapping structure to convert the defect semantic embedding vector into a continuous quality score of 0 to 1, resulting in high computational efficiency. Three configurable boundary values ​​receive grading standard parameters through a user interface, allowing quality inspectors to set the boundaries of quality levels as needed. When customer standards, industry specifications, or application scenarios change, only the boundary value parameters need to be adjusted to adapt to the new grading standard, without requiring any retraining of the multi-task semantic parsing branch and the scoring network branch. This mechanism completely decouples the description of objective defect attributes from the application of subjective grading standards, significantly reducing system switching and maintenance costs.

[0015] Optionally, the training process of the first feature extraction network, the second feature extraction network, the third feature extraction network, and the cross-modal gating fusion module includes a pre-training stage. In the pre-training stage, a diffusion model is used to output synthetic defect images to construct a synthetic training set. During the generation process, the diffusion model performs decoupling control of background texture, defect morphology, and lighting conditions. Using a normal stone texture image as a condition, texture features are injected into the diffusion denoising process through a cross-attention mechanism. A specified type of defect morphology is generated at a specified location, and different lighting environments are simulated, thereby obtaining a multimodal synthetic defect image covering the combination space of texture background, defect type, and lighting conditions. The multimodal synthetic defect image is used to pre-train each feature extraction network and the cross-modal gating fusion module.

[0016] As described above, using a diffusion model to output synthetic defect images to construct a training set allows for the generation of multimodal synthetic defect images covering various combination spaces, conditioned on normal stone texture images and through decoupling control specifying background texture, defect morphology, and lighting conditions. This approach effectively overcomes the data bottleneck of scarce and imbalanced defect samples in real stone production lines, significantly reducing the cost of manual annotation. The pre-trained feature extraction network and fusion module already possess basic perception capabilities for multimodal defect patterns, requiring only a small number of labeled samples for rapid convergence during subsequent fine-tuning with real data, significantly improving the model's deployment efficiency and generalization performance.

[0017] Optionally, the method is deployed in an edge-cloud collaborative architecture. The edge deploys a lightweight model that has undergone knowledge distillation and integer quantization to perform real-time inference and classification in steps one to four. The cloud deploys the diffusion model and continuously uses newly added normal stone texture images to output synthetic defect samples. During the inference process, the edge performs confidence evaluation on each frame of image, marks samples with defect classification confidence below a first threshold and above a second threshold as uncertain samples and uploads them to the cloud. The cloud uses the synthetic defect samples in combination with the uncertain samples to incrementally adjust the model and sends the adjusted model parameters to the edge via OTA.

[0018] As described above, a lightweight model is deployed at the edge to perform real-time inference, while the cloud continuously outputs synthetic defect samples using a diffusion model and makes incremental adjustments based on uncertain samples returned from the edge, forming a closed loop for continuous model evolution. This architecture physically separates high-real-time inference tasks from computationally intensive training tasks, meeting the millisecond-level response requirements of the production line while fully utilizing cloud computing power for model iteration. The edge automatically filters uncertain samples for upload through confidence assessment, transmitting only high-value data and effectively controlling network bandwidth consumption. OTA (Over-The-Air) updates enable production line upgrades without downtime, avoiding production interruptions caused by model updates.

[0019] Optionally, the confidence assessment step performed at the edge specifically involves: obtaining the maximum category probability value output by the defect classification head in the hierarchical decision network as the confidence level, setting the first threshold to 0.85 and the second threshold to 0.5; classifying samples with the maximum category probability value between 0.5 and 0.85 as uncertain samples; and comparing the multimodal image data corresponding to the uncertain samples with the abnormal response heatmap generated by the edge inference. Figure 1 The data is then packaged and uploaded to the cloud, where the model can be incrementally adjusted to improve its ability to identify boundary ambiguity defects.

[0020] As described above, the maximum category probability value output by the defect classification head in the hierarchical decision network is used as the confidence level. With dual thresholds of 0.85 and 0.5, samples with confidence levels between these two thresholds are classified as uncertain samples. These samples are located near the model's decision boundary and contain information most helpful in improving the model's discriminative ability. The multimodal image data of these uncertain samples is then compared with anomaly response heatmaps. Figure 1 The data was packaged and uploaded to the cloud, providing precise uncertainty samples and spatial context information for incremental adjustments in the cloud. This enabled the cloud to perform targeted training on weak areas of the model, improving the model's ability to identify boundary ambiguity defects and avoiding the ineffective transmission and processing of massive amounts of full-sample data.

[0021] Optionally, the step of generating quality grading results based on the cross-modal defect characterization in step four is implemented in the following way: Multiple normal, defect-free stone samples are pre-acquired in multimodal images; cross-modal defect characterizations of normal samples are extracted according to the processing flow of steps one to three; the cross-modal defect characterizations corresponding to normal samples are used as cross-modal fusion benchmark features to construct a cross-modal fusion feature memory for normal samples; during the detection stage, the cross-modal defect characterizations obtained from the processing of the stone image to be detected in steps one to three are used for K-nearest neighbor retrieval in the cross-modal fusion feature memory for normal samples; the reconstruction deviation of each feature level is calculated; and the reconstruction deviations of each level are weighted and fused to generate an abnormal response heatmap; the grading decision network uses the abnormal area ratio, response intensity peak, and spatial distribution entropy in the abnormal response heatmap as at least a part of the defect semantic attributes and inputs them into the scoring network branch to generate grading results.

[0022] As described above, by pre-constructing a cross-modal fusion feature memory library of normal samples, the multimodal fusion features of normal stone are stored as a benchmark. During detection, the cross-modal defect characterization of the sample to be detected is used for K-nearest neighbor retrieval in the memory library, and the reconstruction deviation at each level is calculated. Areas with larger deviations are the potential defect locations, and the fusion deviation generates an anomaly response heatmap. This method only requires normal, defect-free samples to complete the benchmark construction, eliminating the need to collect and label various defect samples. It is particularly suitable for production line environments where defect types are inexhaustible and new stone varieties are frequently switched, significantly reducing data preparation costs and deployment barriers.

[0023] Optionally, the normal sample cross-modal fusion feature memory includes three levels: a global structure sub-memory, a local texture sub-memory, and a micro-texture direction sub-memory. The global structure sub-memory stores the global average pooling vector of the high-level cross-modal fusion features of each sample. The local texture sub-memory stores the local feature vectors of the intermediate-level cross-modal fusion features after meshing. The micro-texture direction sub-memory stores the directional statistical feature vectors of the low-level cross-modal fusion features after directional filtering. The reconstruction bias of each feature level includes the global structure bias retrieved from the global structure sub-memory, the local texture bias retrieved from the local texture sub-memory, and the directional feature bias retrieved from the micro-texture direction sub-memory. The three are weighted and summed using learnable fusion weights to generate the abnormal response heatmap.

[0024] As described above, the global structure sub-memory stores global pooling vectors of high-level features to capture macroscopic deviations in the overall texture layout; the local texture sub-memory stores gridded local vectors of intermediate-level features to detect regional texture anomalies; and the micro-texture direction sub-memory stores directional statistical vectors of low-level features to perceive subtle directional texture changes. These three layers of memory correspond to the macroscopic structure, local particles, and microscopic direction of stone texture, respectively. The reconstruction deviations at each level are adaptively weighted and summed using learnable fusion weights, enabling the anomaly response heatmap to simultaneously reflect both global structural anomalies and local detail anomalies, providing comprehensive detection sensitivity for stone defects with large scale spans and diverse forms.

[0025] The beneficial effects of this invention are as follows: First, at the information perception level, by simultaneously acquiring three modalities—RGB image, thermal radiation image, and depth image—of the surface of the stone to be inspected, the complementary characteristics of visible light texture information, infrared thermal radiation difference information, and structured light depth geometric information are fully utilized. The physical information provided by the thermal radiation image and the depth image, along with the texture information in the visible light image, are independent yet mutually corroborative in defect characterization, significantly enhancing the ability to distinguish between natural textures and real defects from the information source.

[0026] Second, at the feature fusion level, a cross-modal gating fusion module independently calculates inter-channel attention weights and spatial attention weights for each modality's feature map, and dynamically weights and fuses the feature maps of each modality using these attention weights. This module can autonomously learn the discriminative contribution of different modalities at different spatial locations and feature levels. For example, it automatically suppresses the weight of the RGB modality in regions with complex textures and automatically enhances the contribution of the depth modality in regions with abrupt changes in depth. This avoids information redundancy and noise introduction caused by simple splicing or weighted fusion, resulting in a cross-modal defect representation with stronger discriminative power and robustness.

[0027] Third, at the hierarchical decision-making level, the adaptive hierarchical decision network encodes the multidimensional semantic attributes of defects—type, area proportion, location criticality, and structural impact index—into embedding vectors, maps them to a continuous quality scoring space, and finally converts them into discrete quality level outputs through configurable hierarchical thresholds. This architecture bases hierarchical decisions on a comprehensive evaluation of the multidimensional attributes of defects, rather than a hard threshold determination based on a single indicator, significantly improving the accuracy and rationality of hierarchical classification. Furthermore, because the hierarchical threshold is decoupled from semantic parsing and scoring mapping, when application scenarios or customer standards change, only the hierarchical threshold parameters need to be adjusted without retraining the model, greatly reducing the deployment and maintenance costs of the system. Attached Figure Description

[0028] Figure 1 This is a flowchart of an intelligent detection and real-time grading method for image defects on stone surfaces, according to an embodiment of the present invention. Detailed Implementation

[0029] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0030] Embodiment 1 of the present invention is as follows: This embodiment provides an intelligent detection and real-time grading method for surface defects in stone materials, primarily addressing the limitations of existing technologies where single visible light images struggle to distinguish between natural textures and genuine defects, and where single-index hard threshold grading cannot flexibly adapt to diverse application needs. This embodiment simultaneously acquires RGB images, thermal radiation images, and depth images, utilizing a cross-modal gating fusion module to dynamically weight and fuse multimodal features, generating a highly discriminative cross-modal defect representation. Based on this, a grading decision network comprehensively evaluates the multidimensional semantic attributes of defects, establishing grading decisions based on a comprehensive assessment of defect type, area proportion, location criticality, and structural influence index. Flexible grading output is achieved through configurable grading thresholds. This embodiment is applicable to surface defect detection and grading of common building and decorative stones such as marble, granite, quartz, and artificial stone. For different stone varieties, only training data for the corresponding variety needs to be collected during initial deployment for model adjustment to adapt to different stone types. The technical solution of this embodiment is described in detail below with reference to specific implementation methods.

[0031] Reference Figure 1 As shown in the figure, this embodiment provides an intelligent detection and real-time grading method for image defects on stone surfaces, including the following steps: Step 1: Acquire multimodal images of the surface of the stone to be inspected. The multimodal images include RGB images, thermal radiation images, and depth images. The RGB images, thermal radiation images, and depth images are acquired synchronously in the same field of view through a synchronous triggering imaging system.

[0032] In practice, the synchronous triggering imaging system consists of a high-resolution industrial RGB camera, an infrared thermal imager, and a structured light depth sensing module. The optical axes of these three components are aligned using a beam splitter or mechanical support, and a hardware trigger signal ensures that the synchronization error of the three images in terms of timestamps does not exceed 1 millisecond. After preprocessing, the acquired multimodal images are: the RGB image is a three-channel visible light image with a resolution of 2048×1024 pixels; the thermal radiation image is a single-channel temperature distribution image with a resolution of 512×256 pixels, upsampled to the same spatial resolution as the RGB image through bilinear interpolation; and the depth image is a single-channel distance information image with a resolution of 1024×768 pixels, converted to the RGB image coordinate system after calibration and registration. The spatial resolution of the three images remains consistent before being input into the subsequent network. Multimodal synchronous acquisition provides a physically complementary data foundation from the information source for the accurate identification of subsequent textures and defects.

[0033] Step 2: Input the RGB image into the first feature extraction network, the thermal radiation image into the second feature extraction network, and the depth image into the third feature extraction network. The three feature extraction networks extract the hierarchical features of each modality. The hierarchical features include multi-scale feature maps from shallow to deep layers. Feature maps with the same spatial resolution in different modalities belong to the same feature level. Multiple feature levels form a hierarchical sequence according to resolution from high to low.

[0034] The first feature extraction network adopts the ConvNeXt architecture. ConvNeXt effectively captures the macroscopic direction and long-range dependencies of stone texture through 7×7 depthwise separable large convolutional kernels. Its inverse bottleneck structure and hierarchical design enable the network to output multi-scale feature maps with progressively decreasing spatial resolution and progressively increasing semantic abstraction levels at different depth stages. The second feature extraction network adopts the EfficientNet architecture. EfficientNet extracts global temperature distribution and local thermal anomaly features from thermal radiation images while maintaining a low number of parameters through a composite scaling strategy. Its multiple sets of MBConvBlocks output hierarchical features in stages. The third feature extraction network adopts a PointNet variant architecture. The PointNet variant processes the 3D point cloud data corresponding to the depth image through symmetric functions and local feature aggregation operations to extract the spatial distribution features of surface geometry. All three networks are initialized with weights pre-trained on the ImageNet dataset and jointly adjusted on the stone multimodal dataset to ensure that the hierarchical features output by each modality feature extraction network are consistent in channel dimension and spatial resolution, enabling the cross-modal gating fusion module to perform feature alignment. In practice, all three networks output four feature levels with spatial resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively. Within the same level, the RGB feature map, thermal radiation feature map, and depth feature map have the same spatial resolution and a uniform number of channels, providing directly operable inputs for subsequent layer-by-layer fusion.

[0035] Step 3: Through the cross-modal gating fusion module, for each feature level, the inter-channel attention weight and spatial attention weight of each modal feature map are calculated. The inter-channel attention weight and spatial attention weight are used to dynamically weight and fuse the RGB feature map, thermal radiation feature map and depth feature map of the same feature level to generate the cross-modal fusion feature of that feature level. The cross-modal fusion features of each feature level are then aggregated through skip connections to form a unified cross-modal defect representation.

[0036] The cross-modal gated fusion module comprises four gated fusion units, each corresponding to a feature level. (Based on resolution...) Figure 1Taking the intermediate feature layer of / 8 as an example, the input RGB feature map, thermal radiation feature map, and depth feature map of this layer are all 256×128 pixels in size and have 256 channels. For each modality feature map of this layer, the feature map of each channel is first compressed into a scalar through global average pooling. Then, a channel gating vector is generated through two fully connected layers and an activation function. This vector has a length of 256, where the value range of each element is [0,1], representing the importance of the corresponding channel in the current task. At the same time, each modality feature map is input into a spatial attention branch consisting of a 1×1 convolution and a sigmoid activation, generating a spatial gating matrix with the same spatial size as the input feature map. The value range of each element in the matrix is ​​[0,1], representing the importance weight of the corresponding spatial location. The channel gating vector is multiplied channel-by-channel with the corresponding modality feature map, and the spatial gating matrix is ​​multiplied spatially with the corresponding modality feature map, resulting in each modality feature map after double weighting. Each weighted feature map is processed by the ReLU activation function and then concatenated along the channel dimension to obtain a concatenated feature map with 768 channels. A 1×1 convolutional layer then compresses the number of channels back to 256, outputting the cross-modal fusion feature for this feature level. Four gated fusion units process the four feature levels respectively, generating four cross-modal fusion features with spatial resolutions from high to low. Through skip connections, the low-resolution fusion features are progressively upsampled and concatenated with the high-resolution fusion features along the channel dimension. After convolutional fusion, the final unified cross-modal defect representation is formed, which simultaneously preserves shallow detail texture information and deep semantic structure information.

[0037] Step four: Based on the cross-modal defect characterization, a quality grading result for the stone surface is generated through a hierarchical decision network. The hierarchical decision network includes a multi-task semantic parsing branch and a scoring network branch. The semantic parsing branch is used to predict four semantic attributes: defect type, defect area ratio, defect location criticality, and structural influence index, and concatenate them into an embedding vector. The scoring network branch maps the embedding vector to a continuous quality scoring space and converts the continuous score into a discrete quality level output through a configurable grading threshold.

[0038] The multi-task semantic parsing branch takes a unified cross-modal defect representation as input and connects four task heads in parallel. The defect classification head consists of two fully connected layers and a Softmax activation layer, with an output dimension of 5, corresponding to the probability distributions of five defect types: cracks, stains, holes, scratches, and edge defects. The category corresponding to the highest probability value is taken as the predicted defect type. The area regression head consists of two fully connected layers and a ReLU activation layer, outputting a scalar value representing the ratio of the defect area to the total board area. The mean squared error loss function is used during training. The location criticality evaluation head consists of two fully connected layers and a Sigmoid activation layer, outputting a scalar value in the range [0,1], representing the degree of influence of the defect location on the usability of the board. The training of this head uses manually labeled criticality scores as the supervision signal. The scoring rule is: the closer the defect is to the edge of the board and the more significant the stress characteristics of the area, the higher the criticality score. The structural impact index assessment head receives defect type information and depth features as auxiliary inputs and outputs a scalar value in the range [0,1], representing the degree of impact of the defect on the structural integrity of the plate. The structural impact index of through-cracks is higher than that of shallow surface cracks. Since the probability distribution output by the defect classification head is already in the range [0,1], the area ratio output by the area regression head is already in the range [0,1], and the outputs of the sigmoid activation layer of the location criticality assessment head and the structural impact index assessment head are already in the range [0,1], all four outputs are in the range [0,1], and can therefore be directly concatenated into a fixed-dimensional defect semantic embedding vector.

[0039] The scoring network branch consists of two fully connected layers and one sigmoid activation layer. The input is a defect semantic embedding vector, and the output is a continuous quality score between 0 and 1. A score closer to 1 indicates higher board quality. Configurable grading thresholds include three cutoff values, defaulted to 0.9, 0.7, and 0.5, which are received via a user interface. The continuous quality score is compared to the cutoff values: a score greater than or equal to 0.9 is grade A, a score greater than or equal to 0.7 and less than 0.9 is grade B, a score greater than or equal to 0.5 and less than 0.7 is grade C, and a score less than 0.5 is grade D or scrap. When customer standards or industry specifications change, quality inspectors can modify the three cutoff values ​​via the user interface. The system can then re-determine the quality level based on the updated grading thresholds, while the network parameters of the multi-task semantic parsing branch and the scoring network branch remain unchanged, requiring no retraining.

[0040] Through steps one through four above, this embodiment realizes a complete technical process from multimodal image acquisition, multi-scale feature extraction and dynamic fusion, to multidimensional semantic parsing and configurable hierarchical decision-making, effectively improving the accuracy and flexibility of stone defect detection in complex texture backgrounds.

[0041] Embodiment 2 of the present invention is as follows: This embodiment, building upon Embodiment 1, further addresses the challenges of scarce real defect samples, high annotation costs, and the difficulty in continuously adapting deployed models to new varieties and defect types in the field of stone defect detection. In stone processing production lines, approximately 80% to 90% of the slabs are qualified products. Real defect samples are not only limited in number but also exhibit extremely uneven distribution of defect types, with some rare defect types having zero samples, severely restricting the training effectiveness of supervised deep learning models. Furthermore, when new varieties of stone are introduced to the production line or new defects appear, the performance of deployed models gradually degrades. Traditional solutions require downtime to re-collect and re-train data, resulting in high maintenance costs. This embodiment constructs a large-scale, diverse training dataset by introducing a diffusion model to output synthetic defect images. The model is deployed in an edge-cloud collaborative architecture, utilizing uncertain samples from edge feedback combined with continuously generated synthetic data from the cloud to incrementally adjust the model, achieving continuous evolution of model capabilities. The technical solution of this embodiment is described in detail below with reference to specific implementation methods.

[0042] Reference Figure 1 As shown, this embodiment further optimizes the training process and system deployment method of the feature extraction network and cross-modal gating fusion module based on steps one to four of embodiment one.

[0043] The training process of the first feature extraction network, the second feature extraction network, the third feature extraction network, and the cross-modal gating fusion module includes a pre-training stage. In the pre-training stage, a diffusion model is used to output synthetic defect images to construct a synthetic training set. During the generation process, the diffusion model performs decoupling control of background texture, defect morphology, and lighting conditions. Using a normal stone texture image as a condition, texture features are injected into the diffusion denoising process through a cross-attention mechanism. A specified type of defect morphology is generated at a specified location, and different lighting environments are simulated, thereby obtaining a multimodal synthetic defect image covering the combination space of texture background, defect type, and lighting conditions. The multimodal synthetic defect image is used to pre-train each feature extraction network and the cross-modal gating fusion module.

[0044] In practice, the diffusion model employs an improved Stable Diffusion architecture, performing denoising operations in the latent space to enhance generation efficiency. Decoupling control is achieved through the following methods: the background texture encoder extracts texture feature maps from normal stone texture images, the defect morphology encoder encodes defect type and morphological parameters into conditional vectors, and the illumination conditional encoder encodes light source direction, intensity, and color temperature into conditional vectors. At each time step of the diffusion denoising process, the texture feature maps and defect morphology conditional vectors are injected into the feature layers of the U-Net denoising network through a cross-attention mechanism, allowing the generated defect morphology to naturally blend with the background texture in space, rather than simply being superimposed. The illumination conditional vectors control the global illumination style of the generated images through an adaptive instance normalization layer. By controlling the combination parameters of defect type labels (cracks, blemishes, holes, scratches, edge defects), texture background sources (marble, granite, artificial stone, etc.), and illumination conditions (front lighting, side lighting, diffuse lighting, etc.), a synthetic training set covering hundreds of combined scenes can be generated, with at least 500 images generated for each combination. While the diffusion model outputs a synthetic RGB image, a pre-trained monocular depth estimation model infers depth information from the synthetic RGB image to generate a corresponding depth image. A thermal radiation simulation model based on material thermal conductivity parameters and defect geometry generates a corresponding thermal radiation image, ensuring that the three-modal synthetic images maintain spatial and semantic correspondence. Using these multimodal synthetic defect images, the three feature extraction networks and the cross-modal gating fusion module are pre-trained for 50 to 100 epochs, allowing the network parameters to converge to an initial state with basic perceptual ability for multimodal defect patterns. After pre-training, only a small number of real-world labeled samples are needed for 10 to 20 epochs of fine-tuning to achieve practical detection accuracy, significantly shortening the deployment cycle for new types of stone.

[0045] The method is deployed in an edge-cloud collaborative architecture. At the edge, a lightweight model that has undergone knowledge distillation and integer quantization is deployed to perform real-time inference and classification in steps one to four. At the cloud, the diffusion model is deployed and continuously uses newly added normal stone texture images to output synthetic defect samples. During the inference process, the edge performs confidence evaluation on each frame of image. Samples with defect classification confidence scores below a first threshold and above a second threshold are marked as uncertain samples and uploaded to the cloud. The cloud uses the synthetic defect samples in combination with the uncertain samples to incrementally adjust the model and sends the adjusted model parameters to the edge via OTA to achieve continuous evolution of model capabilities.

[0046] In practical implementation, the edge is deployed in the production line's industrial control computer, equipped with an NVIDIA Jetson Orin computing module with a computing power of 275 TOPS. The lightweight model uses the feature extraction network and hierarchical decision network trained in Example 1 as the teacher model, and compresses them using knowledge distillation technology. The number of parameters is about 1 / 8 of the teacher model, or about 5M parameters. The teacher model is about 160MB in size when stored with FP32 precision. The student model is about 20MB in size with FP32 precision. After INT8 integer quantization, the weight parameters are compressed from 4 bytes to 1 byte, and the quantized weight part is about 5MB in size. Adding the statistical parameters of the BatchNormalization layer and the model structure metadata, the final model size is about 25MB. The single-frame inference latency is no more than 30 milliseconds, which meets the real-time requirement of the production line to detect 3 to 5 boards per second. The cloud is deployed on a server cluster equipped with NVIDIA A100 GPUs to run the diffusion model and incremental training tasks. New normal stone texture images added daily from the production line are automatically uploaded to the cloud. The diffusion model uses these new texture images as conditions to continuously output diverse synthetic defect samples, maintaining consistency between the synthetic data and the texture distribution of the current production batch. The specific strategy for edge-end confidence assessment is as follows: the maximum category probability value output by the defect classification head in the hierarchical decision network is used as the confidence level, with the first threshold set to 0.85 and the second threshold set to 0.5; samples with maximum category probability values ​​between 0.5 and 0.85 are classified as uncertain samples, and the multimodal image data corresponding to these uncertain samples are compared with the abnormal response heatmap generated by edge-end inference. Figure 1 The data is then packaged and uploaded to the cloud for targeted incremental adjustments to the model, enhancing its ability to identify defects with blurred boundaries. The cloud performs incremental adjustments every preset period, using uncertain samples as difficult examples and combining them with newly generated synthetic defect samples. Based on the existing model weights, incremental fine-tuning is performed over 5 to 10 epochs. The adjusted model parameters are then wirelessly deployed to the edge processing unit via OTA (Over-The-Air). The edge processing unit updates the model during the interval between inspections of the next sheet material, requiring no manual intervention or production line downtime throughout the entire process.

[0047] Through the aforementioned pre-training strategy and edge-cloud co-evolutionary architecture, this embodiment effectively solves the problems of scarce training data and continuous model adaptability in stone defect detection, enabling the system to become more accurate with use in actual operation and significantly reducing long-term maintenance costs.

[0048] Embodiment 3 of the present invention is as follows: This embodiment provides an alternative implementation to step four of Embodiment 1, primarily addressing the problem of the inexhaustible number of defect types in stone production lines and the excessively high cost of collecting and labeling a large number of defect samples when frequently switching to new stone varieties. In the stone processing industry, the texture characteristics of different mineral sources vary significantly, and new defects may arise with the emergence of new mining batches. Requiring sufficient training samples in advance for each new stone variety and each possible defect type is difficult to achieve in practice. The solution in Embodiment 1 achieves optimal performance when sufficient labeled data is available, but requires a certain number of various defect samples to support the training of multi-task semantic parsing branches. This embodiment introduces an unsupervised comparative learning mechanism based on a cross-modal fusion feature memory library of normal samples. It only requires collecting normal, defect-free stone samples to complete the deployment of the quality grading model, making it particularly suitable for production line scenarios where defect samples are scarce but normal samples are readily available. The technical solution of this embodiment will be described in detail below with reference to specific implementation methods.

[0049] Please refer to Figure 1 As shown, in this embodiment, based on steps one to three of Embodiment One, the step of generating quality grading results based on the cross-modal defect characterization in step four is implemented in the following way: The multimodal images of multiple normal, defect-free stone samples are pre-acquired. Following steps one to three, cross-modal defect representations of the normal samples are extracted. These cross-modal defect representations are used as cross-modal fusion baseline features to construct a cross-modal fusion feature memory for normal samples. During the detection phase, the cross-modal defect representations obtained from the stone images to be detected through steps one to three are used in the normal sample cross-modal fusion feature memory for K-nearest neighbor retrieval. The reconstruction deviation at each feature level is calculated, and the reconstruction deviations at each level are weighted and fused to generate an anomaly response heatmap. The hierarchical decision network uses the anomaly area ratio, response intensity peak, and spatial distribution entropy in the anomaly response heatmap as at least a part of the defect semantic attributes, inputting them into the scoring network branch to generate a hierarchical result.

[0050] In practice, the construction process of the normal sample cross-modal fusion feature memory library is as follows: Before production line goes into operation, at least 200 multimodal images of normal, defect-free stone slabs are collected, covering the common texture variation range of this type of stone. These 200 normal samples are processed one by one through steps one through three. The output of step three is uniformly referred to as cross-modal defect characterization in this invention. Since the input samples are normal, defect-free samples, this characterization essentially depicts the multimodal fusion feature patterns of normal stone. The feature representations of these normal patterns are used as the cross-modal fusion benchmark features. The cross-modal defect characterizations of these 200 sets of normal samples are organized and stored to form the normal sample cross-modal fusion feature memory library, which serves as the benchmark for subsequent anomaly judgment. The construction of the memory library is completed offline in one go. When switching to a new type of stone, only normal samples of the new type need to be collected to rebuild the memory library, without the need for the collection and annotation of any defect samples throughout the process.

[0051] The normal sample cross-modal fusion feature memory library comprises three levels: a global structure sub-memory library, a local texture sub-memory library, and a micro-texture direction sub-memory library. The global structure sub-memory library stores the global average pooling vector of the high-level cross-modal fusion features of each sample. The local texture sub-memory library stores the local feature vectors of the intermediate-level cross-modal fusion features after gridding. The micro-texture direction sub-memory library stores the directional statistical feature vectors of the low-level cross-modal fusion features after directional filtering.

[0052] In practice, the construction of the three sub-memories corresponds to the four feature levels output in step three. The highest level (with spatial resolution equal to the original) is taken. Figure 1 The cross-modal fusion features of / 32 are processed by global average pooling to obtain a 512-dimensional global feature vector. The global feature vectors of all normal samples constitute a global structure sub-memory, which captures the overall texture layout and macroscopic structural information of the board material. The intermediate level (spatial resolution is the original) is then selected. Figure 1 The cross-modal fusion features of ( / 16) are divided into 16 4×4 grids. The features within each grid are subjected to local average pooling to obtain a 256-dimensional local feature vector. The local feature vectors of all grids for all normal samples constitute a local texture sub-memory, which captures regional texture particles and local anomaly information. The lowest level (spatial resolution equal to the original) is selected. Figure 1The cross-modal fusion features of / 4) employ a Gabor filter bank in four directions to extract texture responses in the 0°, 45°, 90°, and 135° directions, respectively. Each direction response is averaged and pooled to obtain a 128-dimensional directional feature vector. The directional feature vectors of all normal samples in all directions constitute a micro-texture direction sub-memory, which captures information on subtle texture orientation and directional changes. The three sub-memories characterize the multimodal feature distribution of normal stone from three granular levels: macrostructure, local particles, and micro-direction.

[0053] During the detection phase, the cross-modal defect characterization obtained from steps one to three of the stone image to be detected is used to extract global feature vectors, local feature vectors, and directional feature vectors according to the aforementioned rules. In the global structure sub-memory, the global feature vector is searched using the FAISS efficient vector retrieval library, with K set to 5. The average Euclidean distance between the global feature vector and its five nearest normal sample global feature vectors is calculated as the global structure reconstruction bias. Similarly, in the local texture sub-memory, the local feature vectors of each grid are retrieved and the local texture reconstruction bias is calculated. In the micro-texture direction sub-memory, the directional feature vectors of each direction are retrieved and the directional feature reconstruction bias is calculated. The reconstruction biases at each feature level include the global structure bias retrieved from the global structure sub-memory, the local texture bias retrieved from the local texture sub-memory, and the directional feature bias retrieved from the micro-texture direction sub-memory. These three biases are weighted and summed using learnable fusion weights to generate the abnormal response heatmap. The initial values ​​of the three fusion weights are all set to 1 / 3. When some labeled data is available, the fusion weights can be optimized and adjusted through backpropagation to further improve detection accuracy. When no labeled data is available, the initial value of 1 / 3 can be used directly. The spatial resolution of the anomaly response heatmap is consistent with that of the input image, and the value of each pixel in the heatmap represents the degree to which that location deviates from the normal pattern.

[0054] The hierarchical decision network takes the anomaly response heatmap as input and calculates the percentage of anomaly area (the proportion of pixel area with heat values ​​exceeding a set threshold to the total area, where the preset threshold can be 50% of the maximum value of the heatmap or set according to actual detection sensitivity requirements), peak response intensity (the maximum heat value in the heatmap), and spatial distribution entropy (reflecting the degree of clustering of anomaly areas; clustered defects have a greater impact on the usability of the board material than dispersed anomalies). These three statistics are input as semantic attributes of the defects into the scoring network branch. The scoring network branch maps these semantic attributes to a continuous quality scoring space and converts the continuous scores into discrete quality level outputs through a configurable hierarchical threshold.

[0055] By using the above-described method of comparison based on a normal sample memory bank, this embodiment only needs to collect normal, defect-free stone samples to complete the construction of quality grading capabilities. During the detection phase, any feature that deviates from the normal pattern will generate an abnormal response. Therefore, it naturally has the ability to detect new defects that have not appeared in the training set, effectively solving the data preparation problem caused by frequent changes in stone varieties and the inexhaustible number of defect types.

[0056] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for intelligent detection and real-time grading of image defects on stone surfaces, characterized in that, Including the following steps: Step 1: Acquire multimodal images of the surface of the stone to be inspected. The multimodal images include RGB images, thermal radiation images, and depth images, which are acquired simultaneously in the same field of view. Step 2: Input the RGB image into the first feature extraction network, the thermal radiation image into the second feature extraction network, and the depth image into the third feature extraction network. The three feature extraction networks extract the hierarchical features of each modality. The hierarchical features include multi-scale feature maps from shallow to deep layers. Feature maps with the same spatial resolution in different modalities belong to the same feature level. Multiple feature levels form a hierarchical sequence according to resolution from high to low. Step 3: Through the cross-modal gating fusion module, for each feature level, the inter-channel attention weight and spatial attention weight of each modal feature map are calculated respectively. The inter-channel attention weight and spatial attention weight are used to dynamically weight and fuse the RGB feature map, thermal radiation feature map and depth feature map of the same feature level to generate the cross-modal fusion feature of that feature level. The cross-modal fusion features of each feature level are aggregated through skip connections to form a unified cross-modal defect representation. Step four: Based on the cross-modal defect characterization, a quality grading result for the stone surface is generated through a hierarchical decision network. The hierarchical decision network includes a multi-task semantic parsing branch and a scoring network branch. The semantic parsing branch is used to predict four semantic attributes: defect type, defect area ratio, defect location criticality, and structural influence index, and concatenate them into an embedding vector. The scoring network branch maps the embedding vector to a continuous quality scoring space and converts the continuous score into a discrete quality level output through a configurable grading threshold.

2. The intelligent detection and real-time grading method for image defects on stone surfaces according to claim 1, characterized in that, In step two, the first feature extraction network adopts the ConvNeXt architecture, the second feature extraction network adopts the EfficientNet architecture, and the third feature extraction network adopts the PointNet variant architecture. Each feature extraction network is initialized with weights pre-trained on the ImageNet dataset and jointly adjusted on the stone multimodal dataset to ensure that the hierarchical features output by each modal feature extraction network are consistent in channel dimension and spatial resolution, and to enable the cross-modal gating fusion module to perform feature alignment.

3. The intelligent detection and real-time grading method for image defects on stone surfaces according to claim 1, characterized in that, In step three, the cross-modal gated fusion module includes multiple gated fusion units, each corresponding to a feature level; for The RGB feature map, thermal radiation feature map, and depth feature map of the current feature level are used to generate channel gating vectors for each modality through global average pooling and fully connected layers, and spatial gating matrices for each modality are generated through convolutional layers. The channel gating vectors and spatial gating matrices are multiplied by the feature maps of the corresponding modalities through channel-wise multiplication and spatial position-wise multiplication, respectively, to obtain weighted modal feature maps. The weighted modal feature maps are concatenated along the channel dimension and fused through convolutional layers to output the cross-modal fusion feature of this feature level.

4. The intelligent detection and real-time grading method for image defects on stone surfaces according to claim 1, characterized in that, In step four, the multi-task semantic parsing branch takes the cross-modal defect representation as input and connects in parallel the defect classification head, area regression head, location criticality assessment head, and structural influence index assessment head. The defect classification head outputs the probability distribution of five defect types: cracks, spots, holes, scratches, and edge defects. The area regression head outputs the ratio of the defect area to the total board area. The location criticality assessment head outputs a criticality score based on the distance of the defect from the edge of the board and the stress characteristics of the area. The structural influence index assessment head outputs a structural integrity influence index based on the defect type and depth information. The outputs of the defect classification head, area regression head, location criticality assessment head, and structural influence index assessment head are normalized and then concatenated into a fixed-dimensional defect semantic embedding vector.

5. The intelligent detection and real-time grading method for image defects on stone surfaces according to claim 4, characterized in that, The scoring network branch specifically consists of two fully connected layers and a Sigmoid activation layer, used to map the defect semantic embedding vector to a continuous quality score between 0 and 1; the configurable grading threshold includes three boundary values, and the grading standard parameters are received through the user interface, the continuous quality score is compared with the boundary values, and the grading result of A, B, C, D or scrap is output.

6. The intelligent detection and real-time grading method for image defects on stone surfaces according to claim 1, characterized in that, The training process of the first feature extraction network, the second feature extraction network, the third feature extraction network, and the cross-modal gating fusion module includes a pre-training stage. In the pre-training stage, a diffusion model is used to output synthetic defect images to construct a synthetic training set. During the generation process, the diffusion model performs decoupling control of background texture, defect morphology, and lighting conditions. Using a normal stone texture image as a condition, texture features are injected into the diffusion denoising process through a cross-attention mechanism. A specified type of defect morphology is generated at a specified location, and different lighting environments are simulated to obtain a multimodal synthetic defect image covering the combination space of texture background, defect type, and lighting conditions. The multimodal synthetic defect image is used to pre-train each feature extraction network and the cross-modal gating fusion module.

7. The intelligent detection and real-time grading method for image defects on stone surfaces according to claim 6, characterized in that, The method is deployed in an edge-cloud collaborative architecture. At the edge, a lightweight model that has undergone knowledge distillation and integer quantization is deployed to perform real-time inference and classification in steps one to four. At the cloud, the diffusion model is deployed and continuously uses newly added normal stone texture images to output synthetic defect samples. During the inference process, the edge performs confidence assessment on each frame of image. Samples with defect classification confidence scores below the first threshold and above the second threshold are marked as uncertain samples and uploaded to the cloud. The cloud uses synthetic defect samples in combination with the uncertain samples to incrementally adjust the model and sends the adjusted model parameters to the edge via OTA.

8. The intelligent detection and real-time grading method for image defects on stone surfaces according to claim 7, characterized in that, The confidence assessment step performed at the edge is as follows: the maximum category probability value output by the defect classification head in the hierarchical decision network is obtained as the confidence value, the first threshold is set to 0.85, and the second threshold is set to 0.5; samples with the maximum category probability value between 0.5 and 0.85 are judged as uncertain samples; the multimodal image data corresponding to the uncertain samples and the abnormal response heatmap generated by the edge inference are packaged together and uploaded to the cloud for the cloud to make targeted incremental adjustments to the model in order to improve the model's ability to distinguish defects with blurred boundaries.

9. The intelligent detection and real-time grading method for image defects on stone surfaces according to claim 1, characterized in that, The step of generating quality grading results based on the cross-modal defect characterization in step four is implemented in the following way: pre-collect the multimodal images of multiple normal and defect-free stone samples, extract the cross-modal defect characterization of normal samples according to the processing flow of steps one to three, use the cross-modal defect characterization corresponding to the normal samples as the cross-modal fusion benchmark features, and construct a cross-modal fusion feature memory library for normal samples. During the detection phase, the cross-modal defect characterization obtained from the processing of the stone image to be detected through steps one to three is used to perform K-nearest neighbor retrieval in the cross-modal fusion feature memory of normal samples. The reconstruction deviation of each feature level is calculated, and the reconstruction deviation of each level is weighted and fused to generate an abnormal response heatmap. The hierarchical decision network takes the abnormal area ratio, response intensity peak value, and spatial distribution entropy in the abnormal response heatmap as at least part of the defect semantic attributes and inputs them into the scoring network branch to generate a hierarchical result.

10. The intelligent detection and real-time grading method for image defects on stone surfaces according to claim 9, characterized in that, The normal sample cross-modal fusion feature memory comprises three levels: a global structure sub-memory, a local texture sub-memory, and a micro-texture direction sub-memory. The global structure sub-memory stores the global average pooling vector of the high-level cross-modal fusion features of each sample. The local texture sub-memory stores the local feature vectors of the intermediate-level cross-modal fusion features after meshing. The micro-texture direction sub-memory stores the directional statistical feature vectors of the low-level cross-modal fusion features after directional filtering. The reconstruction bias of each feature level includes the global structure bias retrieved from the global structure sub-memory, the local texture bias retrieved from the local texture sub-memory, and the directional feature bias retrieved from the micro-texture direction sub-memory. These three biases are weighted and summed using learnable fusion weights to generate the abnormal response heatmap.

Images